The Case for Enhancing the Quality of Primary Evaluations through Metaevaluation in Developing Country Contexts
DOI:
https://doi.org/10.25159/2663-6522/5708Keywords:
Metaevaluation, Evaluation, Constructivist Evaluation, Evaluative InquiryAbstract
This article makes a case for metaevaluation in the context of further evaluative inquiry beyond primary evaluation level and advances the plausible perspective that metaevaluation improves the quality of primary evaluations and the evidence used to support decision-making. It argues that metaevaluation is a form of evaluative enquiry that is aimed at assessing the strengths and weaknesses of primary evaluations. Through so doing, metaevaluation makes a valuable contribution toward enhancing the credibility and reliability of primary evaluations and the evidence provided to decision-makers. Metaevaluation has been utilised in different contexts in some developed countries to assess the quality of evaluations of development interventions, although scanty evidence exists of its use in developing country contexts.
References
Alkin, M. C. 1969. “Evaluating the Cost-effectiveness of Instructional Programs. Symposium presented at the Center for the Study of Evaluation.” Report No. 25, University of California, Los Angeles. https://files.eric.ed.gov/fulltext/ED031818.pdf. https://doi.org/10.1037/e684812011-001.
Alkin, M. C. (Ed.). 2012. Evaluation Roots: A Wider Perspective of Theorists’ Views and Influences. Thousand Oaks, CA: Sage Publications.
Apthorpe, R., and D. Gasper. 1982. “Policy Evaluation and Meta-evaluation: The Case of Rural Co-operatives.” World Development 10 (8): 651–668. https://doi.org/10.1016/0305-750X(82)90091-2.
Baxter, P., and S. Jack. 2008. “Qualitative Case Study Methodology: Study Design and Implementation for Novice Researchers.” The Qualitative Report 13 (4): 544–559.
Borich, G. D. 1990. Decision Oriented Evaluation. Oxford, England: Pergamon Press.
Bollen, K., P. Paxton, and R. Morishima. 2005. “Assessing International Evaluations: An Example from USAID’s Democracy and Governance Program.” American Journal of Evaluation 26 (2): 189–203. https://doi.org/10.1177/1098214005275640.
Caracelli, V. J, and L. J. Cooksy. 2008. “Meta-evaluation in Practice: Selection and Application of Criteria.” Journal of Multidisciplinary Evaluation 6 (11): 1–15.
Cooksy, L. J., and V. J. Caracelli. 2005. “Quality, Context, and Use: Issues in Achieving the Goals of Meta-evaluation.” American Journal of Evaluation 26 (1): 31–42. https://doi.org/10.1177/1098214004273252.
Cronbach, L. J. 2000. “Course Improvement through Evaluation.” In Evaluation Models: Viewpoints on Educational and Human Services, 2nd edition, 235–247, edited by D. L. Stufflebeam, G. F. Madaus, and T. Kellaghan. Hingham, MA: Kluwer. https://doi.org/10.1007/0-306-47559-6_14.
Cronbach, L. J., S. R. Ambron, S. M. Dornbusch, R. D. Hess, R. C. Hornik, D. C. Phillips, and S. S. Weiner. 1980. Toward Reform of Program Evaluation, 3. San Francisco: Jossey-Bass.
Fitzpatrick, J. L., J. R. Sanders, and B. R. Worthen. 2004. Program Evaluation: Alternative Approaches and Practical Guidelines. Boston, MA: Pearson Publishing.
Glass, G. V. 1969. “The Growth of Evaluation Methodology.” Research paper no. 27, Boulder, Colo., University of Colorado: Laboratory of Educational Research.
Greene, J. C. 1992. “A Case Study of Evaluation Auditing as Meta-evaluation.” Evaluation and Program Planning 15 (1): 71–74. https://doi.org/10.1016/0149-7189(92)90063-Z.
Guba, E. G., and Y. S. Lincoln. 1989. Fourth Generation Evaluation. Thousand Oaks, CA: Sage Publications.
Guba, E. G., and Y. S. Lincoln. 2001. Guidelines and Checklist for Constructivist (aka Fourth Generation) Evaluation. Beverly Hills, CA: Sage Publications.
Hanssen, C. E., F. Lawrenz, and D. O. Dunet. 2008. “Concurrent Meta-evaluation: A Critique.” American Journal of Evaluation 29 (4): 572–582. https://doi.org/10.1177/1098214008320462.
Hedler, H. C., and N. Gibram. 2009. “The Contribution of Metaevaluation to Program Evaluation: Proposition of a Model.” Journal of MultiDisciplinary Evaluation 6 (12): 210–223.
Nevo, D. 1983. “The Conceptualization of Educational Evaluation: An Analytical Review of the Literature.” Review of Educational Research 53 (1): 117–128. doi:10.3102/00346543053001117/.
Odom, S. L., and R. R. Fewell. 1983. “Program Evaluation in Early Childhood Special Education: A Meta-evaluation.” Educational Evaluation and Policy Analysis 5 (4): 445–460. https://doi.org/10.3102/01623737005004445.
OECD. 2004. The Organisation for Economic Co-operation and Development (OECD). Paris: Routledge Publishing.
Orr, R. R., S. J. Cameron, and D. M. Day. 1991. “Coping with Stress in Families with Children who have Mental Retardation: An Evaluation of the Double ABCX Model”. American Journal on Mental Retardation 95 (4): 444–450.
Owen, J. M., and P. Rogers. 1999. Programme Evaluation: Forms and Approaches. Thousand Oaks, CA: Sage Publications.
Patton, M. Q. 1997. “Toward Distinguishing Empowerment Evaluation and Placing it in a larger Context.” Evaluation Practice 18 (2): 147–163.
Pierre, R. G. 1983. Reanalyzing Programme Evaluation. Basel: Jossey-Bass Publishers.
Rossi, P. H., H. E. Freeman, and M. W. Lipsey. 1999. Evaluation. A systematic Approach. Thousand Oaks, CA: Sage Publications.
Scriven, M. 1966. Primary Philosophy. Thousand Oaks, CA: Sage Publications.
Scriven, M. 1969. An Introduction to Metaevaluation. Educ Prod Rep 2 (4): 36–38.
Scriven, M. 2007. Key Evaluation Checklist (KEC). Thousand Oaks, CA: Sage Publications.
Shadish, W. R., T. D. Cook, and L. C. Leviton. 1991. Foundations of Program Evaluation: Theories of Practice. Thousand Oaks, CA: Sage Publications.
Shinkfield, A. D., and L. Stufflebeam. 2007. Evaluation Theory, Models and Applications. San Francisco: CA: John Wiley. Thousand Oaks, CA: Sage Publications.
Stige, B., K. Malterud, and T. Midtgarden. 2009. “Toward an Agenda for Evaluation of Qualitative Research.” Qualitative Health Research 19 (10): 1504–1516. https://doi.org/10.1177/1049732309348501.
Stufflebeam, D. L. 1971. The Relevance of the CIPP Evaluation Model for Educational Accountability. Abingdon, London: Routledge Publishing.
Stufflebeam, D. L. 1974. Meta-evaluation. MI: Western Michigan University Evaluation Centre.
Stufflebeam, D. L. 2000. Foundational Models for 21st Century Program Evaluation. Dordrecht, Netherlands: Springer.
Stufflebeam, D. L. 2001. “The Meta-evaluation Imperative.” American Journal of Evaluation 22 (2): 183–209. https://doi.org/10.1177/109821400102200204; https://doi.org/10.1016/S1098-2140(01)00127-8.
Stufflebeam, D. L., and W. J. Webster. 1983. “An Analysis of Alternative Approaches to Evaluation.” In Evaluation Models, 23–43, edited by D. L. Stufflebeam. Dordrecht, Netherlands: Springer. https://doi.org/10.1007/978-94-009-6669-7_2.
The Joint Committee on Standards for Educational Evaluation. 1994. The Programme Evaluation Standards, 2nd edition. Thousand Oaks, CA: Sage Publications.
Tyler, R. W. 1942. “General Statement on Evaluation.” The Journal of Educational Research 35 (7): 492–501. https://doi.org/10.1080/00220671.1942.10881106.
Tyler, R. W. 1950. “Basic Principles of Curriculum and Instruction: Syllabus for Education 360.” IL: Chicago.
Uusikylä, P., and P. Virtanen. 2000. “Meta-evaluation as a Tool for Learning: A Case Study of the European Structural Fund Evaluations in Finland.” Evaluation 6 (1): 50–65. https://doi.org/10.1177/13563890022209118.
Weiss, C. H. 1998. Evaluation. Upper Saddle, NJ: Prentice Hall Publishing.
Wholey, J. S., H. P. Hatry, and K. E. Newcomer (Eds). 2004. Handbook of Practical Programme Evaluation. San Francisco, CA Jossey-Bass Publishers.
Wingate, L. A., C. L. S. Coryn, A. R. Gullickson, and L. J. Cooksy. 2010. The Program Evaluation Standards Applied for Meta-evaluation Purposes: Investigating Interrater Agreement and its Implications for Use.” Manuscript submitted for publication.
Wittrock, M. C., and D. E. Wiley. 1970. The Evaluation of Instruction: Issues and Problems. New York, NY: Holt, Rinehart and Winston Publishing.
World Bank. 2010. African Development Indicators 2008. Washington, DC: World Bank Publications.
Alkin, M. C. (Ed.). 2012. Evaluation Roots: A Wider Perspective of Theorists’ Views and Influences. Thousand Oaks, CA: Sage Publications.
Apthorpe, R., and D. Gasper. 1982. “Policy Evaluation and Meta-evaluation: The Case of Rural Co-operatives.” World Development 10 (8): 651–668. https://doi.org/10.1016/0305-750X(82)90091-2.
Baxter, P., and S. Jack. 2008. “Qualitative Case Study Methodology: Study Design and Implementation for Novice Researchers.” The Qualitative Report 13 (4): 544–559.
Borich, G. D. 1990. Decision Oriented Evaluation. Oxford, England: Pergamon Press.
Bollen, K., P. Paxton, and R. Morishima. 2005. “Assessing International Evaluations: An Example from USAID’s Democracy and Governance Program.” American Journal of Evaluation 26 (2): 189–203. https://doi.org/10.1177/1098214005275640.
Caracelli, V. J, and L. J. Cooksy. 2008. “Meta-evaluation in Practice: Selection and Application of Criteria.” Journal of Multidisciplinary Evaluation 6 (11): 1–15.
Cooksy, L. J., and V. J. Caracelli. 2005. “Quality, Context, and Use: Issues in Achieving the Goals of Meta-evaluation.” American Journal of Evaluation 26 (1): 31–42. https://doi.org/10.1177/1098214004273252.
Cronbach, L. J. 2000. “Course Improvement through Evaluation.” In Evaluation Models: Viewpoints on Educational and Human Services, 2nd edition, 235–247, edited by D. L. Stufflebeam, G. F. Madaus, and T. Kellaghan. Hingham, MA: Kluwer. https://doi.org/10.1007/0-306-47559-6_14.
Cronbach, L. J., S. R. Ambron, S. M. Dornbusch, R. D. Hess, R. C. Hornik, D. C. Phillips, and S. S. Weiner. 1980. Toward Reform of Program Evaluation, 3. San Francisco: Jossey-Bass.
Fitzpatrick, J. L., J. R. Sanders, and B. R. Worthen. 2004. Program Evaluation: Alternative Approaches and Practical Guidelines. Boston, MA: Pearson Publishing.
Glass, G. V. 1969. “The Growth of Evaluation Methodology.” Research paper no. 27, Boulder, Colo., University of Colorado: Laboratory of Educational Research.
Greene, J. C. 1992. “A Case Study of Evaluation Auditing as Meta-evaluation.” Evaluation and Program Planning 15 (1): 71–74. https://doi.org/10.1016/0149-7189(92)90063-Z.
Guba, E. G., and Y. S. Lincoln. 1989. Fourth Generation Evaluation. Thousand Oaks, CA: Sage Publications.
Guba, E. G., and Y. S. Lincoln. 2001. Guidelines and Checklist for Constructivist (aka Fourth Generation) Evaluation. Beverly Hills, CA: Sage Publications.
Hanssen, C. E., F. Lawrenz, and D. O. Dunet. 2008. “Concurrent Meta-evaluation: A Critique.” American Journal of Evaluation 29 (4): 572–582. https://doi.org/10.1177/1098214008320462.
Hedler, H. C., and N. Gibram. 2009. “The Contribution of Metaevaluation to Program Evaluation: Proposition of a Model.” Journal of MultiDisciplinary Evaluation 6 (12): 210–223.
Nevo, D. 1983. “The Conceptualization of Educational Evaluation: An Analytical Review of the Literature.” Review of Educational Research 53 (1): 117–128. doi:10.3102/00346543053001117/.
Odom, S. L., and R. R. Fewell. 1983. “Program Evaluation in Early Childhood Special Education: A Meta-evaluation.” Educational Evaluation and Policy Analysis 5 (4): 445–460. https://doi.org/10.3102/01623737005004445.
OECD. 2004. The Organisation for Economic Co-operation and Development (OECD). Paris: Routledge Publishing.
Orr, R. R., S. J. Cameron, and D. M. Day. 1991. “Coping with Stress in Families with Children who have Mental Retardation: An Evaluation of the Double ABCX Model”. American Journal on Mental Retardation 95 (4): 444–450.
Owen, J. M., and P. Rogers. 1999. Programme Evaluation: Forms and Approaches. Thousand Oaks, CA: Sage Publications.
Patton, M. Q. 1997. “Toward Distinguishing Empowerment Evaluation and Placing it in a larger Context.” Evaluation Practice 18 (2): 147–163.
Pierre, R. G. 1983. Reanalyzing Programme Evaluation. Basel: Jossey-Bass Publishers.
Rossi, P. H., H. E. Freeman, and M. W. Lipsey. 1999. Evaluation. A systematic Approach. Thousand Oaks, CA: Sage Publications.
Scriven, M. 1966. Primary Philosophy. Thousand Oaks, CA: Sage Publications.
Scriven, M. 1969. An Introduction to Metaevaluation. Educ Prod Rep 2 (4): 36–38.
Scriven, M. 2007. Key Evaluation Checklist (KEC). Thousand Oaks, CA: Sage Publications.
Shadish, W. R., T. D. Cook, and L. C. Leviton. 1991. Foundations of Program Evaluation: Theories of Practice. Thousand Oaks, CA: Sage Publications.
Shinkfield, A. D., and L. Stufflebeam. 2007. Evaluation Theory, Models and Applications. San Francisco: CA: John Wiley. Thousand Oaks, CA: Sage Publications.
Stige, B., K. Malterud, and T. Midtgarden. 2009. “Toward an Agenda for Evaluation of Qualitative Research.” Qualitative Health Research 19 (10): 1504–1516. https://doi.org/10.1177/1049732309348501.
Stufflebeam, D. L. 1971. The Relevance of the CIPP Evaluation Model for Educational Accountability. Abingdon, London: Routledge Publishing.
Stufflebeam, D. L. 1974. Meta-evaluation. MI: Western Michigan University Evaluation Centre.
Stufflebeam, D. L. 2000. Foundational Models for 21st Century Program Evaluation. Dordrecht, Netherlands: Springer.
Stufflebeam, D. L. 2001. “The Meta-evaluation Imperative.” American Journal of Evaluation 22 (2): 183–209. https://doi.org/10.1177/109821400102200204; https://doi.org/10.1016/S1098-2140(01)00127-8.
Stufflebeam, D. L., and W. J. Webster. 1983. “An Analysis of Alternative Approaches to Evaluation.” In Evaluation Models, 23–43, edited by D. L. Stufflebeam. Dordrecht, Netherlands: Springer. https://doi.org/10.1007/978-94-009-6669-7_2.
The Joint Committee on Standards for Educational Evaluation. 1994. The Programme Evaluation Standards, 2nd edition. Thousand Oaks, CA: Sage Publications.
Tyler, R. W. 1942. “General Statement on Evaluation.” The Journal of Educational Research 35 (7): 492–501. https://doi.org/10.1080/00220671.1942.10881106.
Tyler, R. W. 1950. “Basic Principles of Curriculum and Instruction: Syllabus for Education 360.” IL: Chicago.
Uusikylä, P., and P. Virtanen. 2000. “Meta-evaluation as a Tool for Learning: A Case Study of the European Structural Fund Evaluations in Finland.” Evaluation 6 (1): 50–65. https://doi.org/10.1177/13563890022209118.
Weiss, C. H. 1998. Evaluation. Upper Saddle, NJ: Prentice Hall Publishing.
Wholey, J. S., H. P. Hatry, and K. E. Newcomer (Eds). 2004. Handbook of Practical Programme Evaluation. San Francisco, CA Jossey-Bass Publishers.
Wingate, L. A., C. L. S. Coryn, A. R. Gullickson, and L. J. Cooksy. 2010. The Program Evaluation Standards Applied for Meta-evaluation Purposes: Investigating Interrater Agreement and its Implications for Use.” Manuscript submitted for publication.
Wittrock, M. C., and D. E. Wiley. 1970. The Evaluation of Instruction: Issues and Problems. New York, NY: Holt, Rinehart and Winston Publishing.
World Bank. 2010. African Development Indicators 2008. Washington, DC: World Bank Publications.
Downloads
Published
2022-09-20
How to Cite
Wandwasi, Peter Misigalo. 2022. “The Case for Enhancing the Quality of Primary Evaluations through Metaevaluation in Developing Country Contexts”. Africanus: Journal of Development Studies, September, 14 pages. https://doi.org/10.25159/2663-6522/5708.
Issue
Section
Articles
