The Case for Enhancing the Quality of Primary Evaluations through Metaevaluation in Developing Country Contexts

Peter Misigalo Wandwasi

doi:10.25159/2663-6522/5708

Authors

Peter Misigalo Wandwasi University of Johannesburg

DOI:

https://doi.org/10.25159/2663-6522/5708

Keywords:

Metaevaluation, Evaluation, Constructivist Evaluation, Evaluative Inquiry

Abstract

This article makes a case for metaevaluation in the context of further evaluative inquiry beyond primary evaluation level and advances the plausible perspective that metaevaluation improves the quality of primary evaluations and the evidence used to support decision-making. It argues that metaevaluation is a form of evaluative enquiry that is aimed at assessing the strengths and weaknesses of primary evaluations. Through so doing, metaevaluation makes a valuable contribution toward enhancing the credibility and reliability of primary evaluations and the evidence provided to decision-makers. Metaevaluation has been utilised in different contexts in some developed countries to assess the quality of evaluations of development interventions, although scanty evidence exists of its use in developing country contexts.

Author Biography

Peter Misigalo Wandwasi, University of Johannesburg

Postdoctoral Fellow

References

Alkin, M. C. 1969. “Evaluating the Cost-effectiveness of Instructional Programs. Symposium presented at the Center for the Study of Evaluation.” Report No. 25, University of California, Los Angeles. https://files.eric.ed.gov/fulltext/ED031818.pdf. https://doi.org/10.1037/e684812011-001. DOI: https://doi.org/10.1037/e684812011-001

Alkin, M. C. (Ed.). 2012. Evaluation Roots: A Wider Perspective of Theorists’ Views and Influences. Thousand Oaks, CA: Sage Publications.

Apthorpe, R., and D. Gasper. 1982. “Policy Evaluation and Meta-evaluation: The Case of Rural Co-operatives.” World Development 10 (8): 651–668. https://doi.org/10.1016/0305-750X(82)90091-2. DOI: https://doi.org/10.1016/0305-750X(82)90091-2

Baxter, P., and S. Jack. 2008. “Qualitative Case Study Methodology: Study Design and Implementation for Novice Researchers.” The Qualitative Report 13 (4): 544–559.

Borich, G. D. 1990. Decision Oriented Evaluation. Oxford, England: Pergamon Press.

Bollen, K., P. Paxton, and R. Morishima. 2005. “Assessing International Evaluations: An Example from USAID’s Democracy and Governance Program.” American Journal of Evaluation 26 (2): 189–203. https://doi.org/10.1177/1098214005275640. DOI: https://doi.org/10.1177/1098214005275640

Caracelli, V. J, and L. J. Cooksy. 2008. “Meta-evaluation in Practice: Selection and Application of Criteria.” Journal of Multidisciplinary Evaluation 6 (11): 1–15.

Cooksy, L. J., and V. J. Caracelli. 2005. “Quality, Context, and Use: Issues in Achieving the Goals of Meta-evaluation.” American Journal of Evaluation 26 (1): 31–42. https://doi.org/10.1177/1098214004273252. DOI: https://doi.org/10.1177/1098214004273252

Cronbach, L. J. 2000. “Course Improvement through Evaluation.” In Evaluation Models: Viewpoints on Educational and Human Services, 2nd edition, 235–247, edited by D. L. Stufflebeam, G. F. Madaus, and T. Kellaghan. Hingham, MA: Kluwer. https://doi.org/10.1007/0-306-47559-6_14. DOI: https://doi.org/10.1007/0-306-47559-6_14

Cronbach, L. J., S. R. Ambron, S. M. Dornbusch, R. D. Hess, R. C. Hornik, D. C. Phillips, and S. S. Weiner. 1980. Toward Reform of Program Evaluation, 3. San Francisco: Jossey-Bass. DOI: https://doi.org/10.2307/1163707

Fitzpatrick, J. L., J. R. Sanders, and B. R. Worthen. 2004. Program Evaluation: Alternative Approaches and Practical Guidelines. Boston, MA: Pearson Publishing.

Glass, G. V. 1969. “The Growth of Evaluation Methodology.” Research paper no. 27, Boulder, Colo., University of Colorado: Laboratory of Educational Research.

Greene, J. C. 1992. “A Case Study of Evaluation Auditing as Meta-evaluation.” Evaluation and Program Planning 15 (1): 71–74. https://doi.org/10.1016/0149-7189(92)90063-Z. DOI: https://doi.org/10.1016/0149-7189(92)90063-Z

Guba, E. G., and Y. S. Lincoln. 1989. Fourth Generation Evaluation. Thousand Oaks, CA: Sage Publications.

Guba, E. G., and Y. S. Lincoln. 2001. Guidelines and Checklist for Constructivist (aka Fourth Generation) Evaluation. Beverly Hills, CA: Sage Publications.

Hanssen, C. E., F. Lawrenz, and D. O. Dunet. 2008. “Concurrent Meta-evaluation: A Critique.” American Journal of Evaluation 29 (4): 572–582. https://doi.org/10.1177/1098214008320462. DOI: https://doi.org/10.1177/1098214008320462

Hedler, H. C., and N. Gibram. 2009. “The Contribution of Metaevaluation to Program Evaluation: Proposition of a Model.” Journal of MultiDisciplinary Evaluation 6 (12): 210–223.

Nevo, D. 1983. “The Conceptualization of Educational Evaluation: An Analytical Review of the Literature.” Review of Educational Research 53 (1): 117–128. doi:10.3102/00346543053001117/. DOI: https://doi.org/10.3102/00346543053001117

Odom, S. L., and R. R. Fewell. 1983. “Program Evaluation in Early Childhood Special Education: A Meta-evaluation.” Educational Evaluation and Policy Analysis 5 (4): 445–460. https://doi.org/10.3102/01623737005004445. DOI: https://doi.org/10.3102/01623737005004445

OECD. 2004. The Organisation for Economic Co-operation and Development (OECD). Paris: Routledge Publishing.

Orr, R. R., S. J. Cameron, and D. M. Day. 1991. “Coping with Stress in Families with Children who have Mental Retardation: An Evaluation of the Double ABCX Model”. American Journal on Mental Retardation 95 (4): 444–450.

Owen, J. M., and P. Rogers. 1999. Programme Evaluation: Forms and Approaches. Thousand Oaks, CA: Sage Publications.

Patton, M. Q. 1997. “Toward Distinguishing Empowerment Evaluation and Placing it in a larger Context.” Evaluation Practice 18 (2): 147–163. DOI: https://doi.org/10.1177/109821409701800206

Pierre, R. G. 1983. Reanalyzing Programme Evaluation. Basel: Jossey-Bass Publishers. DOI: https://doi.org/10.2307/1164135

Rossi, P. H., H. E. Freeman, and M. W. Lipsey. 1999. Evaluation. A systematic Approach. Thousand Oaks, CA: Sage Publications.

Scriven, M. 1966. Primary Philosophy. Thousand Oaks, CA: Sage Publications.

Scriven, M. 1969. An Introduction to Metaevaluation. Educ Prod Rep 2 (4): 36–38.

Scriven, M. 2007. Key Evaluation Checklist (KEC). Thousand Oaks, CA: Sage Publications.

Shadish, W. R., T. D. Cook, and L. C. Leviton. 1991. Foundations of Program Evaluation: Theories of Practice. Thousand Oaks, CA: Sage Publications.

Shinkfield, A. D., and L. Stufflebeam. 2007. Evaluation Theory, Models and Applications. San Francisco: CA: John Wiley. Thousand Oaks, CA: Sage Publications.

Stige, B., K. Malterud, and T. Midtgarden. 2009. “Toward an Agenda for Evaluation of Qualitative Research.” Qualitative Health Research 19 (10): 1504–1516. https://doi.org/10.1177/1049732309348501. DOI: https://doi.org/10.1177/1049732309348501

Stufflebeam, D. L. 1971. The Relevance of the CIPP Evaluation Model for Educational Accountability. Abingdon, London: Routledge Publishing.

Stufflebeam, D. L. 1974. Meta-evaluation. MI: Western Michigan University Evaluation Centre.

Stufflebeam, D. L. 2000. Foundational Models for 21st Century Program Evaluation. Dordrecht, Netherlands: Springer.

Stufflebeam, D. L. 2001. “The Meta-evaluation Imperative.” American Journal of Evaluation 22 (2): 183–209. https://doi.org/10.1177/109821400102200204; https://doi.org/10.1016/S1098-2140(01)00127-8. DOI: https://doi.org/10.1016/S1098-2140(01)00127-8

Stufflebeam, D. L., and W. J. Webster. 1983. “An Analysis of Alternative Approaches to Evaluation.” In Evaluation Models, 23–43, edited by D. L. Stufflebeam. Dordrecht, Netherlands: Springer. https://doi.org/10.1007/978-94-009-6669-7_2. DOI: https://doi.org/10.1007/978-94-009-6669-7_2

The Joint Committee on Standards for Educational Evaluation. 1994. The Programme Evaluation Standards, 2nd edition. Thousand Oaks, CA: Sage Publications.

Tyler, R. W. 1942. “General Statement on Evaluation.” The Journal of Educational Research 35 (7): 492–501. https://doi.org/10.1080/00220671.1942.10881106. DOI: https://doi.org/10.1080/00220671.1942.10881106

Tyler, R. W. 1950. “Basic Principles of Curriculum and Instruction: Syllabus for Education 360.” IL: Chicago.

Uusikylä, P., and P. Virtanen. 2000. “Meta-evaluation as a Tool for Learning: A Case Study of the European Structural Fund Evaluations in Finland.” Evaluation 6 (1): 50–65. https://doi.org/10.1177/13563890022209118. DOI: https://doi.org/10.1177/13563890022209118

Weiss, C. H. 1998. Evaluation. Upper Saddle, NJ: Prentice Hall Publishing.

Wholey, J. S., H. P. Hatry, and K. E. Newcomer (Eds). 2004. Handbook of Practical Programme Evaluation. San Francisco, CA Jossey-Bass Publishers.

Wingate, L. A., C. L. S. Coryn, A. R. Gullickson, and L. J. Cooksy. 2010. The Program Evaluation Standards Applied for Meta-evaluation Purposes: Investigating Interrater Agreement and its Implications for Use.” Manuscript submitted for publication.

Wittrock, M. C., and D. E. Wiley. 1970. The Evaluation of Instruction: Issues and Problems. New York, NY: Holt, Rinehart and Winston Publishing.

World Bank. 2010. African Development Indicators 2008. Washington, DC: World Bank Publications.