This week I have been asked to comment on a presentation related to the importance of evaluation to program development given by Roger Rennekamp entitled, "Dare to Evaluate."
He makes three major points in his presentation:
1) Logic models must be better understood-- need deeper efforts.
2) Build capacity for increased rigor in evaluation.
3) Rethink the purpose of evaluation in Extension for program improvement.
In response to this presentation, the main point I plan to make are that evaluation is critical to improving programs. To do this we this we need to strengthen the development of logic or program models. To often current logic models do not are too superficial to inform program improvement. We need to development three levels of logic models--
1) a theoretical model of the problem that describes the factors that contribute to the problem.
2) a program model that is based on theory model, but identifies the specific change mechanisms that the program is designed to use to solve the problem and
3) an instructional model that describes the instructional processes that are designed to implement the program mechanisms.
In evaluation, too much of the focus has been on outcomes and not on program improvement. In many cases, it makes little sense to move to assessing outcomes because there are basic weaknesses in the program design that need improvement. When I champion the need for evaluation, I have Jacob's 5-tiered model of evaluation in mind-- needs assessment, utilization data collection, program clarification, measuring short-term outcomes and finally, measuring longer-term outcomes.
Additional sources:
Jacobs, F. (2003). Child and Family Program Evaluation: Learning to enjoy complexity. Applied Developmental Science, 7 (2), 62-75.
Small, S. A., Cooney, S. M., & O'Connor (2008). Evidence-informed program improvement: A manual. Retrieved Feb 17, 2009, http://www.uwex.edu/ces/flp/families/whatworks_manual.pdf
Small, S. A., Cooney, S. M., & O'Connor (2009). Evidence-informed program improvement: Using principles of effectiveness to enhance quality and impact of family-based prevention programs. Family Relations, 58, 1-13.
Taylor-Powell, E. & Henert, E. (2008). Developing a logic model: Teaching and training guide. Retrieved Feb. 17, 2009, http://www.uwex.edu/ces/pdande/evaluation/pdf/lmguidecomplete.pdf
No comments:
Post a Comment