- Susan P. Curnan and Lisa A. LaCava

The increasing pressure to provide evidence of the effectiveness of social programs and initiatives has led to a strong focus on "outcome evaluation" as the primary method of evaluating programs. While we believe that demonstrating effectiveness and measuring outcomes and impact are important and valuable--and that outcome evaluation is a key component to any project-level evaluation--we have found that grantees, evaluators (and yes, even foundations!) often focus too quickly and too exclusively on outcome evaluation.

One of the unfortunate and unintended results of the pressure to hasten the implementation of an outcome evaluation is that many grantees begin outcome evaluation components before they are ready, and before equally important context and implementation evaluation (which might inform the outcome evaluation design and implementation process) have been conducted. The ultimate result is twofold: (1) outcome evaluations are not particularly useful and miss important aspects related to what outcomes are actually being achieved; and (2) programs get evaluated for outcomes too early in their development--before they have clearly determined what outcomes they are attempting to achieve and what processes and activities they are implementing to achieve these outcomes.

Getting ready for outcome evaluation may look different for different organizations. This depends on a variety of factors, including the type of program, the stage the program is in (new versus more established) the clientele being served, the types of outcomes the program is attempting to achieve, etc. In most cases, however, the development and use of a program logic model, or theory of change, is an effective way to prepare for outcome evaluation. A program logic model is a picture of how your program works: the theory and assumptions underlying the program. This model links outcomes (both short- and long-term) with program activities/processes and the theoretical assumptions/principles of the program. This model provides a roadmap of your program, highlighting how it is expected to work, what activities need to come before others, and how desired outcomes are achieved.

There are multiple benefits to the development and use of a program logic model, including clarification and design of the program; a base from which to conduct ongoing evaluation of the program; and the ability to evaluate complex initiatives with intangible outcomes (such as increased community participation) or long-term outcomes that will not be achieved for several years.

In addition, there is value in the process of developing a logic model. The process is an iterative one that requires stakeholders to work together to clarify the underlying rationale for the program and the conditions under which success is most likely to be achieved. It also requires stakeholders to build consensus on long-term as well as interim outcomes, and the activities and processes that will lead to these outcomes. Through this process, gaps in activities, expected outcomes, and theoretical assumptions can be identified, resulting in changes based on consensus-building and a logical process rather than on personalities, politics, or ideology. The clarity of thinking that occurs from the process of building the model becomes an important part of the overall success of the program. The model itself provides a focal point for discussion. It can be used to explain the program to others and to create a sense of ownership among the stakeholders.

Logic models can be created in many different ways. The starting place could be the elements of an existing program which are then organized into their logical flow. For a new program that is in the planning phase, the starting place could be the mission and long-term goals of the program. The intermediate objectives that lead to those long-term goals are added to the model, followed by the short-term outcomes that will result from those intermediate objectives.

The key to building any model is to prepare a working draft that can be refined as the program develops. Most of a logic model's value is in the process of creating, validating, and then modifying the model. In fact, an effective logic model will be refined and changed many times throughout the evaluation process as staff and stakeholders learn more about the program, how and why it works, and how it is being operationalized. As you test different pieces of the model, you will discover which activities are working and which are not. You may also discover that some of your initial assumptions were wrong, resulting in necessary model revisions to adapt it to current realities.

Although logic models come in many shapes and sizes, three types of models seem to be the most useful: an outcomes model, an activities model, and a theory model. While we will not go into the intricacies of the way these models work here, it is important to note that program staff often need to combine two or three of them. The Community Youth Development Framework is an example of a "hybrid logic model." Hybrids that combine theoretical constructs and assumptions to activities and processes to interim and long-term outcomes are the most effective models from which to develop evaluation designs aimed at documenting outcomes.