| Spring 1999, v15-2 |
by Meredith Honig, Joseph Kahne, Milbrey W. McLaughlin
Consider the following:
- A director of a youth organization in
a northwestern town searches the Internet for information on the impact of an experiential
education program on youths' leadership skills. Searching for "leadership"
and "youth development" she turns up thousands of documents. The suggestion
on the computer screen reads: "Refine Your Search."
- In southern California, a youth worker
reviews the "final" report from an evaluation of her five year old youth
development program--the first time she has seen information on the outcomes of her
program. In leafing through the 150 page document, she reads that the program has
generally failed to reach its goals even though her day-to-day experience suggests
that the program has been extremely successful.
- A national foundation issues a policy
report in which they summarize lessons learned from their recent efforts to fund
successful community initiatives. This report is widely praised, circulated, and
read but a year later many readers agree that they have not incorporated these lessons
into their own work.
On what basis should the youth director judge
the information she finds on the Internet? What does it mean when evaluation results
do not match the day-to-day experiences of the staff? How can research findings and
other lessons learned from practice guide current and future youth development work?
And how can we better connect research and practice?
As researchers in the field of youth development, we are concerned about building
a stronger professional base for youth work in research and practice. Using practice
to inform research and conducting and presenting research in ways that better support
practice is a certain beginning. It is this task--translating research into practice
and vice versa--that defines the purpose of this column, "Theory and Practice:
Completing the Circle." In each issue we will present recent research of relevance
to practitioners in youth development and suggest specific, practical ways this information
can be used day-to-day in youth development work.
To highlight the importance of connecting research and practice and to introduce
our approach to this topic, we launch this column with a discussion of Theory of
Action approaches to various aspects of youth development work.
The Theory of Action Approach: Focusing
on Strengths
A theory of action approach is increasingly
being used in youth development and other initiatives to focus program development,
implementation, and evaluation in ways that build on program strengths and improve
the overall quality of youth development work. Generally, it requires us to perform
the following two tasks:
- Éxplicitly state the often invisible
assumptions on which programs are based, and the conditions, procedures/resources,
and agreements on which their success depends.
- Critically examine whether hese assumptions
make sense: Is there data and other evidence to support the belief that the chosen
strategy is the best or correct one? Do the conditions, procedures/resources, and
agreements necessary for success exist? Is the program being evaluated according
to criteria that are fair, given available resources and other conditions? Is the
entire program not working or are there strong elements which, given more favorable
conditions, might lead to success?
Why Take a Theory of Action Approach?
In the following sections we describe several ways a theory of action approach
can better focus and improve program development and implementation, evaluation,
and links between research and practice.
Program Development and Implementation
Often the best programs and plans fail because the conditions on which success
depends are not met. This is particularly true in youth development organizations
where a variety of unforseen factors can greatly affect program implementation and
success. The daily demands of youth work may mean that, day to day, the program isn't
implemented as planned. Similarly, partnerships or interorganizational groups which
often govern/advise youth development programs can become dysfunctional due to a
variety of reasons:
- Conflicting ideas regarding the best strategies
for achieving goals.
- Lack of shared understanding of the conditions
required for success.
- Failure to commit the necessary time and
resources.
A theory of action approach requires program designers to clearly identify why the
chosen strategies can lead to the targeted goals. This means asking questions such
as: why do we believe an after-school program will lead to improvements in school
performance? Given the particular needs and strengths of our youth, is a job readiness
program the best approach to career education? Similarly, a theory of action approach
means clarifying the conditions under which a particular strategy may be successful.
For example, was attendance at the after-school program low because the program didn't
meet youth's needs or was it for other reasons? Since participation in the program
required regular attendance early on a Saturday morning, were there barriers related
to time--i.e., did the interested youth have alarm clocks or other ways to wake up
on time? did travel to the program require youth to cross gang boundaries? did the
Saturday bus schedule enable youth to reach the field? Since participation in the
program required membership in the organization, were there barriers to that membership--i.e.,
could interested youth afford the $5.00 membership fee? did membership require parental
consent that might be difficult for some youth to acquire? And finally, even though
the sports program was offered at a field easily accessible by public transportation,
were the youth able to travel across town to the main office to apply for membership?
Of course, it is impossible to monitor and control all the factors that might contribute
to a program's success or failure. But if youth workers and their partners are to
have a good chance of achieving their goals. it is vitally important to continuously
clarify those conditions that may directly impact the program's chance of success.
Focusing Evaluation
Evaluation--particularly traditional process and outcome evaluations--has been
problematic for the field of youth development. Not only do findings tend to come
too late to inform action; they fail to focus on the information youth development
programs need. In addition, evaluations of youth development programs often make
broad statements about overall programs and are insufficiently discriminating about
which aspects of the program do and do not work--and why.
To avoid these problems, a theory of action approach requires:
- Developing outcome measures and other standards
for accountability that are realistic and appropriate.
- Identifying the range of conditions on which
the success of a program depends and using evaluation to gauge whether these conditions
are met.
- Uncovering alternative explanations for program
success or failure and using them to guide the collection and analysis of data.
By focusing on specific assumptions (e.g., that students will have transportation
to get to the program and that there will be jobs available for youth once the job
training is provided) rather than simply whether or not the program worked, evaluation
can be used to build on the successful aspects of a given strategy and the strengths
of the participating youth.* This approach can also be very useful when evaluating
an initiative that involves more than one program or project. For example, let's
say a national organization comprised of local chapters wants to evaluate its efforts
to increase youth employment nationwide. The chapters involve many participants with
different expectations and separate strategies in towns and cities across the country
ranging from those that teach job and leadership skills through community service
projects to those that offer vocational education courses. On what basis can these
different strategies be compared and the effectiveness of the organization as a provider
of job training nationwide be evaluated? By grouping local programs according to
shared theories of action, the national organization can study their sites on its
own terms, based on standards appropriate to the strategy each site has chosen. It
can compare programs that share common conditions to better understand exactly what
component in which program might affect its success. Grouping local programs can
also explain the factors that constrain or enable a given reform strategy, enabling
practionerers to use this information to guide program improvement.
Process for Linking Research and Practice
When we identify theories of action through any one of the processes described
above, we are essentially asking: What do we know about the programs that work for
the youth in my community? On what conditions did the success of prior efforts depend?
Do those conditions still exist? What additional information do we need to test these
assumptions? Given the particular youth development strategy and the assumptions
underlying it, what other programs can we use for comparison? Such questions can
help us think through what research will be useful and how to evaluate the research
we already have.
In this column we will use a theory of action approach for the presentation of research
and examples from practice. Rather than simply presenting recent research, we will
highlight the theories of action underlying it. For instance, which measures of success
were chosen by the evaluators? Are these the best or most relevant outcomes to measure?
To which activities are the findings applicable--was the program the researchers
studied based on a theory of change that you share? What conditions constrained/enabled
the success of the program? Given various and sometimes conflicting theories about
what youth need to develop and learn, which seem to hold up against the test of research
and practice? How should we compare various approaches to youth development? Finally,
how can action inform theory?
Exercise: Theory of Action in Action
In the table that follows, we have developed an inquiry process that directors,
staff, and others involved in youth development programs can adapt and use to try
out a theory of action approach. While there are many variations to and uses for
such a process, Table 1 offers one example of how a theory of action approach might
be used for program planning and development. Table 2 provides a blank template of
the same for your own experimentation. You may find it helpful to review the Sample
Mapping Exercise in Table 1 before you begin.
|
|
| |
Suggested Directions
- In Table 2 complete the box marked "Goal"
with one goal you aim to achieve in your program. Complete the first two columns
of the table ("Indicators" and "Activities") using your own knowledge,
official documents, your evaluation forms or activity logs, job descriptions, and
so on. Depending on the experience and expertise of various members of your organization,
these columns can be completed by the program/organization director or staff.
- Convene your staff, your partners, or other
key players in the design and operation of your program.
- Divide your group into pairs or groups of
three and have each group complete the "Underlying Assumption" column.
(Be sure to leave enough time for discussion--about 15 minutes for groups of two
to three.) Consider having each group write their assumptions on pieces paper (one
assumption per page) that can be pasted on a wall or large board and easily read
by the large group.
- Reconvene your staff as a whole and have
each small group share their assumptions. Allow time for all participants to review
the small groups' assumptions, and have someone chart the assumptions each group
shares. Does the group agree on a set of underlying assumptions? Continue your discussion
until the group comes to consensus on a core set of underlying assumptions.
- Complete the "Assumption Check"
column: as a large group or in pairs discuss whether the assumptions are valid, and
why or why not. What information do you need to "test" these assumptions?
- For the "Future Actions" column,
make a list of the additional information you need and share the responsibility for
collecting this information among your group. Be sure not to be overly ambitious
about what data you need. Schedule a follow-up meeting to discuss your findings
References
1. Argyris, Chris and Donald A. Schon,
Organizational Learning II, Addison-Wesley, Reading MA, 1996. This
book elaborates on a theory of action approach to program planning and capacity building
in orgainzations. It includes transcripts of workshops conducted by Chris Argyris
in which he uses a theory of action aproach to help organizational directors and
executives reflect on their leadership.
2. Connell, James P., Anne C. Kubisch, Lisbeth B. Schorr, and Carol H. Weiss, editors,
New Approaches to Evaluating Community Initiatives, The Aspen Institute, Washington,
D.C., 1995. This document focuses on the challenges of collaborative community initiatives,
many of which are focused on children and youth. It includes an article on the theory
of change by Carol Weiss, cited in this article.
3. Kahne, Joseph and Milbrey McLaughlin, New Designs for Youth Development,
Vol 14 No. 1, "Framing Isues for Policy and Practice," pp 17-22, Winter,
1996.
4. Kibel, Barry, New Designs for Youth Development, "Evaluation Using
Results Mapping," Vol. 12 No. 1, pp 9-15, Winter 1996.
5. Terry, John and Ann Dosher, New Designs for Youth Development, "How
Organizations Learn," Vol. 13 No. 2, pp 10-15, Spring, 1997.
Authors
Meredith Honig has had various professional experiences in policy
and program development in the areas of school-linked services, child welfare, education,
and youth development. She is currently a researcher and doctoral student at Stanford
University where her research focuses on
challenges of connecting communities and schools and taking such efforts to scale.
Joseph Kahne teaches
at the University of Illinois at Chicago. He currently studies and works on a variety
of educational reform, service-learning, and youth development initiatives. His book,
Reframing Educational Policy: Democracy, Community, and the Individual, was
recently published by Teachers College Press.
Milbrey W. McLaughlin is
the author of several educational books, including Urban Santuaries: Neighborhood
Organizations in the Lives and Futures of Inner-City Youth. Milbrey currently
teaches at Stanford University.
|
|
