Earlier this month we outlined a side-by-side comparison of outcome measurement and program evaluation. In the next two blog articles, we unpack the outcome measurement process specifically.
Outcome measurement: Steps One Through Five
1. Assess your organization’s readiness.
Outcome measurement is the determination and evaluation of a program’s results, and their comparison with the intended or projected results. This definition requires a starting point.
2. Select the program and create a plan.
After understanding the state of your organization, decide which program you will measure. Choose a program that is undergoing a certain change or requires transformation. Outline a specific plan for outcome measurement, including a timeline with milestones by which it will be completed.
3. Describe the program’s mission and activities using a logic model.
Visually depict the program you choose with a logic model, such as this:
Complete the logic model with your outcome measurement team during a series of working sessions.
4. Identify the program’s intended results, or outcomes.
Program outcomes comprise the second half of the logic model. These goals are the “meat” of outcome measurement, and the focus of subsequent steps of the process.
5. Identify indicators of success for each outcome.
Bring your program’s outcomes to life by assigning each goal a metric, which ensures completion can be successfully measured.
Outcome measurement: The Story of Sue Ito
To illustrate this first half of the outcome measurement process, we are introducing Sue Ito, Director of Planning and Performance at the Jefferson County Library (JCL). Sue has been tasked by the County Board of Directors to report a set of agency outcomes on an annual basis.
To begin, Sue uses a “Start With the End in Mind” planning product, which shows that JCL’s Young Adult Education Program (YAEP) is ripe with opportunity, and much of the organization is eager to support its growth. Sue decides to conduct outcome measurement for YAEP and forms a cross-department team to assist her.
The team meets weekly and for the first month focuses its working sessions on creating a logic model to better visualize and understand the current program. Below is a snapshot of the YAEP inputs, outputs, and outcomes.
What we invest
What we produce
Career development materials
|Career education and resources for disadvantaged youth
Technology training for program participants
Job skills training for program participants
Life skills coaching
Safe space for disadvantaged youth to meet weekly
|New learning, such as:
IT knowledge gained
Job skills learned
Positive attitudes toward higher education developed
|New actions, such as:
Participants apply to post-high school colleges or apprenticeship programs
YAEP participation grows
JCL receives increased community funding
|New conditions, such as:
Rate of college-educated youth in community increases
Young adult crime rate decreases
Community reputation is elevated
Sue reviews the logic model with her team and realizes the outcomes are strong, but vague. How will the new learnings, actions, and conditions be proven? The team creates specific metrics, or indicators of success for each, captured below.
|IT knowledge gained
|60 percent of participants show evidence of increased IT knowledge on pre- and post- program survey
|Increase in college or apprenticeship program participation
|80 percent of participants apply to higher education programs
|Rate of college-educated youth in community increases
|Number of college-educated youth in community increases by 20 percent
|Job skills learned
|100 percent of participants report acquisition of new job skills
|Increase in YAEP participation
|Program enrollment rate grows by 30 percent in the first year
|Young adult crime rate decreases
|Young adult crime rate decreases
by 5 percent
|Positive attitudes toward higher education developed
|75 percent of participants cite intentions to pursue higher education opportunities
|Increase in community funding for JCL
|JCL receives a $25,000 grant from the local government to support growing YAEP
|Community reputation elevated
|Average real estate prices in Jefferson County grow by $10,000 year-over-year
What are your thoughts about the above outcomes and indicators? Which are easy wins, and which require a great deal of work to achieve? Please share your feedback in the comment section below.
In our next blog article, we’ll wrap up the outcome measurement process with our story of Sue and YAEP. Stay tuned!