Program Evaluation in Practice

Earlier this month we described the nine-step program evaluation process, a measurement and evaluation best practice that organizational development firm Brighter Strategies has helped many nonprofit agencies to implement. To give you a real-world idea of what this process looks like in action, below we revisit Sue Ito and her Young Adult Education Program at Jefferson County Library.

Fast-forward a few years since we last heard from Sue. Now the YAEP is a well-established and thriving program at JCL. However, the library itself has experienced recent change. County government funding cuts have affected many community organizations, and JCL is no exception. The YAEP is at risk of losing much of the technology support it has relied on for success.

Program evaluation for change

As we’ve learned, program evaluation is the process of collecting information about a program or some aspect of a program to make necessary decisions about that program. Sue believes a process-based program evaluation, which determines how a program operates and achieves its results and is beneficial when long-standing programs experience recent change, will help YAEP to survive and even thrive through budget constraints.

Sue mobilizes her program planning team, and they outline their program evaluation strategy going forward. They revisit the logic model they created to ensure they have a clear picture of the original program, including inputs, outputs, and outcomes. They review trending outcome data since the program’s inception, to determine if any results have slipped. The program “picture” is relatively unchanged, and the program outcomes have overperformed against their goals each year.

Now, with the external environment changing a handful of the program inputs that rely on monetary resources, Sue asks the team to remove these items from the logic model. This brings total inputs from five to a mere two: volunteer staff and library space.

With this bleak backdrop, the team creates a data collection plan to determine if the YAEP could achieve its existing performance outcomes with barebone resources. Through one-on-one interviews with program participants, volunteers, JCL employees, and community representatives, Sue’s staff learns that collective passion for the YAEP is at an all-time high. The program has helped the community to strengthen relationships with its at-risk young adult population, and many volunteers have made personal connections with participants as a result.

Sue’s team takes its data digging one step further. It gathers a focus group with a variety of stakeholders to ask the question: How could Jefferson County Library’s Young Adult Program survive if the community had no money to invest in it? This time-efficient process produces a variety of feedback, which Sue’s team incorporates within its revised logic model. They can now show how YAEP will be able to create the same outputs and subsequent outcomes with zero funding from the library. Volunteers are engaged in the program to such an extent that they are more than willing to contribute their time, tools, and creativity to keep YAEP alive.

Sue is thrilled. Her team’s investment in program evaluation has helped to redirect resources for maximum impact during change at JCL. She completes the process by creating an action plan for next steps, including new volunteer recruitment and training. She shares evaluation results with all individuals involved in the process, as well as vested community stakeholders.

For more details on program evaluation, check out organizational development consulting firm Brighter Strategies’s resource guide, “Evaluating Performance Outcomes: A Guide to Implementing Program Evaluations.”