Program Evaluation Simplified

Many nonprofit leaders are nervous about program evaluation. They assume it is a hard-to-learn, complex science that requires a lot of time and resources. They delay evaluation or implement shortcuts to ease the pain of what they believe is a difficult process.

Are you one of them?

The truth is that program evaluation is not as scary as it may sound. And it is a best practice of every high-performing nonprofit organization. It is part of a broader strategic management process, serving as a tool to help assess the relationship between an organization’s business functions. It identifies how well your organization is working as one system, including the areas that are most efficient and opportunities for improvement.

Program evaluation is an in-depth look at a program based on focused evaluation questions. The questions you ask in an evaluation are essential to determining every other component, so defining these questions is the first step of program evaluation.

The types of program evaluation

Formative evaluation

Choose formative evaluation to increase the likelihood your program will achieve its goals. This type benefits organizations that are entering a new phase of program planning, starting a program, or applying a program at a different location or with a new population.

Formative evaluation may answer questions such as:

• Is the program adjusting the needs of the population?
• What is the program’s status regarding reaching its goals?
• Are resources and program components allocated appropriately to ensure goals are met?

Formative evaluation is the type used most often in situations of change. If your organization is launching a new program and must determine the intended population’s needs and how the program will meet those needs, this type of evaluation is for you.

Process-based evaluation

Process-based evaluation focuses on implementation. It determines how a program operates and achieves its results. This type is beneficial when programs appear to contain major inefficiencies or must be visually illustrated to external stakeholders.

Process-based evaluation may answer questions such as:

• How do we represent the program to stakeholders?
• What is the lifecycle from beginning to end?
• How much of the planned program is being implemented?

For example, your organization aims to provide detailed information about how one of your programs operates to attract greater awareness and participation. You ask the first evaluation question in the list above and conduct a process evaluation to determine the activities delivered, populations served, and resources used.

Outcome or impact-based evaluation

Learn to what extent your program is delivering the outcomes it is designed to achieve with outcome or impact-based evaluation. This type is beneficial when you need to justify the existence of the program to external stakeholders or want to track performance over time.

Outcome-based evaluation may answer questions such as:

• What indicators measure to what extent the program is meeting intended outcomes?
• How effectively is the program meeting those desired outcomes?

When regulatory agencies or funders seek evidence that your program is meeting the goals you promise to deliver, it’s time for an outcome-based evaluation. Outcome-based evaluation may be the most-used type.

Enter the logic model

After you’ve decided which type of evaluation you will use, it’s time to implement a logic model. A logic model is a systematic and visual way of representing relationships, a blueprint that helps organize information, and a tool to evaluate programs.

A logic model contains the following core elements:

• Inputs: The resources used to meet the program’s needs—money, time, materials, and equipment
• Activities: Daily happenings that comprise the work of the program—employees’ tasks and participants’ experiences
• Outputs: Direct and tangible products of the program’s activities that have little inherent value to program participants—indicate program efficiency
• Outcomes: Benefits of change for individuals because of participation in the program—indicate program effectiveness

Determining program outcomes is the most important component of the logic model. You will revisit your program outcomes time and again to measure actual program effectiveness against intended program effectiveness.

Outcomes can be short-term, medium-term, and long-term. Short-term outcomes produce increasing knowledge, skills, and attitudes. Intermediate outcomes affect behavior. Long-term outcomes show the sustainable impact of a program.

• Short-term outcome: City leaders believe opening more soup kitchens is important (attitude).
• Medium-term outcome: City leaders vote to open more soup kitchens across the metropolitan area (behavior).
• Long-term outcome: More city residents have access to hot food, and fewer people go hungry (impact).

The bigger picture impact of program evaluation is its ability to inform your organization’s strategic management approach. This is critical for helping to justify program resources to both internal and external stakeholders. Additionally, tracking outcomes over time provides rich data about trends.