Nonprofit leaders often struggle with a critical question: How do I know my program is working as intended?
You know what efforts you’ve taken, but you don’t know how effective they are unless you define a goal and then collect and analyze data to measure impact..
While many people think an evaluation will be complicated, I’ve seen how just a few hours of thinking through a logic model can dramatically change the results of a program.
To measure impact, you must evaluate the execution
To illustrate the importance of impact measurement, let’s look at a short case study.
The National School Lunch Program (NSLP) impact measurement
In order to combat childhood obesity, the NSLP began providing fruits and vegetables in school lunches.
The program assumed that providing healthier foods would create healthier students. However, an evaluation showed that kids were often discarding these items. – up to 70% of vegetables and 40% of fruits were thrown away, leading to significant food waste.
Program evaluations found a few key issues in the execution:
- Lack of student buy-in: The healthier options were often unfamiliar, or less appealing to students, leading them to discard the food rather than eat it.
- Inadequate education: The program was not coupled with educational components to help teach the importance of nutrition and why these new foods were being introduced to begin with.
- Peer influence: In many cases, peer behavior influenced food choices. If one student discarded their apple or vegetables, others were likely to follow suit.
Program adjustments:
In response to these findings, some schools began to make adjustments, such as:
- involving students in the selection of menu items
- offering taste tests
- implementing nutrition education programs to increase acceptance of healthier foods
Schools also explored alternative strategies, such as slicing fruits like apples, offering dips, or serving fruits and vegetables in more appealing ways.
These changes helped reduce waste, improve consumption and ultimately move the needle on their mission.
This story illustrates why evaluation is vital for:
- identifying areas that require improvement
- ensuring that the programs align with their goals
- adapting strategies to meet the needs of the communities they serve.
However, despite its importance, evaluation as a tool frequently encounters resistance and hesitance.
Misconceptions in gauging program effectiveness
Many organizations struggle to properly measure impact due to these common misconceptions:
- Data collection: Organizations often collect data that is easy to gather and possibly interesting, but not actionable. This wastes a lot of time and money compared to an evaluation focused on collecting a few actionable data points that are aligned with key outcomes.
- Qualitative vs. quantitative: There is a tendency to prefer quantitative data over qualitative because people prefer to review charts and tables. However, qualitative analysis can provide rich insights into your outcomes or can serve as a complement to quantitative data.
- Timing of the evaluation: Many grants require an impact measurement at the end of a campaign, which has led organizations to believe program evaluation is something you only do after implementation. The truth is that evaluating a program before it begins can dramatically change the design of a program toward clear outcomes
Measure your impact through evaluation
You can’t know your impact unless you define a target outcome, and then measure against it.
Think of every program as a hypothesis for change, and then test that hypothesis, to understand what works and what doesn’t.
If impact measurement is a challenge for your organization, please reach out and let’s discuss how we can integrate the evaluation process into your programs.
