As we celebrate our 15th anniversary, we’re publishing a series of articles about how our business has changed over the years. This month’s article focuses on the shifts in program evaluation.
Your nonprofit exists to make a difference. Your vision declares what aspirations you aim to achieve; your mission describes how you will embody that purpose. And how do you know whether your organization is accomplishing what it has set out to do and be? You evaluate—by consistently and continuously monitoring progress against goals. Because as the age-old management principle attributed to Peter Drucker wisely advises: You cannot improve what you do not measure.
Rarely is evaluation a nonprofit’s favorite practice, and 15 years ago it was less so. Many mission-driven organizations in 2008 shirked measurement, viewing it as a distraction from their daily operations. But boards, funders, organizational leaders and community members have increasingly required organizations to show detailed evidence of their impact and progress.
Today the sector is steeped in savvier measurement and evaluation processes. In this article we look at seven ways evaluation has changed since Brighter Strategies first partnered with nonprofit agencies to advance their people, planning, processes, and performance.
Increased Emphasis on Evidence-Based Practices and ROI
The social sector has always been held to a higher standard when demonstrating return-on-investment (ROI) for their interventions, and this has been even more true in the last 15 years. Policymakers and funders increasingly require rigorous evaluations to inform resource allocation and policy development. Organizations have developed best practices around evaluation for greater accountability.
For example, within the past decade and a half, nonprofits have widely adopted program evaluation and outcome measurement, which use a scientific approach to show the impact an organization’s initiatives are having on their mission and goals. Even more recently, randomized controlled trials (RCTs) have grown in popularity. These involve randomly assigning individuals or groups to treatment and control conditions and are seen as a gold standard for establishing causal relationships between interventions and outcomes.
Emphasis on Stakeholder Engagement
Stakeholder engagement has become central to the evaluation process. Including the perspectives and input of program participants, funders, and other key constituents is essential for producing relevant and actionable findings. Organizations today are engaging stakeholders in program evaluation in various ways, including seeking their input on evaluation questions, logic models, and data recommendations; collaborating on data collection; and co-communicating evaluation results. This partnership elevates the importance of measurement and evaluation to the very people who are critical for a program’s success.
Advancements in Technology and Data
Technology has transformed evaluation—and all aspects of nonprofit strategy since 2008. Organizations now integrate technology throughout program evaluation with data collection via online surveys and mobile apps, and data visualization tools for reporting results.
Additionally, the proliferation of digital data collection methods, the use of big data, and the development of sophisticated statistical and analytical tools have revolutionized program evaluation. Researchers can now analyze larger datasets and conduct more complex analyses, allowing for more nuanced insights. Today, artificial intelligence improves data collection efficiency and accuracy, and predictive analytics allow organizations to draw smarter conclusions from a vast amount of data.
Greater Attention to Ethical Considerations
With all this data comes great responsibility. Ethical considerations in program evaluation, including informed consent, privacy, and data security, have gained increased attention in response to concerns about data ethics and participant rights. And the advent of the General Data Protection Regulation (GDPR) in 2016 reinforced the rights of individuals whose data organizations are collecting. Nonprofits have adapted their evaluation processes to comply with these extra layers of regulation and compliance.
Application of Mixed Methods
Despite the massive amount of quantitative data at an organization’s fingertips, many are choosing qualitative information in favor of the numbers. With a growing acceptance of qualitative data, the methods nonprofits use to collect it have increased, too. Interviews, focus groups, and observation allow for critical nuances that survey data often overlook. Simultaneously, storytelling has arisen as a significant communication trend, and stakeholders want understand evaluation through narrative. Visualization tools like word clouds, timelines, graph databases, concept maps, and infographics provide a more holistic view of such impact.
Focus on Diversity, Equity, and Inclusion
There has been a growing recognition of the importance of diversity, equity, and inclusion (DEI) in program evaluation. Evaluators are more conscious of the need to consider the unique experiences of diverse populations and to address issues of bias and discrimination. Culturally responsive evaluation methods have been developed and implemented with greater vigor and attention. A DEI lens matters for evaluation because it allows organizations to involve more people and more perspectives to improve the quality of their projects.
Additionally, DEI has turned our attention to the systematic injustice that exists within and around our organizations. Today there is a growing interest in evaluating policy interventions and advocacy campaigns to assess their effectiveness in bringing about social and political change.
Increased Demand for Real-Time Evaluation and Continuous Improvement
About ten years ago the field donned the acronym VUCA (volatile, uncertain, complex, and ambiguous) to describe the landscape in which organizations were operating. Now the new normal, a VUCA environment requires nonprofits to adopt a more flexible and agile approach to evaluation—one that considers measurement an ongoing and dynamic process, rather than a linear means to an end.
In addition to traditional summative evaluations that occur at the end of a learning period or program and provide a final performance grade, there is a growing demand for real-time and formative evaluations that help design programs and give ongoing feedback to program implementers. This allows for mid-course corrections and improvements, which is different from a traditional “gotcha” evaluation.
Nonprofit measurement and evaluation practices have faced much disruption in the past 15 years as these seven trends illustrate. And nonprofit leaders have adapted, evolved, and grown in concert. I’m thrilled with the progress the social sector has made in embracing new ways of assessing and improving their stories of impact. If you’re ready to make a change within your own agency and are seeking expert guidance, consider Brighter Strategies Consulting. Since 2008 we have partnered with dozens of nonprofits to revolutionize their people, planning, processes, and performance. Contact us today to learn more.