You may have heard the term “developmental evaluation” and wondered what that is. How is it different from traditional evaluation, and is it something your organization should adopt?
Pioneered by Michael Quinn Patton in the mid-2000s, developmental evaluation applies to innovative, emergent, and complex programs that need to adjust their operations multiple times as things develop. In such contexts, it isn’t very helpful to hire an external evaluator at the end of grant cycle to judge a program on its initial plans. It is more helpful to work with an evaluator from the very beginning to help provide on-time insights into what is working and what should be changed. Developmental evaluation “is embedded rather than detached, continuous rather than episodic, and—most importantly —it has as its goal learning, not judgment.” (J.W. McConnell Family Foundation)
Most importantly, developmental evaluation asks questions about what you should be doing; where as formative and impact evaluations seek to learn how to do the same things more effectively or efficiently. Developmental evaluation helps with the design of a program or initiative rather than just seek to refine it.
How it works
Unlike traditional evaluators, who will usually come in to evaluate a program at regular intervals (often at the end of an annual funding cycle, just in time for report season), developmental evaluators are embedded as part of the program team nearly from the beginning.
In the early phases, evaluators and program stakeholders co-create a framework for learning about the program, including developing data collection methods, and spotting potential opportunities or weak spots. This is also a time when evaluators can work on building high-quality relationships with program team members themselves. These relationships are crucial in order to create a supportive, open, and honest arena for evaluation. This early inclusion of an evaluation “point of view” helps the evaluator understand the program more completely: its goals, stakeholders, limits, and particular strengths.
Once the program is in place, developmental evaluators stay involved. They often take a leadership role in group dynamics as the program is delivered, helping frontline and management staff talk through their expectations, finding areas where learning can occur, and helping to foster a “no wrong answers” culture of evaluation. Evaluators observe the program as a whole, looking for opportunities to strengthen the work, and for points where assessment and data collection fit into the flow of the activities. Developmental evaluators should expect to not only attend team meetings, but to speak up often about what they’re hearing.
In reporting, developmental evaluators typically use a feedback-loop approach. They invite stakeholders to comment on evaluation findings, which helps the evaluator spot larger patterns and weave together information from many points of view. Team members can then act quickly to integrate these findings back into the program itself, strengthening the delivery model and outcomes for all involved.
What to expect
If your organization is launching a new program, working with a developmental evaluation approach for the first phase can help spot opportunities and make course-corrections much sooner than traditional evaluation. However, it does require working with a new teammate—the evaluator—and being open to honest, sometimes challenging, conversations.
Remember that good developmental evaluators are not simply critical of a program: they bring skills in pattern recognition and strategic thinking, they prioritize their relationships with stakeholders, and they are comfortable taking the lead to guide conversations, elicit feedback, and spot challenges. They are often as invested in the success of a program as the frontline staff.
Why we use it
When a program or initiative is new or operating in complex and rapidly changing environments, it will need to change over time in order to be as effective as possible. Holding a program or initiative accountable to its initial plans and outcome targets will discourage such adaptations and results in reduced effectiveness. Instead, developmental evaluation focuses the whole team on learning rather than on a particular set of outcomes or results. In addition, it tracks developments, learnings, adjustments, and results to demonstrate accountability to supporters, donors, and funders.
How to do it
Over the course of many developmental evaluations and coaching many Innoweave grantees in the approach, we have developed two useful templates that you can download and use to guide your developmental evaluation efforts.
Principles-focused Developmental Evaluation
Often when things are rapidly changing, programs or initiatives cannot be effectively guided by rules that prescribe taking the same actions always, regardless of context. Just think of the COVID 19 pandemic; what worked best at the end of 2019, is likely not what worked best during the height of the pandemic, and likely not what works best today.
Instead, initiatives can be better guided by effectiveness principles, which provide context sensitive guidance on how to act to achieve a desired result. When following an effectiveness principle, the specific actions to take will vary by contextual factors. For instance, the rule of treating others based on how they would like to be treated, means that for each of your interactions, you need to consider how other people would like you to act, and that will vary by person and by specific contexts.
When programs or initiatives are guided by effectiveness principles, we take a principles-focus developmental approach to evaluation, which asks:
· Is the principle meaningful to those that are to follow it?
· Do those that are meant to follow the principle adhere to it?
· Does adhering to the principle lead to desired results?
Questions taken from Quinn Patton, M. (2018). Principles-Focused Evaluation. The GUIDE. The Guildford Press: New York & London.