Outcome Mapping: unpacking the black box between outputs and impacts
My recent internship at the International Institute for Democracy and Electoral Assistance (International IDEA) in Stockholm raised a number of important questions for me about how to monitor and evaluate international development programs. Trying to demonstrate a program’s ‘impact’ can often feel overwhelming, given that development goals are long-term and complex processes dependent on a myriad of political, economic, cultural and other factors. And while the methods for measuring, monitoring and evaluating impact remain hotly contested in the development community, two key approaches stood out in my time at International IDEA.
The first is the logical framework approach. Anyone involved in planning, monitoring or evaluating international development programs will be familiar with ‘logframes’. Pioneered in the 1970s by USAID to demonstrate what donor money was achieving, logframes have proved a very useful tool for mapping out and thinking critically about how a program leads to results.
Logframes provide a line of sight on the causal links between a program’s activities, outputs, outcomes and ultimate impacts. They offer a sense of simplicity and structure in an otherwise complex environment, can be used to communicate intentions to stakeholders, enable standardised reporting on indicators, and allow independent monitoring and evaluation of results (among other benefits).[1]
But what do outcomes really look like? How do the activities and outputs of a program lead to development impacts? To unpack this ‘black box’ in the logframe, outcome mapping (OM) has emerged as an increasingly popular methodology.
What is outcome mapping?
OM recognises that development programs are all about social change, and that social change is complex, unstable, non-linear, two-way, incremental, cumulative and often, beyond our control. Conducting evaluations in these open and changing environments poses a myriad of challenges, from defining success and deciding when to evaluate, to capturing emerging objectives and establishing cause and effect.[2]
To tackle these challenges, OM provides a framework for unpacking a program’s theory of change and collecting data on outcomes as they unfold. Importantly, it redefines ‘outcomes’ as changes in behaviour—the actions, activities and relationships—of the stakeholders directly in contact with the program (known as ‘boundary partners’). [3] This concept of boundary partners is fundamental to OM but not always present in logframes and, as a result, the two approaches often produce very different outcome statements. According to OM, behavioural change of boundary partners is critical to moving up the results chain.
OM also recognises that, in reality, programs have limited control over whether their ultimate goal is achieved, given the range of social, political, environmental, economic and other factors that support or hinder intended outcomes. Rather than claiming attribution of a development impact, OM claims contribution to it.[4] It teaches that programs have control over their inputs, activities and outputs; influence over their outcomes; and simply an interest in the ultimate impact. In short, OM focuses on a program’s sphere of influence.[5]
In practice, OM offers 12 tools for planning, monitoring and evaluating outcomes, which can be adapted to suit individual contexts. These tools are intended to help stakeholders identify and think critically about:
- why the program has ultimately been established
- who the program has direct influence over (who the boundary partners are)
- what changes in behaviour (outcomes) the program would ‘expect’, ‘like’ and ‘love’ to see from its boundary partners
- what the qualitative and quantitative indicators (‘progress markers’) are for these outcomes
- what strategies are in place to influence each boundary partner
- which monitoring tools the program should use
- what the evaluation priorities are for an evaluation plan.
It helps to build a credible picture of how a program contributes to results, putting people at the centre of development and recognising the complex and non-linear nature of social change.
So while the logframe approach remains engrained in most development agencies, practitioners should consider the value in an OM approach. As put by Michael Quinn Patton, OM affirms that ‘being attentive along the journey is as important as, and critical to, arriving at the destination’.[6]
To learn more about OM, visit the Outcome Mapping Learning Community, a one-stop shop for all things OM.
