Empowerment Evaluation by David Fetterman – 2019 AES Conference Post-Workshop
As emerging evaluators, we are constantly looking to explore and learn about concepts and theories of evaluation. In the lead up to the Australia Evaluation Society (AES) International Evaluation Conference, we had particularly been exposed to certain principles associated with ‘Empowerment Evaluation’ such as capacity building and inclusion in our project work. This triggered our fascination around the approach and so the three of us attended David Fetterman’s Empowerment Evaluation Workshop.
We gained our first impressions of David Fetterman while observing him on Day One of the Conference. By watching him walk around to all the stalls, introducing himself and having frank conversations, he established that he was ‘down to earth’ from the start. Another feature noticed was his charismatic charm – how he could draw a crowd together. By the time he got around to the ARTD booth, he had us all captured in conversation with him. It was very refreshing to meet such a humble character who was a keynote speaker, but who also took the time to connect with everyone at the conference. This was truly inspiring for us three, as relatively young and emerging evaluators, to receive this kind of attention from someone with an amazing reputation in the field.
This connection really motivated us to ‘take the journey to climb the mountain’ that is empowerment evaluation. We attended his workshop and took away the following.
Empowerment evaluation – what is it?
Empowerment evaluation is ‘the use of evaluation concepts, techniques, and findings to foster improvement and self-determination.’ It seeks to challenge power dynamics by opening up evaluation to all and encouraging discussion at all levels. In doing so, it also works to institutionalise evaluation within organisations. Put simply, it makes evaluation common practice by making it easy, inclusive and meaningful.
This approach and the 10 guiding principles are in alignment with community development principles.
Source: Fetterman, D., Kaftarian, S.J., & Wandersman, A. (2015). Empowerment evaluation: Knowledge and Tools for Self-assessment, Evaluation Capacity Building, and Accountability, Second Edition. SAGE Publications Inc., USA.
Three simple steps!
The key to making empowerment evaluation work is to keep it simple and include everyone. For this reason, Fetterman is adamant – no more than three steps!
Step 1 – Establishing the mission
The process begins with defining the organisation’s mission. It’s important to take time with this step because it is crucial for buy-in.
Have everyone call out what they think the organisation’s mission is. Write everything down and then synthesise to build towards consensus. After the workshop, it is important to write this in the form of a short paragraph, and you may even wish to publish in other forms (e.g. a pamphlet).
Step 2 – Taking stock
The second step involves identifying the key activities that contribute to the organisation achieving its mission, which forms the basis of what will be evaluated. Sound difficult? Fear not, all you need is some butchers paper, a marker and a roll of sticky dots.
During this step, facilitating discussion between stakeholders – no matter how differing their perspectives may be – is vital for disrupting the power dynamics in the room and reaching consensus. There are two parts to this process.
- Prioritisation: Stakeholders start by listing all key activities on butchers’ paper. They are then given five stickers each to place next to the activities they think are most important to achieving the mission. The 10 activities with the most stickers will inform the next part of the process.
- Ratings: Part two requires stakeholders to individually rate how well the organisation is doing on each activity. These ratings are written up under an individual’s initials for everyone to see. It is then up to the evaluator to prompt people about why they have given these ratings, encouraging debate among stakeholders. Once everyone has had an opportunity to adjust their scores, the evaluator calculates overall ratings by activities and individuals, and the group decides on three activities to evaluate.
Step 3 – Planning for the future
The final step requires stakeholders to set a goal for each activity, develop strategies for achieving that goal, and identify the evidence required to track achievement.
Once this process is complete, a simple dashboard can be set up that records the goal, milestones, actual performance and the baseline figures. It is then up to the evaluator to identify data collection tools, and build the capacity of stakeholders to use them.
Top tips
We walked away from the workshop with many learnings for our ‘tool belt’, but these our top six tips.
- Inclusion! Inclusion! Inclusion!: While you should aim to incorporate as many of the key principles as possible when doing Empowerment Evaluation, including everyone in the process is key! To emphasise this point, Fetterman tells a story about Albert Einstein – well, an actor that looked like Einstein – who he hired to first introduce the concept of empowerment evaluation 21 years ago. As the actor’s microphone malfunctioned and Fetterman hopped around the stage behind him holding the plug in to make it work, all seemed lost… until one man in the audience – the one others didn’t want to invite – got up and saved the day. And that, Fetterman argues, is why you include everyone.
- Evaluator as a coach or ‘critical friend’: Don’t assume all stakeholders have the same skill sets or experience under their belts – take the time to adopt a process that allows capacity building so stakeholders can walk away and apply these skills beyond the evaluation (e.g. building graphs in Microsoft Excel).
- Take your time while maintaining stakeholder engagement: “In empowerment evaluation it is important to work as inefficiently as you can” – this quote from David highlights the importance of spending adequate time as a group to scaffold the core mission and take stock. This will save time in the long-run and is crucial for creating buy-in during the initial stages of the evaluation! However, remember to not conduct the three steps too far apart as this can impact on momentum and the interest of stakeholders.
- Reframing the way organisations think about evaluation: Make routine data collection useful! Institutionalise evaluation within the organisation – for example, present findings regularly in staff meetings instead of waiting for special findings meetings that often occur less frequently.
- Address weaknesses while building on strengths: When planning for the future, each strategy should be tracked through measures that visually illustrates current levels of progress (e.g. bar graphs) in improving in both areas of weakness and strength. This way, stakeholders can monitor progress easily, motivating them to stay accountable for reaching goals.
- All tools should fit the empowerment perspective: Data collection tools should be free to access or inexpensive, easy to use and easy to disseminate and modify.
What’s next?
Parallels can already be drawn from the 10 principles of empowerment evaluation and applied in our work with Aboriginal people, communities and organisations. We commonly utilise capacity building, community ownership and knowledge and inclusion to make sure the voice of Aboriginal people and communities are heard and embedded in program design, methods, development, implementation and reporting. An example of this is our work with community and Aboriginal sectors across NSW, as we work with them to build program evidence and capacity within Aboriginal Community Controlled Organisations as well as embedding evaluation tools into programs to foster an environment of program improvement.
In our work in the disability sector, we often take participatory and capacity building approaches to evaluation. This workshop has illuminated innovative ways to adapt these approaches – particularly in the design phase – where agreeing on a mission and taking stock have the potential to change the way people conceptualise program logic.
If you are interested in finding out more about Empowerment Evaluation, then you can order a copy of Fetterman, Kaftarian & Wandersman’s Empowerment Evaluation: Knowledge and Tools for Self-Assessment, Evaluation Capacity Building, and Accountability Second Edition from www.footprint.com.au. The second edition celebrates 21 years of empowerment evaluation practice. Fetterman explores the 10 principles, models and tools he has used through case studies of a range of diverse projects he has worked on over the years.
