AES Conference – Recap of Day 2
Day 2 of AES26 kicked off with an insightful keynote from Emmanuel Jimenez from an international development perspective, focussed on putting money where the evidence is: something that’s particularly vital in today’s climate of constrained funding. Emmanuel emphasised the importance of embedding evidence early, even before project design in the diagnostic phase, to determine where a project might be best placed e.g. in which location or sector. I’ve often heard evaluators speak about the importance of embedding evaluation throughout programs, but this made me reflect on how evaluation could be brought in not just from the point of design but from an idea’s inception.
Next, I attended a presentation by my colleagues Simon and Abel on their recent developmental evaluation of a domestic violence prevention pilot for young men. Although I work with them, I continue to be astounded at the talent and knowledge of my colleagues, and it was great to be able to hear about their learnings from the project. I learned about the importance of the ‘program-participant fit’ and some of the challenges in recruiting participants who have a need that the program is targeting and who are also willing to participate in a program that challenges their views.
I took a break to soak up some Garramilla/Darwin sunshine and practice my own presentation (more on this later!). After recharging, I joined a round table discussion on navigating media scrutiny as evaluators where we joined forces to brainstorm strategies for situations where media narratives move faster than evaluation findings. The strategy my table came up with focused on communicating what was intended, what has been accomplished, and future plans of the program and evaluation to maintain credibility and trust. The discussion made me think about how important it is to communicate purpose, progress and future directions clearly and proactively, especially with the rapid pace of social media and news cycles.
After lunch, I attended a skills building session on process tracing. Process tracing is a theory-based evaluation method that tests whether the evidence found is consistent with either the hypothesised theory of change or counter-factual pathways to change. It can be paired with contribution analysis or quasi experimental studies, and can help to reduce evaluator confirmation bias. It goes beyond contribution analysis by making explicit statements about what else could have produced the outcome and testing for this in data collection. I had a lot of fun doing some process tracing using different pieces of evidence, but it was definitely a brain exercise. The session pushed me to think more critically about how we test theories of change, and the importance of challenging our own assumptions by considering alternative explanations.
Then it came time for Steph and I to present our guide to reproducible reporting. Sharing our work and toolkit on reproducible reporting in low resource settings with a wider audience was a rewarding experience, and I was grateful for the opportunity to contribute some guidance and considerations for those thinking about implementing this for their programs or organisations. If you’re interested you can find our monitoring client satisfaction toolkit here Monitoring Client Experience on a Shoestring: Toolkit and an example of what reproducible report code looks like here ARTD GitHub.
After a big day of learning and presenting I am looking forward to an evening under the stars at the AES gala dinner!
