The many ways we can un-box evaluation
To outsiders, evaluation can seem like a mystery and something to fear. While evaluators see its potential to inform better public policy, the evidence says evaluation does not always live up to it’s potential.
This is part of why our theme for this year’s Australian Evaluation Society Conference in Sydney is ‘Evaluation Un-boxed’.
I shared my initial thoughts about why and how we should un-box evaluation in the first of a series of blogs for Better Evaluation. I mentioned the need to open up evaluation to end users and to be open to what we can learn from community, thinking about the basis on which we value and how we engage with lived experience. I also questioned how the idea of un-boxing evaluation fits with conversations we are having about pathways to advance professionalisation within the context of the Australasian Evaluation Society.
Comments from other evaluators reminded me that un-boxing evaluation is also about un-boxing evaluation reporting. How could I have neglected one of my favourite topics?
Then I got to continue the conversation with Carolyn Camman and Brian Hoessler on their podcast, Eval Café. We found many more ways we can un-box evaluation – by translating the jargon, making it meaningful, getting it integrated into practice, and bridging professions. All of this raised big questions about who we are and how we go about our work.
How do we un-box reporting?
Much has been said about evaluation reports gathering dust on shelves. If we want evaluations to be used in a world of competing demand and information overload, do we need to rethink evaluation reporting? The short answer: yes. How? Let me count the ways. At the 2018 Australasian Evaluation Society the conference in Launceston, evaluators came up with a range of ideas for evolving the evaluation deliverable in a session facilitated by my colleague, Gerard Atkinson.
But un-boxing reporting is about more than finding new and engaging formats. It’s about finding ways to enable shared learnings across evaluations of similar initiatives, so that evaluation contributes to the broader knowledge base.
And, for me, it’s also about focusing on the process as much as the product. If you’re engaged and learning on the journey, the report at the end becomes less important. As I learned in my creative writing degree – you can’t control what people do with your writing once you let it out into the world. But if you’re having the right conversations along the way, the product will more likely be based on shared understandings, and useful and used.
Do we all need to call ourselves evaluators to un-box evaluation?
Like most people, I fell into evaluation from somewhere else – communications and creative writing. Not all of us are solely evaluators. And some of us don’t call ourselves evaluators in general conversation.
Carolyn asked if this is a problem? My first answer was yes – even though I’m one of those people that doesn’t call myself an evaluator – because if we’re not out there promoting what we do, how will people know the value of evaluation in a world where co-design, behavioural insights, implementation science, social impact measurement, and customer experience are on the rise?
But then Carolyn questioned whether this is placing too much emphasis on the “evaluator” instead of “evaluation”. Perhaps.
If the end game is integrating evaluation into practice, not everyone involved in the ‘doing’ of an evaluation would be an evaluator. They would just need to think evaluatively. But what does this mean for professionalisation?
Bridging or un-boxing?
Professionalisation could help with coherence and communication – enabling outsiders to better understand what this thing we call evaluation is. However, as Patricia Rogers and Greet Peersman found diverse competencies are required for different types of evaluation. Pinning us down is not so easy.
As evaluators, we need to draw on myriad skills – from facilitation to statistical analysis. Paul Kishchuk has suggested we look at evaluation as a bridging profession. Brian tied this back to bonding and bridging ties. What professions do we draw from as well as inform?
Continue the conversation
There’s a lot more to un-boxing evaluation and we’re keen to continue the conversation.
Tune into the Eval Café podcast.
Submit a presentation proposal by March 7 and join us in Sydney in September.
Join the conversation on the Better Evaluation blog.
