Capitalising on the (public) good of evaluation
It’s been about two months since the Australasian Evaluation Society (AES) International Evaluation Conference in Canberra. I thought I’d share some of the things that have stuck in my mind since – the ideas that have had a lasting resonance, not necessarily what leapt out to me at the time.
But first, let me set the scene. I hopped on the train down to Canberra – something which I’d not done before. It proved to be a great way to travel – plenty of room to walk about and the space made it easy to get quite a bit of work done! I’ve quickly become quite the fan. The reason I share this with you is that it got me thinking – what an underutilised (quasi) public good the train is. At $60 for a first-class ticket one could almost say it’s ‘the people’s train!’. And yet there were only a handful of other folk on board.
In a way that I hadn’t anticipated, this idea of the public good – or more specifically, of making better use of evaluation as a public good – seemed to be motivating many of the conference presentations. But maybe I should have seen this coming – the public good is a fitting concept for a conference located in Canberra and well attended by public servants.
Referring back to the conference theme ‘capital’, there was a current of concern underlying many presentations about what is needed to better realise the worth of evaluation (often paid for by the tax payer) i.e. to make the knowledge evaluation generates something that is better integrated into decision-making and accountability mechanisms of government, so that its benefits are, ultimately, made social. Let me give you some examples.
- Multiple papers covered challenges related to strengthening the internal evaluation capabilities of government agencies, both in terms of capacity building of staff and creating evaluation systems.
- Evaluation: what’s the use? (Jade Maloney): lessons from AES survey/ interviews about supply and demand side factors conducive to evaluation use.
- Evaluation and the enhanced Commonwealth performance framework (Brad Cook & David Morton): the framework provides an opportunity for evaluation to inform an understanding of agency performance, not simply discrete programs, which may lead to more strategic evaluations rather than just more evaluations.
- Why Australia needs an Evaluator General (Nicholas Gruen): this would be a mechanism to bring ‘thicker’ program insights and practical wisdom into overly ‘thin’ policy conversations, reporting to the Minister (i.e. a way of integrating evaluation findings into policy making) and to Parliament (i.e. a way of bolstering the independence of evaluation and its accountability to the people).
One could say that the underutilisation of the train to Canberra – owing not so much to its quality, but more to a lack of awareness and sub-optimal integration with the wider public transport network – proved somewhat of an analogy to the problem of evaluation use.
This, however, is not an analogy that is easy to push much further (quit while ahead!) without seeming to imply rather unflattering associations between evaluators and out-of-date infrastructure, which would be neither fair nor accurate….
Indeed, the other thing that stood out at the conference is the strength of the evaluation community. In the spirit of heathy debate, there were different takes on how best to build evaluation capital. According to keynote speaker Sandra Mathison, independence from not integration with systems of government is key to the public good of evaluation. In my opinion, it’s likely that there is a considered and circumspect middle ground that will both strengthen evaluation as a public good and support evaluation use by government.
This leads me to a final thought – that one could frame the notion of the public good of evaluation another way. One might think about promoting and cultivating the practice of evaluative thinking, not just in evaluations but in everyday life. We would all benefit from being part of a more reflective and perceptive world in which we apply critical reasoning to the values and assumptions attached to our judgements.
This post is an adaption of a presentation given to the NSW AES ‘Conference Highlights’ Seminar on 2 November 2017.
