AES Panel: the role of evaluation in the APS reform agenda
The recent elevation of evaluation in the national dialogue – through the announcement of an Australian Centre for Evaluation in Treasury presents an important opportunity for evaluation.
At the very least, we evaluators may get fewer blank looks when asked what we do at dinners and events. But it also has the potential to enhance the use of evaluation and enable evaluation to deliver on its promise to enhance the value of government initiatives to individuals and communities.
This was a focus of the recent Australian Evaluation Society panel on the role of evaluation in the APS reform agenda, with Suzanne Butler from the Department of Finance, Harry Greenwell from Treasury, Wendy Jarvie –Adjunct Professor at the Public Service Research Group at the University of Canberra, Dr Russell Ayres – Associate Professor at the University of Canberra, and Professor Janine O’Flynn – Director of the ANU Crawford School of Public Policy.
While the headlines have focused on the need for more randomised control trials (RCTs) – and there is a place for these in impact evaluation – there is recognition that these will not answer all evaluation questions.
There is also recognition, that to make a difference, the Centre cannot only focus on the supply of quality impact evaluation, it also needs to support evaluation capability building across the public sector. As Wendy Jarvie pointed out, this will require providing not only training but also opportunities to be involved in and do evaluation. This reflects our experience at ARTD that training only sticks when there’s a hands-on element (like mentoring to complete an evaluation), and evaluative practice is embedded in day-to-day roles and responsibilities.
The panel discussion explored how we can integrate evaluative thinking into core business. While there will be different views on whether and when to institutionalise requirements for evaluation, for evaluation to be embraced as more than a tick-a-box exercise, evaluators and evaluation champions need to understand and enable senior leaders and staff to see what’s in it for them. As Jarvie noted, evaluators can get caught up in seeing evaluation as a technical exercise, while senior leaders will see it as a political act. It’s also a human one that is intertwined with people’s values, hopes and fears. I’m grateful this is something AES fellow and ARTD founding Partner, Chris Milne, taught me early in my career. We need to embrace the technical, political and human dimensions to build a culture of evaluation.
Lastly, we need ways to track progress. As Professor Janine O’Flynn highlighted, we don’t have a good picture of public service capability or a good track record of evaluating reform. This will be a complex undertaking. But one tool that may support this is an evaluation maturity model to enable organisations and teams to track where they are in terms of evaluation capability, identify priority actions and track progress.
I look forward to continued conversations as the Australian Evaluation Centre is established.
