Reflections of an accidental evaluator
We’re currently looking for a research assistant to join our growing Queensland evaluation practice. This has been a wonderful opportunity to reflect on what it means to be an evaluator. As evaluators, we find that people we bump into socially don’t know what evaluation is. (I’m sure most of us are experts at tackling that conversation at weekend barbeques…You know, the one that starts with, ‘So, what do you do?’ and gets infinitely trickier when you say, ‘I’m an evaluator.’)
As the word implies, evaluation involves systematically collecting and analysing information to make a judgement or assessment about the merit or worth of something (Scriven, 1991), where the ‘something’ is usually a policy or a program funded by a government agency, non-government organisation or philanthropic fund.[1] I’ve found that explaining evaluation in a way that’s consistent with the realist influence of the work that we do at ARTD tends to stick for most people I meet. I explain that my work helps governments understand what works, for which people and why, and that information helps governments know what to do going forward. Usually, I also give an example of the work I’m currently doing.
One of the reasons evaluation isn’t well known, particularly in Australia, is because there are very few opportunities to study it in undergraduate programs. Many evaluators ‘fall’ into the profession by accident. In my case, the data I analysed for my PhD came from a larger evaluation project my supervisor was contracted to deliver. As an academic researcher, I was sceptical of anything that wasn’t a randomised controlled trial. A few years later, once I’d transitioned into the role of early career evaluator, I discovered there’s a critical role for evaluative thinking in translating ‘what works’ in a research context into the real world of complex social systems.
It’s been my experience that evaluation suits people who are insatiably curious about the world around them. Evaluation is an opportunity to learn about parts of the world you otherwise might miss. As an evaluator, I’ve learnt how social housing is allocated, how policies and programs are developed across a range of sectors, how much of our English lexicon derives from the law, why there are different types of power lines and the difference between a syllabus and curriculum.
A few Saturdays ago, I was chatting with a civil engineer over rainbow-coloured cake and sausage rolls at a kid’s birthday party. It was a fascinating conversation, which gave me an insight into his world where the logistics of building a runway at an already-operating commercial airport were front and centre. During the conversation it struck me that the questions we learn to ask as evaluators help us go beyond the surface of something, and really get to the heart of ‘how’ and ‘why’ something works. (This has been an unexpected social bonus!)
At a more philosophical level, evaluation teaches me about something that’s important to me, which is being in service. As an evaluator, I reckon it’s my responsibility to deliver evidence that helps decision-makers make good choices. There’s dignity in doing work that contributes to the evidence base of ‘what works’ in social policy, why it works and for whom. The biggest gift of this profession is connecting with people who generously share their experience of how government programs can be reshaped to better meet the needs of people like them. It’s an absolute privilege to sit as witness to these stories, to fairly and faithfully capture their essence, and share them with our clients to help drive change.
If you’re keen to understand how the world works, and you enjoy being in service, you can find details of how to apply for our research assistant position in Brisbane here.
