Seeing the Forest for the Trees: Are we evaluators?
Are we evaluators? We’ve been asking ourselves this question a lot over the past year. So, Naomi and I (Sophie) headed to our first Australian Evaluation Society (AES) Conference in Canberra (Ngunnawal and Ngambri Country) to find out more.
We come from completely different worlds: Naomi’s background is in social and political science and international studies; mine’s in psychology and data science. Neither of us knew evaluation was even a field before we started at ARTD (and honestly, not immediately after either).
Over the past year, we’ve had the chance to work on a wide variety of evaluations across many different sectors. We have learnt a lot about the methods, we’ve contributed to reports and proposals, we’ve sat in on client meetings. But do we really understand the foundations of the profession? Do we feel like we are evaluators? The conference kicked off on a rainy Tuesday evening in Canberra when we walked into the National Portrait Gallery for the ARTD sponsored welcome drinks, looking ahead to 3 days of talks, ideas, and conversations. Hopefully we would have a clearer sense of what it means to be an evaluator…and whether we were one.
Sophie’s learnings
The conference opened with Dr. Andi Fugard whose ‘beyond the bubble’ keynote challenged the binaries that often shape traditional evaluation thinking. It was a call to question the assumptions we bring into our work, and to be vigilant of the complexities that can oversimplify realities. The keynote put me in the mindset of going beyond my own ‘bubble’ for the rest of the conference and to think about the nuance and complexity we face in evaluation and helped me get more out of the sessions that followed.
Matt Healy, Joe Lea, and Taimur Siddiqi from the Victorian Legal Services Board and Commissioner (VLSB+C) presented what I would describe as a standout session on lawyer wellbeing. The session introduced a systems effects method, exploring how lawyer wellbeing is shaped, not as a single issue to be solved, but as something that emerges from a network of contributing factors. One aspect of the session I found particularly inspiring was hearing Taimur speak about how VLSB+C were actively using the findings to provide guidance on where the legal profession can focus their efforts to mitigate the identified factors that affect lawyer wellbeing. It was a great example of the impact evaluation can have when used by decision-makers. It also raised deeper questions: how do we, as evaluators, decide what matters? What shapes the choices we make, our methods, our focus, and our assumptions?
That question came into sharper focus during Brad Astbury’s session on research on evaluation (RoE) and its importance in evaluation’s response to the polycrisis. This made me think about the reasons why we make certain decisions and what shapes our choices of methods. A session from Ruth Aston, Katina Tan, Stephanie White, and Kat Franks discussed the importance of RoE, sharing case studies on how they’ve used it in their own evaluations and the benefits it can bring. Their session reinforced the value of taking a step back to critically think about our own work, not just the policies and programs we evaluate. It was also great to hear about the team’s new repository of open-access RoE articles, which I’m excited to dive into further.
Maia Grange, Naomi Agius, Sharon Marra-Brown, Sophie Henness and Theebana Tharmakumar at the AES
Image owner: Australian Evaluation Society (AES)
Naomi’s learnings
When I first started at ARTD I was ready to dive in and explore what evaluation is. So, when the opportunity came up to go to the AES conference, I felt it would be a good way to discover so much more and consolidate some of the things I had already learnt.
On my first day at the conference, I attended a presentation by Jonathan McGuire and Monica McKenzie on their culturally responsive evaluation of the Connected Communities Strategy. I was initially interested in this talk because I had recently completed cultural capability training with Uncle Syl and Aunty Julie which had prompted me to think about when and how we engage with First Nations communities. That morning we had also heard Liz Wren, the Gilibanga lead, speak about shifting the conversation away from ‘using a First Nations lens’ to coming together and engaging in storytelling and knowledge sharing that places Indigenous people, relationships, culture and ways of knowing and being at the centre of evaluative work.
The evaluation involved mob throughout the entire process and stories were collected on country where people were able to connect and have a yarn and a cuppa. The report shared videos of some of these stories and experiences of Aboriginal people with schools, and of the Connected Communities work, which struck me as something so simple but innovative for authentic and impactful communication. The impact of the evaluation was profound, with all 15 recommendations accepted, and the Connected Communities Strategy was award 10 more years of funding: the entire school life of children in their early years of schooling.
Seeing the impact of an evaluative approach that recognised and understood Indigenous knowledge, culture and relationships was eye-opening. It highlighted for me how important it is to actively listen and build relationships with the communities that are involved in evaluations. It was inspiring to see how good evaluations can make a big impact on communities and policy. After only the first session on day one, I felt motivated to take what I had learnt and apply it to current and future projects at ARTD so that we can improve the ways we engage Indigenous communities and report on their stories.
It was also interesting to hear about the importance of relationship building in the context commissioning work. Rae Fry facilitated a panel on relational commissioning where we heard from people on both sides of the commissioning process. As someone who is still very new to this world, I found it beneficial to understand how complex the commissioning process is and the effort that needs to go into building relationships and making connections before a project has even started.
During the panel, one person reflected on the difficulties in being asked to tender only to be unsuccessful and receive minimal feedback about the submitted proposal. I was aware of tensions in balancing proposal writing with paid work in how I managed my time but hearing people talk about it and the challenges it brings for small businesses gave me a different depth of understanding. I also learnt a lot from the discussion around the relationships between project teams and procurement teams. The client side of this work is unknown to me and hearing about their experiences and the people involved gave me context for understanding our clients better.
In this session I really welcomed the opportunity to hear from people who have worked in evaluation for much longer than I have and learn things from them that I had never thought about before. It wasn’t a discussion I could contribute to, but I was very aware that I was listening to a conversation that is shaping the way the evaluation sector develops. The AES conference helped me fill some knowledge gaps and better understand the kind of work I am involved in. I am pleased to say that I am more confident about my own knowledge of evaluation and some of the ways people a looking improve the profession.
So after all that, do we feel like we can call ourselves evaluators? Well the answer is, it’s complicated. As our colleagues Leah Carroll and Simon Alaba pointed out in their ignite session, evaluators come for all different disciplines. It is clear to us that evaluation is so much more than just an approach to problem solving. The conference was just the tip of the iceberg but now we feel like we know things that we can start applying to our work like building relationships, thinking about problems as part of system and even what our role is as evaluators, as well as chasing our more specific interests within evaluation.
The conference also made us more appreciative of being able to work at ARTD. Watching our amazing colleagues run workshops, facilitate panel discussions and lead presentations, was inspiring. Not to mention, seeing ARTD be presented with the public sector evaluator of the year award for the evaluation of Towards Zero Suicides. We are grateful to be working with such skilful and experienced people and this is something we do not take for granted. We get the opportunity to learn from these wonderful people and grow with their support. So, while we may not feel like we can call ourselves evaluators right now, we think we are well on our way, it’s only a matter of time.
Want to find out more from the AES? Read the presentations here:
https://www.aes.asn.au/annual-conference-papers/2025-aes-international-conference-canberra-presentations
