See you at AES26!
We’re thrilled to be attending this year’s AES Conference in Garramilla (Darwin)! AES is a highlight in the ARTD calendar and we’re looking forward to catching up with new and familiar faces, whether it’s at one of our presentations, by our booth, or at one of the many social events throughout the conference. Be sure to stop by our booth for a chat, some fun activities, and a chance to put your problem-solving skills to the test. There will even be prizes up for grabs!
You can read details on our sessions and register by clicking on the links below:
The case for key monitoring questions in simplifying performance frameworks with Jack Rutherford
Have you ever felt that the Monitoring and Evaluation (M&E) Framework you developed or had to implement was potentially too complicated? This presentation introduces Key Monitoring Questions (KMQs); a novel approach that builds on the well‑established logic of Results‑Based Accountability (RBA) to simplify M&E frameworks while strengthening their usefulness for decision‑making and reflection. Instead of attempting to measure every activity, output or incremental outcome, KMQs concentrate attention on purposeful questions drawn from RBA that the monitoring system must answer. This shift helps teams focus on what matters most by clarifying the purpose of monitoring and identifying the minimum information required to support meaningful learning.
The Evaluation Pub Test: Simple Questions for Public Value with Amanda Taylor-Short
Publicly funded evaluations carry an implicit responsibility to contribute to public good. In practice, evaluations are often viewed as mechanisms for accountability in government decision‑making. This can reinforce a focus on compliance, program‑specific detail, and predefined questions, limiting broader reflection about the contribution of a program and leaving simple but important questions unexplored. This creates a missed opportunity to connect evaluation work with the people and decisions it exists to inform. This presentation explores the Evaluation Pub Test: examining evaluator responsibility to support public value, the benefits of answering simple questions, and considerations for deciding which questions are worth answering.
What works for whom? Developmental evaluation of a domestic violence prevention pilot for young men with Simon Alaba
Primary prevention programs targeting boys and young men at risk of using violence operate in a space where the evidence base is still developing, and knowing what works, for whom, and in what circumstances, is far from settled. This presentation draws on 3 years of developmental evaluation of a primary prevention pilot to share three interconnected findings at the intersection of evaluation practice and implementation science. Drawing on findings across multiple pilot phases, we discuss the tension between adapting to improve engagement and holding firm on program fidelity when discomfort is itself part of the change process. We close by examining what these findings mean for how evaluators interpret and communicate success in primary prevention settings, where uniform outcomes are neither expected nor realistic, and where the most meaningful impacts may be concentrated among a subset of participants.
Attendees will leave with practical insights applicable to developmental evaluation, pilot program design, and the evaluation of complex social programs more broadly. The session will close with audience discussion that invites participants to reflect on how these findings apply to their own evaluation contexts.
Monitoring on a shoestring: How reproducible reporting can help make better use of data with Dr Stephanie Quail, Sophie Henness
Non-government organisations (NGOs) and small organisations often face a range of barriers, such as limited funding and staffing, to collecting, analysing, and utilising client and program data to support continuous improvement and decision making. For these organisations, taking an empowerment evaluation approach, which enables them to assess and improve their own programs while building capacity and ownership, has the potential to provide sustainable and ongoing value. Empowerment evaluation strengthens organisational capacity and equips organisations with tools to collect, analyse, and interpret data for ongoing program improvement. This presentation explores low-cost and high-impact opportunities for embedding monitoring, as demonstrated by our recent work with Bonnie’s Support Service to develop internal capability to administer and analyse a new client satisfaction survey.
Audience members will leave this presentation with an understanding of common challenges to setting up and analysing client satisfaction survey data, the advantages and disadvantages of available open-source platforms, and practical approaches for setting up reproducible analysis and reports. Attendees will receive a free online toolkit we have developed which organisations can use to guide the development of their own low-cost monitoring and reproducible reporting.
Who defines a ‘professional’ evaluator? Roots and routes across government reform and the evaluation field with Jade Maloney, Eleanor Williams (ACE), Liz Wren, Ruth Nicholls, Tony Kiessler, Nige McPaul
In an era of public sector reform, the evaluation profession is being reshaped from multiple directions. In Australia, this is occurring through two intersecting pathways: the Australian Public Service (APS) Evaluation Profession, established as part of broader APS Reform; and the ongoing question of professionalisation within the Australian Evaluation Society (AES) and the evaluation field more broadly. This panel explores how these two “routes” to professionalism interact, reinforce and sometimes challenge one another. The discussion asks how evaluation can remain grounded in its core professional principles of rigour, ethics, cultural responsiveness and learning; while adapting to new institutional expectations, roles and accountabilities.
Rather than assuming these perspectives naturally align, the panel will surface key tensions and questions. Who defines what “good” or “professional” evaluation looks like? How do institutional reform agendas interact with professional norms developed within the evaluation community? What happens when professional judgement, independence or methodological standards are tested by political urgency, contested evidence or strong beliefs? And how do unconscious biases and assumptions shape whose knowledge is valued and whose evidence is trusted? What might be lost with professionalisation – in particular diversification of evaluators and building evaluative thinking into all types of roles – and how we might avoid this.
Evaluation professionalisation: where to from here? with Jade Maloney, Eleanor Williams
Did you grow up wanting to be an evaluator? For most of us the answer is no, and there is value in the way we have all fallen into evaluation profession from diverse backgrounds. But what does this mean for our career pathways and the way others think of evaluation and evaluators? What does it mean for evaluation as a profession?
The 2024-2028 AES Strategic Plan firmly put considering pathways to professionalisation back on the agenda. A working group has since been exploring options for professionalisation, drawing on learnings from the review undertaking by Peersman and Rogers in 2017, consistent with the AES values, and informed by the experiences of other evaluation associations and other professionalisation associations in Australia.
Now it’s time to seek your views because professionalisation is for you, the Australian evaluation community. Harnessing the success of the fishbowl approach to exploring quality evaluation that makes a difference at the 2025 conference, we are inviting everyone to jump in and share their thoughts about the value professionalisation would provide, and the risks to be managed to ensure the pathway are accessible, inclusive and respectful of diverse ways of knowing. The working group understands there are diverse views on professionalisation and invites these to be surfaced in this conversation. And yes, there will again be chocolate fish rewards for contributors.
Just think: What would it look like if the next generation could grow up wanting to be an evaluator? What if when you became an evaluator you could see a clear pathway forward?
Embedding lived experience: strengthening the evaluation of Australia’s Disability Strategy with diverse ways of knowing with Fergus Bailey, Jade Maloney
Australia’s Disability Strategy 2021-2031 emphasises the importance of inclusive practices across all sectors, including evaluation. ARTD Consultants is conducting the mid-term evaluation of Australia’s Disability Strategy 2021-2031, using the guide and their own lived experience evaluation framework for a lived experience centred approach. This a significant test of the guide in action. In this panel, DHDA staff, Jade Maloney (CEO, ARTD Consultants), Fergus Bailey (Senior Consultant, ARTD Consultants), and lived experience representatives will discuss how the perspectives of people with lived experience were centred throughout the evaluation – in key evaluation team roles (including ARTD staff and a lived experience team), governance roles (e.g. Steering Committee), and consultation with people with disability across Australia.
They will provide perspectives as commissioners, evaluators, and people with lived experience on how the guide was operationalised for this process and the benefits to the evaluation. They will share practical advice for how evaluators can effectively engage people with disability in leading and contributing to evaluations, which are applicable to engaging people with lived experience more broadly in evaluation governance, design and delivery roles, including tailored approaches for diverse communities.
Learnings from the field: why prisoner voice matters in the evaluation of criminogenic programs with Paula Shaw, Syl Johns
Incarcerated people are (almost by definition) excluded from public discourse. Our criminal justice system offers imprisonment; a deprivation of liberty, which includes severe restrictions on a person’s ability to communicate with the outside world, as its most common consequence for committing a crime. This session, targeted at intermediate evaluators, explores the ethics of evaluation in the corrections context and the importance of including prisoners’ voices. Over the last five years, the presenters, (ARTD Associate Uncle Syl Johns, and Senior Manager, Paula Shaw) have worked on several evaluations of criminogenic programs, and between them, have interviewed well over 100 prisoners across Qld, and in SA, NSW, Victoria, WA and the NT. This session will be delivered as brief presentations and facilitated group discussions on each of the topics below:
• The ethics of inclusion of prisoners in evaluation projects – why it matters, what are the power dynamics at play, what are the risks for prisoners, and what do they get out of it?
• The formal ethics processes involved – what are the key considerations?
• Learnings from our work about practical approaches to engaging prisoners in evaluation interviews. – What has worked well? What hasn’t – and why?
Registrations are still open! Register for AES26 here: https://www.aes26.aes.asn.au/conference-and-workshop-registration
