Day 1 at AES25 in Canberra
Several of our staff are at the AES Conference this week. Whether you’re attending and want an overview of sessions you couldn’t get to, or you weren’t able to make it and are having FOMO, we’ve got you with our daily recaps.
The first thing I probably should say is I might be an AES conference old hat. It feels like a school camp getting to come together and see old friends and new, and hear new ideas, celebrate the wins and share the struggles.
The welcome drinks at the National Portrait Gallery were so joyful – though so many evaluators together makes for a loud time. We are a rowdy chatty bunch! My face hurt from smiling so much.
I personally loved seeing the oil paintings. I’ve been doing a course lately to try and learn oil painting and think there might be an analogy about moving paint around that doesn’t want to stay put (oil paint is tricky y’all), and trying to capture what we see when the lines keep blurring. There’s something comparable there too between artists all having a different subjective view and what we try to do as evaluators, what we grapple with and how our work changes.
Jade Maloney, Brad Astbury and Sarah Schofield at the National Portrait Gallery Welcome Drinks
This morning, the highlight for my kids was buffet breakfast – they kept saying ‘this is the life!’. It was a good reminder of enjoying the little things! I got to do a brief wander through the exhibition hall and see the amazing spaces we have all created. I felt a genuine sense of pride in having our ARTD booth this year and knowing how kind and lovely everyone is.
Aunty Serena Williams’ Welcome was warm and generous to Ngunnawal country, this meeting place for AES.
Kirri Parata and the conference committee introduced the themes of beyond the bubble, shifting perspectives and changing paradigms. The piece of my role as an evaluator I am most proud of is this ongoing challenge and reflection on our role and selves.
Liz Wren introducing Gilibanga was really exciting I resonated deeply with what she said about getting rid of the phrase ‘using a First Nations Lens’. It’s not about the lens, it’s about changing the whole game, through storytelling, and coming together, and also in shifting and challenging.
Andi Fugard (they/ them) kicked us off with the first keynote. They began by challenging and unravelling the artificial binaries we make in evaluation and in our judgements.
As the lucky person I am to love a non-binary person (my partner) I feel blessed to live a little outside of some of our gender binaries and our notion of family and roles. This greatly impacts my experience as an evaluator and how I navigate my work, parenting, and life. So I reflected a lot on that in the opening session.
Andi started with Quant and Qual, and how this is less of a binary than we think: that it is not so simplistic. They moved on to the counterfactual and non-counterfactual ‘binary’, and how this can be tested beyond an RCT. They finished with a discussion about the subjective vs objective binary. This one was fun! Andi looked at the social construction of the speed of light over time and how this changed and ‘the defensible, yet subjective analytic choices that influence all research results (Silberzahn, et al 2018).
Andi finished by noting also that there are gender and racial biases in data and a privilege hazard that needs to be understood.
Andi challenged us to think as evaluators: can we mess up our toolbox, can we find different ways to conceptualise and think and use different methods to better serve our community?
Session time
In the morning concurrent session I was lucky to hear from: Monica McKenzie and Jonathan Mcquire on the Connected Community, a culturally responsive evaluation with NSW Education – a process that centered on relationships and families – that looked at a pilot of support for First Nations kids in schools and how to evaluate to include First Nation and mob in the entire process.
Some of their techniques and things that stood out were meeting on Country, connecting in place having a cuppa. It made me reflect on making time to Yarn, and to acknowledge the nature on non-linear conversation in evaluation.
They also talked about the notion of healing through evaluation for people and community when schools have historically done harm to kids and families.
The report uses QR codes to link to videos that tell the story of relations with school and some of the themes from their culturally responsive evaluation framework. The emotional impact evaluation was also explored and it was a good reminder of the possible impacts of evaluation. All recommendations were accepted and the strategy was refunded for a decade – the school lifetime of some of the kids involved in the pilot and evaluation. They had to do 4 print runs because the demand from the school communities was so high, and the reports were displayed in some schools as well. Read the innovative and very successful report here.
Our very own Kate Sunners spoke about logical fallacies and logical leaps. She discussed Tyson Yunkaporta and using logical fallacies to code chapters in his own work in Sand Talk. This talk also had cookies so you gotta love that. The exploration of what logic is, the links between premise and claim, and logical inference is just the sort of nerdy exploration of our work that we all love. I love a talk with a reading list too. My next read is The whole elephant: Defining evaluation by Amy Gullickson.
I had the pleasure of chairing the Concurrent Session Royal Theatre with Gerard Atkinson, Founder and Managing Director of Iris Ethics. Nothing starts a session like a funeral for evaluation as is it stands – complete with Latin eulogy from Gerard. He then led us through a Bohmian dialogue on the future of evaluation in the face of a growing role for AI. Bohmian dialogue is about people suspended certainty and working with each other. We talked through three possible AI led evaluation methods: evaluator free evaluation, where a program staff evaluate their work through AI; Evaluand free evaluation, to use synthetic connections; and evaluation free evaluation – a system that evaluates itself and changes.
The ARTD booth was buzzing in the break with many people coming to take a photo and connect in person and on our connections wall.
The closing plenary discussion was an interesting panel on evaluation in polycrisis with Brad Astbury, Eleanor Williams and Patrica Rodgers. The discussion was on the important and oft discussed (at AES at least) topic of the methodological change needed for evaluation to grow and stay relevant in polycrisis. Brad spoke about the connected and complexity of the policy issues the world faces and the risks to evidence and truth telling. Patricia Rodgers spoke about equity and environmental outcomes as a focus of evaluation activity we have a role in advocating for the use of evidence we should be active participants in calling this out. Eleanor Williams talked about getting more visible as a profession and being more embedded in process.
The last session I attended was on Reclaiming Talanoa: Pacific Perspectives on Indigenous MEL approaches with Allan Mua Illingworth and a Panel of other Pacific islander researchers. This was a new way for me to look at Yarning, and using talking as an evaluative approach with no preconceived idea about where the evaluation takes you. The panel spoke eloquently about the power of this technique and also how to ensure it is not misused and appropriated or over simplified, which risks it losing its power and reciprocity.
The evening ended for me with my kids and partner enjoying the String Theory Booth, having a chat about evaluation approaches, playing guitar, and having fun. It’s always nice to feel your whole self is welcome at AES!
