Masterminding an evaluation approach
Conducting a program or policy evaluation is not always a straightforward process. But bringing stakeholders together to discuss and reflect on the best approaches to tackling evaluation challenges can really help.
This is what happened on Thursday 28 June, when evaluators, non-evaluators and other industry stakeholders attended the latest AES Learning Lab in Sydney, focused on Real-Life Evaluation Challenges. Placed into small groups, attendees were asked to dive deeper into a selection of real-life evaluation challenges that they, their organisation or others, are currently experiencing, using the Mastermind approach.
What’s the Mastermind approach?
The Mastermind approach involves one group member briefly explaining their evaluation challenge in 5 to 10 minutes. This might be as simple as seeking clarity on a small aspect of a larger evaluation or as complex as questioning the overall approach. Other members are then encouraged to probe, ask informative questions and make suggestions based on their own experiences, interests and knowledge. Suggestions may be low or high cost––a recommendation to contact ‘person x’ or ‘organisation y’ might be as beneficial as one to make use of a completely new evaluation approach.
While there is no obligation to act on anything put forward, the Mastermind approach aims to open people’s eyes to things they hadn’t considered or connect them to new networks and resources.
What were the results?
For one participant, the process reinforced the value and worth of the approaches and methods that they were already using. Learning that other evaluation professionals would go about things in the same or similar way can be reassuring, given the complexities and challenges involved in evaluation.
Another participant was surprised at the variety of responses and suggestions. Some, she and her organisation had considered, while others were completely new, such as exploring the possibility of engaging further with the AES’s network to increase survey and interview response rates.
Others found the Mastermind approach helped them to think ahead about potential challenges and identify some solutions, such as thinking about program design and data collection early on. This led to further group questions about whether any data collected would be relevant for different parts of the program or if different types of data were required.
The flexibility of the approach appealed to many, as it can be used at any stage of an evaluation, across all sectors and by all levels of staff. It could just as easily take place in workplace kitchen chat as it could in a formal workshop environment.
What’s the use?
At ARTD, we are big supporters of reflective practice and keen to make use of the Mastermind approach. Sharing your challenges and being open to feedback can lead to innovative, practical or useful suggestions that could improve your work. Understanding why things have not worked as expected can also be more powerful than learning from success (read more on how sharing failures strengthens evaluation).
The next AES Learning Lab session will be on ‘Evaluation in complex settings: taking a systems approach with an eye to the United Nation’s Sustainable Development Goals’ and will take place in Sydney’s CBD on 26 July 2018.
