Thoughts on Michael Quinn Patton’s Criteria for Evaluating Transformation
Michael Quinn Patton’s recently published ‘Evaluation Criteria for Evaluating Transformation: Implications for the Coronavirus Pandemic and the Global Climate Emergency’ is chock full of pithy quotes like “Evaluating transformation means transforming evaluation”[1].
In the article, Michael defines transformation as ‘interventions which create momentum towards tipping points into deep, sustainable systems change.’ Since the pandemic began, we’ve observed the incredible phenomenon that suddenly everyone, everywhere, can more clearly see societies’ systems, the ways they interact and intersect (and don’t). The pandemic has made clear to most that major systems transformations are needed, are happening, and must continue to happen.
The article aims to provide a ‘floor’ of evaluation criteria, upon which evaluators can build a solid, contextually adapted structure, capable of evaluating transformation. Criteria, he writes, are essential to allowing evaluators to render judgements, operationalize what is valued, and direct our line of questioning. [1]
Quinn Patton sees evaluation and evaluators as being part of the transformation process. His article critiques the intentionally technical and neutral qualities of ‘universal criteria’, such as the OECD DAC, and advocates that in our ‘post-normal’ world, evaluators are a part of the solution (to the problem of continued survival of humanity on this planet), and we must engage with ‘the scale, scope, and urgency of transformation.’ [1]
Evaluating transformation requires us to mentally switch from evaluating an ‘intervention’ to evaluating a ‘sensitizing concept’ (one ‘that has to be given meaning and specificity within the context where transformation is targeted.’[1]) It also requires use of evaluation criteria that are explicitly values-based, systems and complexity-informed and which take a particular ethical stance – something that may challenge the traditional conceptualisation of evaluation as an ‘independent technical activity’[1].
The criteria Quinn Patton has developed hinge on:
- measuring along a trajectory toward transformation (rather than focusing on arrival at the transformation destination)
- incorporating systems thinking and complexity concepts
- measuring the full costs and benefits in economic, social, and environmental domains, including externalities
- situating humans within environment and placing emphasis on adaptability as critical to resilience and sustainability
- centering diversity, equity and inclusion in evaluation practice, design, and management
- identifying interconnections within systems and the momentum towards transformation they can create. [1]
Earlier this year, ARTD Partner Andrew Hawkins shared his thoughts with our team about evaluation’s relationship to time and how this also relates to systems thinking [2]. This is also obviously at the heart of Quinn Patton’s thinking on transformation. I’ll leave you with a beautiful quote from the article on this subject:
“Normal evaluation relishes the lessons that flow from hindsight; however, alluring,and seductive but ultimately ephemeral, such lessons are generalizations that decay more rapidly than the half-life of radioactive particles. Evaluation for transformation requires expanding foresight evaluation capacity grounded in the ethical responsibility of having skin-in-the-game (Patton, 2019b, 2020; Taleb, 2018) because we are all affected by how, and how well, humanity endures.” [1]
Resources
- Michael Quinn Patton (2020). Evaluation Criteria for Evaluating Transformation: Implications for the Coronavirus Pandemic and the Global Climate Emergency. Available at: https://journals.sagepub.com/doi/abs/10.1177/1098214020933689?journalCode=ajec
- Evaluation Concerns – past, present or future – subscribe to view past newsletters – https://www.artd.com.au/read/newsletters/
New to systems thinking? Check out these articles.
