Building capacity for evaluative thinking
I reckon the right time to make resolutions isn’t amidst the buzz of New Year’s Eve, but when the fireworks are a dim echo.
So here goes. This year, I’m committing to championing and building capacity for evaluative thinking.
If we’re to believe the hype that we’re living in a post truth world, this may seem like a lost cause. But while many people source their information through the echo chambers of social media, we can take comfort that the Orwellian concept of alternative facts hasn’t caught on.
Also in our work in evaluation, we come across plenty of organisations and stakeholders with a commitment to collecting, reviewing and making decisions based on evidence. While there is often a gap between rhetoric and practice, evidence-based (or at least evidence informed) policy is engrained in the lexicon of Western democracies.
The trouble is that evidence informed decision making can seem out of reach if evaluation is presented, in difficult to decipher jargon, as the remit of independent experts. (Of course, this is not the only trouble. In some cases it is that the commitment to evidence and evaluation is symbolic—to give an impression of legitimacy—but that’s not the situation I’m thinking of here or one that I come across very often).
This is not to say that there is not real expertise involved in evaluation. But if we can’t translate this into language and ways of working that all stakeholders can understand, and then bring them along on the journey, evaluators will be speaking into their own echo chamber.
And—as is clear from the literature on evaluation use (including my own study with Australasian Evaluation Society members)—if we don’t involve stakeholders throughout an evaluation, then it’s unlikely to be used either instrumentally or conceptually.
Focusing on building capacity to think evaluatively (rather than just capacity for evaluation) can help put informed decision making within reach.
This focus fits with the concept of process use (see Schwandt, 2015), which evidence shows can be linked to direct instrumental use of evaluation. It also supports sustainable outcomes from interactions between evaluators and stakeholders.
But what does building capacity for evaluative thinking mean in practice? For me, it means not only focusing on the task of the evaluation at hand or building capacity for evaluation activities, such as developing program logics and outcomes frameworks, but on engaging stakeholders in the practice of critical thinking that underlies evaluation.
As Schwandt (2015) describes it, critical thinking is a cognitive process as well as a set of dispositions, including being ‘inquisitive, self-informed, trustful of reason, open- and fair-minded, flexible, honest in facing personal biases, willing to reconsider, diligent in pursuing relevant information, and persistent in seeking results that are as precise as the subject and circumstances of inquiry permit.’ And its key application in evaluative practice is in weighing up the evidence and making value judgements.
We can crack open this process by engaging stakeholders in it. We can also translate the process into an equivalent in everyday life (for example, using value criteria, such as price convenience, quality and ambience, to make a reasoned choice between different restaurants). This might even help people to understand how others come to different conclusions based on different value criteria.
The more often this happens, the less we may need to worry about echo chambers.
