What does it mean when we talk about mixed methods?
‘Mixed methods evaluation’ is a term that gets thrown around a lot, but what is it, and what kinds of questions can it help us answer? This blog series will tackle these questions, starting with what mixed methods evaluation is. Later in the series, we’ll look at a few different ways of mixing data, and showing various ways mixed methods can be employed through a series of case studies.
Evaluators employ an array of methods to help understand and verify ‘what changed’ and ‘how much’ and answer questions specific to your program or intervention. A mixed methods approach uses a mix of both qualitative and quantitative methods to do this.
A TALE OF TWO METHODS
Qualitative approaches refer to ways we try and capture data that is difficult to measure numerically, such as experiences of a program or behavioural changes over time. This might be data that we gather through focus groups, interviews, program documents, literature or observations.
Quantitative approaches refer to ways of capturing numerical data. These methods are scalable, allowing inferences to be generated that can potentially be applied to a larger population. Think of a survey distributed to a large sample – the data from this sample can then be further analysed to identify relationships between program inputs and program outcomes.
THERE IS A THIRD PATH
In program evaluation, it is quite common to adopt a mixed methods approach. This involves gathering both qualitative and quantitative data, combining or integrating the two in an intentional way and then drawing interpretations based on the combined strengths of both data sets to answer key evaluation questions.
For example, if we collected qualitative data by interviewing a small sample of people who participated in a health promotion program, we may learn about why people enjoy the program and how the program works on the ground. However, these respondents may not be representative of the program’s target population, so it would be difficult to generalise their experiences. We could use insights from those interviews to help develop a survey to inform our understanding of the broader pattern of experiences. This is one of the most basic mixed method designs that evaluators use, although there are many more options to consider.
In our next blog in this series, we’ll show you further examples of basic and advanced mixed method design options. We will also discuss key factors to consider when planning evaluations that draw on qualitative and quantitative data to ensure clarity about when, how and why you are mixing methods.
