New frontiers for BI: beyond scaling
For those catching the train in Sydney, you may have noticed some new billboards on your morning commute:
“9 out of 10 train customers pay their fares”.
Fare evasion on Sydney’s public transport network is costed at about an $84 million loss per year. This ad, designed by the Behavioural Insights Unit (Department of Premier and Cabinet NSW), is a clear attempt to reduce fare evasion. Will it work? And if so, how?
In Sydney, the penalty for fare evasion is about $200.[1] I pay about $25 per week on public transport – that’s $1,250 a year – and I am fortunate enough to live very close to where I work. So, my weekly spending on transport is on the low end of the spectrum. I’ve seen ticket officers on my commute twice in the last 12 months. This might tell me that, even if I was caught on multiple occasions each year, I’d be better off financially by not paying my fare.
So why tell people that 9 out of 10 people are paying their fare? Why not a campaign that tells people about the $84 million of lost revenue caused by fare evasion? Why not increase the fine to ensure that people who never pay their fare cannot get rewarded for doing so?
The theory is that we are likely to substitute a rigorous decision-making process, maybe based on a cost-benefit analysis, for a ‘rule of thumb’. We have busy lives and the world around us can be a noisy and confusing place. Using a rule of thumb is going to save us time. When we’re told that almost everyone pays their fare, we’ll probably just pay our fare. Not because we’re concerned about public debt. Not because it’s the right thing to do. But simply because we think other people are doing it too.
The use of social norms to change behaviour is just one type of intervention in a much larger academic discipline: Behavioural Insights (BI). The growing use and future challenges for BI in public policy were discussed at Behavioural Exchange 2018 (BX18) in Sydney on the 25th and 26th of June.
Welcome to BI
It’s the buzz word. Its big names – Kahneman, Tversky, Thaler and Sunstein have become part of lunchroom chatter in public policy. And it’s winning more and more traction in the public service, with BI units now operating within federal and state governments across the OECD, particularly the UK and Australia.
There’s reason why. BI interventions have produced impressive results from around the world. The UK’s Behavioural Insights Team (BIT), by switching to an opt-out enrolment scheme, achieved a 6.1 million person increase in the number of workers saving into workplace pensions.[2] BIT’s interventions have also led to improvements in the timeliness of people’s tax declarations, leading to significant increases in forwarded payments in the UK and Guatemala.[3] And in a most iconic example, Schiphol Airport in the Netherlands achieved huge cost reductions in bathroom cleaning expenses through the installation of tiny black fly-shaped stickers in the middle of its urinals.
It’s time to go big
There was a general consensus at BX18, including David Halpern (BIT, UK), that it’s time for BI to ‘go bigger’. The new frontier of BI, according to many, was the challenge of ‘scaling up’.
Scaling up is the process of expanding an intervention from an initial successful pilot to a larger population that will achieve the same results as the pilot on a much larger scale. This is extremely difficult, but highly necessary, as the scaling up of BI interventions hasn’t always gone to plan – with promising effect sizes petering out. In Taking Nudges to Scale, Robyn Mildon (Executive Director, Centre for Evidence and Implementation, University of Melbourne) said that sometimes the way in which a program is administered by a particular organisation in its pilot stage can be a key variable. When the program is rolled out elsewhere these administrative variables are overlooked, leading to reductions in effectiveness. Sometimes, interventions can be too complex to either roll from pilot to larger scale within the prescribed budget or to transfer to a new context.
To address the problems of scaling up, John A. List introduced his theory, ‘the science of using science’ in the Plenary on Behavioural Insights in Regulated Markets on day two. His speech involved some complicated statistical concepts, but essentially emphasised the importance of replication. According to List, the issues encountered when scaling-up pilots, especially the unexpected loss of effect size, can be largely nullified by having 4–5 successful replications before scaling up. An intervention retaining strong results across replications can show that interventions are ready to ‘go big’.
This is where the BI experts’ ‘heads are at’ in terms of new frontiers for the discipline. This focus on scale is great, but are there more ways to think bigger about BI?
From programs to systems
It’s a fair observation that the use of BI in public policy so far has been primarily program focused – that is, focused on interventions that attempt to change one particular behaviour or a small set of behaviours in a particular setting (such as a city or region) and/or among a particular population group.
But at BX18, tackling more complex problems was on the agenda. In the Taking Nudges to Scale break-out session, an audience member challenged the exponents of BI to apply their methods beyond programs, to systems. After all, what is the point of cost-effective interventions within the wider context of systemic issues? If an intervention doesn’t address a primary cause of a behaviour, what type of consequences will the intervention have?
A key difference between program-focused and system-focused approaches is scope. NSW Transport’s current strategy assumes that a primary cause of people’s fare evasion is something happening during the decision-making process of whether to or not to pay for a fare. Is this a satisfactory explanation of fare evasion? Maybe people in Sydney are less likely to pay their fares because the cost of public transport, and more generally the cost of living, is high.[4] There might be a larger context to fare evasion, perhaps a deeper systematic cause, that the BI Unit’s new campaign hasn’t taken into consideration. Taking a systems-focused (rather than program-focused) approach means addressing the wider scope of a particular behaviour.
Good intentions don’t always lead to good outcomes
BI interventions, even those that produce positive results, might have unintended consequences when the scope of the intervention is too narrow. An emerging body of evidence from Social Psychology suggests successful nudges can come with secondary unintended behavioural changes that can undermine the value of the intervention.
This phenomenon is known as ‘moral licensing’, and it might have large implications for BI. In a break-out session on day two, Daniel Effron, delivered an interesting presentation about this effect, Morality, Decision-Making and Compliance. The central insight was that when people do good deeds, it can ‘free’ them to do bad things in the future. Effron referenced an interesting study from his career on the left-leaning campus of Stanford University (Effron, Cameron & Monin, 2009). Participants were previously identified as having voted for Kerry in the 2004 election and Obama in 2008. One group was asked who they voted for in 2004, and the other who they voted for in 2008. They were then given a scenario in which they had to decide whether a black or white police officer should be given a job working in a department with a history of racial division. He found that people who said that they voted for Obama were more likely to say that a white officer should be given the job in the police department with a history of racial division. It appeared as though allowing people to endorse Obama gave them some kind of ‘license’ to be racist in the follow-up question.
Unintended moral licensing effects have also been found in environmental conservation campaigns. One study found that a campaign implemented to encourage households to use less energy achieved a reduction in energy consumption of about 6 per cent, but during the same period, household water consumption increased by a similar margin (Tiefenbeck, Staake, Roth & Sachs, 2013).
It’s possible that nudging people to do the right thing may free them to do the wrong thing elsewhere. Maybe encouraging people to pay their train fare might free them to evade the fare on the bus or the ferry. While this doesn’t mean that all nudges are pointless or self-defeating, it could suggest that interventions with a narrow scope – those that target changes in a single or small set of behaviours – may have unintended and counterproductive outcomes.
What does this mean for BI?
This is a tricky question. Testing for unintended behavioural changes during the pilot and replication stages of an intervention is certainly a good place to start. Well-conceived nudges that have been tested for unintended consequences are highly valuable in public policy and should continue to be pursued. What happens if we find sustained unintended outcomes for a particular nudge that undermine the value of the intervention and the targeted behaviour change is really important (e.g. reducing energy consumption)? Can BI still be effective where broader behavioural change is required?
The use of BI to target broader behavioural changes could involve the design of complex interventions that use multiple nudges targeting a range of behavioural changes to achieve broader policy outcomes. If we want to make it easier for people to behave in environmentally sustainable ways, we could use a set of particular nudges to decrease water and energy consumption and increase recycling. I don’t know of peer-reviewed research that has examined complex interventions such as these (please tell me if you do), so I ask your permission to make a few hypotheses…
I’m reminded of something Cass Sunstein said at BX18 – that BI can help us navigate the complex world around us, by making it easier to make decisions that are in our interest, despite the noise. Complex interventions with multiple nudges could have little behavioural impact due to sheer saturation, becoming part of the background ‘noise’ in which people make decisions. For example, ARTD evaluated the impact of a BI-informed trial a few years ago that sought to influence a series of household decisions using multiple nudges. In this case the results showed no differences between treatment and control.
BI units may also look to deliver powerful, ‘one-shot’ interventions that target a wider scope of behaviour; there may already be single nudges that lead to broader behavioural improvements. Unintended consequences do not have to be bad and measuring for them might show unexpected broader improvements for single nudges. Moral licensing is still a compelling issue, but this effect may have its own specific set of conditions that can be mitigated through evidence-based design.
It might also be the case that achieving broader behavioural change requires interventions that activate higher-level cognitive functions that can’t use the cognitive biases on which BI is traditionally based. This could lead to a whole new sub-discipline of behavioural economics.
BI’s experimental approach is having a positive impact on public policy. While RCTs aren’t always appropriate, they place a higher level of accountability on policy makers to question if their interventions are working. RCTs can often be the ultimate test of whether an intervention works. There are some very interesting BI-informed interventions taking place right now, including the expansion (or scaling up) of a program aimed at improving compliance with ADVOs – administered by the Aboriginal Services Unit (NSW Department of Justice) and the NSW BI Unit[5]. And what about the fare evasion reduction intervention from NSW Transport? Will it work?
At ARTD, we are excited to hear the results of these interventions. We have a keen interest in BI and want to see how it plays out in more complex contexts.
