
From recovery to resilience: how can evaluation support community resilience building?
As Australian communities are increasingly impacted by extreme weather events and natural disasters, the government and community sectors supporting preparedness and recovery need more accurate information on what works for whom with resilience building efforts at the individual, household, organisational, community, and societal levels.
Context, context, context
As Covid has continuously reminded us, everything in our society is highly interconnected. When disasters happen, they impact not only individual people, natural and built environments, but also the systems connecting them.
And because community resources and networks differ from community to community, what ‘works’ in disaster resilience is highly contextual. A highly localised, place-based response might adequately support one community to recover, but compromise resilience and recovery in another.
It’s for this reason that program designers and evaluators pay a lot of attention to context. In the case of recovery and resilience, programs and evaluations must be informed by the adaptive capacity and vulnerabilities[1] of the local context, for example:
- socio-economic context (e.g. Will people have enough income to allow them to adequately prepare, or afford insurance? How diverse is the community, in terms of languages spoken? How is the community structured, for example, is there a high ratio of older to younger people?)
- infrastructure (What is the current condition of housing, roads, water treatment facilities? How available and accessible are public facilities, supermarkets, emergency services etc.)
- ecology and environment (Are there low-lying floodable areas? How likely is it that erosion will cause secondary damage?)
- institutions (Which government and non-government organisations are present in the community? What is the nature of business and industry?).
The Australian Natural Disaster Resilience Index steps these out further to encompass governance, policy and leadership, and community and social engagement.[2]
There are a range of useful datasets that program designers and evaluators alike can use to understand community context. For example, the Bushfire and Natural Hazards CRC has mapped Australian communities against these indicators in an interactive map, which can provide helpful baseline data.
Making the intangible, tangible
Given the critical role social cohesion has to play in building resilience[3], evaluating resilience often involves measuring the strength of local connectedness, including collaboration between local organisations and institutions.
Many of our projects have involved measuring the strength and maturity of partnerships and collaborations across government, business, and the community sector. We’ve found that surveys are a useful way to gather objective data about the strength of the partnerships that underpin cohesion. We have developed a partnership assessment tool[4] to measure, indicators such as the quality of information sharing, the degree to which learning is shared, and the extent to which resources are shared and activities altered to achieve a common purpose. Enablers and barriers to collaboration can also be explored. This can be supplemented with additional questions that allow us to measure and analyse ‘the bush telegraph’ – the number, strength and direction of networks at an individual or organisational level – and how information flows through these networks.
To successfully measure the disaster resilience of a community (or communities) means understanding who is in the community, and what preparedness, resilience and access to information and resources looks like to them. The local history of natural disasters is also critically important to informing the design and evaluation of resilience programs. For example, the recent Fire to Flourish national survey[5] showed that preparedness and resilience indicators are higher for people who have recently been through a natural disaster, than for those who had not. It also showed that community cohesion is higher after a natural disaster, and (while it declines over the next decade) remains higher long term than prior to the event.
Co-design and co-evaluation can be used to bring people’s lived experience into the design and evaluation of community disaster resilience programs and ensure the program is: addressing strengths and weaknesses in coping and adaptability equitably; harnessing diverse lived experiences of coping through and rebuilding after disasters, and; is culturally appropriate.
Alternatively, the evaluation might work with an advisory group, to bring knowledge of the local context, culture and history, and expertise in emergency management or community development, for example.
At the individual level, measuring the outcomes of preparedness, resilience and recovery efforts can mean gathering data on access to information and support, risk, mental wellbeing, financial stability, community connectedness, and feelings of agency. Online surveys can be a useful way to reach many people in a community to understand these dimensions of preparedness, resilience, and recovery.
Appreciating maturity and growth
Recognising the importance of context, evaluations of community resilience programs must be sensitive to where each community is starting from, and the progress that is made from that starting point over time. Approaches that support measurement of progress, and which illustrate the differences between and within communities are important. For example, defining key stages of resilience on a maturity continuum can help provide a roadmap to monitor individual and collective program impact on resilience indicators over time. We developed such a model for the National Mental Health Plan for Emergency Services Workers.
Understanding the parts of the whole
Understanding how ‘much’ a program (or programs) contribute to community resilience requires an approach that leans into complexity, and recognises the interconnected drivers of vulnerability, adaptability and transformative capability. The approach must also fit the program goals. If it’s delivered with and for the community, for example, a developmental or empowerment evaluation approach might suit. Larger scale programs might be better aligned with realist or systems approaches.
When more than one program is contributing to strengthening community resilience, a rubric-based approach can provide a useful way of bringing qualitative and quantitative program data together to provide a consistent, comparable assessment of program performance. In our past work evaluating the NSW Natural Disaster Resilience Program, we used a rubric methodology to assess the different dimensions of individual programs, and the program overall.
Sharing what’s learned
Perhaps the most important way in which evaluation work can support community resilience is through sharing learnings with participants, and more broadly with the sector and the general public. Ultimately, the more evidence that is available on what works for whom and in what context, the better informed we all are in our efforts to create more resilient places and communities.
For more on how resilience and recovery outcomes can be measured, watch our short webinar.
References
[1] Al Rifat, S.A. and Liu, W., “Measuring Community Disaster Resilience in the Conterminous Coastal United States”, International Journal of Geo-Information. 2020
[2] https://knowledge.aidr.org.au/resources/ajem-apr-2017-the-australian-natural-disaster-resilience-index/
[3] E.g. Plough, A., et al., “Building Community Disaster Resilience: Perspectives from a Large Urban County Department of Public Health”, Am J Public Health, 2013, https://dx.doi.org/10.2105%2FAJPH.2013.301268
[4] Gomez-Bonnet F, Thomas M (2015) A three-way approach to evaluating partnerships: Partnership survey, integration measure and social network analysis. Evaluation Journal of Australasia, 15(1): 28-37.
[5] https://www.monash.edu/msdi/initiatives/reports/out-of-the-ashes-national-survey-reveals-resilience-grows-after-disaster