How can NGOs do evaluation on a shoestring?
We know there are a range of challenges that NGOs can face when it comes to evaluation. With limited time and resources, NGOs are often left conducting evaluation ‘on a shoestring’.
A recent AES seminar in Sydney explored strategies and tips for conducting lean evaluations. The session was well attended by NGOs, academics, consultants and government.
What challenges do NGOs face when it comes to evaluation?
Attendees discussed the challenges their organisations had faced. Common challenges include:
- a lack of time or staffing to undertake evaluations
- limited funding, and limits in budgeting for evaluation in funding applications
- a lack of evaluation knowledge or expertise within organisations, potentially leading limited methodological rigour
- expectations from funders that NGOs demonstrate outcomes, leading to a focus on what was achieved, rather than learnings for improvement
- difficulties collecting evidence beyond outputs, to demonstrate causality or impact
- embedding evaluation processes across the whole organisation, services and programs and not just for funded services.
One attendee likened evaluation to a diet – “We all know it’s good for us, we occasionally do it, but we often end up lapsing”.
And there was consensus in the room on this. Organisations recognised the importance of monitoring and evaluating what they were achieving for continuous improvement, but sought practical advice on how do this, in a context of having to manage other pressing, on-the-ground demands.
What are some strategies for conducting evaluation ‘on a shoestring’?
To offer some practical solutions, Stuart Loveday, Hepatitis NSW CEO, led the AES seminar and discussed Hepatitis NSW’s approach to evaluation. Hepatitis NSW is an NGO providing support and information services for people undergoing treatment for hepatitis B and C, as well as workforce development and education services.
Loveday said their evaluations were driven by an internal culture of continuous improvement, and needing to report on funding KPIs. However, he identified several monitoring and accountability processes that did not fit under their ‘shoestring’ model, such as their Results Based Accountability reporting and external financial audits.
For evaluation ‘on a shoestring’, Loveday stressed the value of collecting feedback on all of their services and embedding this into service delivery, for example, at the end of a call on their info line, with a postage paid survey in their Tx! Mag, and using volunteers to collect paper surveys at the end of community forums. In their evaluation strategies, Hepatitis NSW make the most of volunteers and offer incentives to respondents. They have volunteers complete data entry into survey monkey. They then report back this feedback in staff and team meetings and use it in business planning and for their internal quality improvement.
It’s important to note that while these efforts help embed a culture of continuous feedback and improvement, they rely on a degree of internal evaluation knowledge to be able to assess what caused outcomes and to make evaluative judgements.
Other strategies discussed by Loveday and participants included:
- investing in membership for an online survey platform
- offering small incentives, such as raffles, for providing feedback
- having internal staff lead evaluation efforts
- drawing on volunteers to collect and enter feedback data
- promoting and encouraging feedback as part of service delivery
- embedding culture of quality improvement and stressing the importance of measuring impact among staff
- being clear from the start about what you’re hoping to achieve to guide what data you need to collect
- building partnerships and relationships with people who know about evaluation.
Where does this leave NGOs?
The challenges for conducting evaluation on a shoestring are ongoing. However, at ARTD we strongly believe that there is great value in building in monitoring data collection and evaluative thinking wherever you can. Embedding ongoing data collection and a culture of continuous improvement will help you understand what you’ve achieved and what you can do to improve.
There are also plenty of resources out there to guide small organisations to self-evaluate, such as the Community Development Resource Package ARTD developed with Ability Links NSW. The Package offers guiding principles and practical tips for low-budget evaluations, as well as links to other useful resources.
If you have any other tips or suggestions for conducting evaluation on a shoestring or would like to partner with us for some evaluation mentoring, we’d love to hear from you!
