In the disability sector, the philosophy of ‘nothing about us without us’ has gathered steady momentum. As evaluators, we need to evolve our methodologies to identify more participatory ways of working and ensure our approaches recognise the expertise of people with disability.
Working alongside people with disability as co-researchers to design and deliver evaluations helps us as evaluators consider the things that really matter to people, ensures that our evaluation methods are accessible, prompts us to think outside the box and, when done right, empowers the people with disability working on the project.
We recently worked with young people with intellectual disability as co-researchers on an evaluation of a project for people with intellectual disability led by people with intellectual disability. During the project, we sat down with one of our co-researchers to reflect on how the project was going – including what had worked/ was working well and what we would do differently next time. Here are our practical takeaways and our co-researcher’s reflections for project designers, evaluation commissioners and evaluators.
DON’T SKIMP ON PLANNING
Planning is one of the most (if not the most) important phases of getting it right when working alongside people with intellectual disability – and especially in engaging them in an evaluation.
Consider the costs
Ensure your budget:
- is sufficient to employ co-researchers – this is best practice to ensure a diverse range of perspectives and enable team members to support each other
- factors in additional time to develop Easy Read data collection tools with the co-researchers
- considers the regularity of the work – for people with intellectual disability, repetition and regular reinforcement are key; if the work is expected to be sporadic, ensure you allocate additional time to each task to reorientate co-researchers to the project and task
- considers the best mode to get different aspects of the work done. Working virtually is lower cost, but some things may have to be done in person, and this needs to be factored into costs, particularly if co-researchers are located in different states.
Be clear in recruitment
- The evaluation commissioner and evaluation team need to work together to develop position criteria (e.g., experience, intersectionality, location) and recruit co-researchers to ensure they are appropriate for the role.
- Position descriptions need to clearly describe the role, level of experience required, training and support provided, time expectation and remuneration so candidates can make an informed decision about whether they would like to apply.
Upon reflection, the co-researcher who worked with us on this project said he preferred regular (e.g. weekly) work as it makes it easier for him to plan his time, finances and know when he will be working.
Spend time on onboarding
- Organise an induction that covers: information about the co-researcher role and project tasks; team member roles and who to contact in different circumstances (e.g. questions related to project work vs. work-related issues; and the process and timing for getting paid).
- Ensure there is sufficient time for people to get to know each other (you need to feel like a team!) and agree on work days that suit everyone – if budget and team members’ locations and preferences allow, it is ideal if this can be done in person.
- Ensure all the information covered in the induction is documented in an Easy Read document for co-researchers to refer back to as needed.
- Consider how familiar co-researchers are with the mode in which you plan to undertake project tasks.
- If you are planning to collaborate virtually (e.g. via Zoom or Microsoft Teams), have they used these platforms before or will they need instructions on how to use them and guidance on virtual etiquette (e.g. muting when others are speaking)?
- Have people attended work meetings before and will they feel comfortable? Before your training session, get clear on the assumptions you may have about the level of work experience people have. Make sure to check in with the co-researchers about this and support them to feel as prepared as possible.
The co-researcher who worked with us on this project reflected that he found it very important to get to know the team, understand what team members roles were and to connect personally.
TAILOR THE TRAINING
Training is key to ensuring co-researchers feel equipped for the role. Ensure training is tailored to the project tasks co-researchers will be working on and their level of experience with these tasks. For our project, we developed the interview guide with the co-researchers to ensure the language was accessible. We also trained them on how to facilitate interviews through theory and practical exercises, like roleplays, where the co-researchers could practice using the interview guide. This allowed the co-researchers to familiarise themselves with the guide, come up with prompts in case people didn’t understand the question and give each other tips on how they asked the questions. It also helped us test whether the questions flowed well.
DATA COLLECTION: DO IT TOGETHER
Prepare the team
Ensure the team is well prepared for data collection.
- Check that meeting information is clear and easily accessible. Our project took place entirely online as we, our co-researchers and the people we interviewed joined from multiple states. We sent out meeting agendas in Easy Read, which included the meeting link, time in different time zones and how long each item on the agenda would take. It also allowed for 10-minute breaks after each hour.
- Check in before interviews. In our project, we met up half an hour before an interview to catch up, prepare together, practice the interview questions again and allow the co-researchers to ask questions. This worked well given the long periods between interviews, and helped our co-researchers to feel prepared and comfortable before each interview.
Time it for success
Make sure that:
- interviews are conducted at times that suit the co-researchers and interviewees
- interviews go for no longer than 1-hour so interviewees and co-researchers do not become disengaged
- 10-minute breaks are scheduled between interviews so the co-researchers can rest and prepare for the next interview.
Do it as a team
To get the best result, it’s key that team members are given roles they are comfortable with. We conducted interviews as a team of three, including two co-researchers with intellectual disability and one ARTD team member. The co-researchers took turns asking interview questions and the ARTD team member was there for support, taking notes and to ask clarifying or probing questions if needed.
Our co-researchers really liked taking turns as they could share the load of interviewing and got similar amounts of talking time.
KEEP REFLECTING AND CHECKING-IN THROUGHOUT
The work on our project took place quite sporadically, which can be the case if there is not a constant stream of interviewees for an evaluation. While this was largely out of our control, we identified some things that could help to ease the time between interviews in future.
Communicating regularly with co-researchers meant that even in times of no work, they feel like a member of the project team and engaged with the project.
Regular check-ins and time for reflection should also be scheduled in to find out if things could be improved. When reflecting on processes with co-researchers with intellectual disability, we have found it essential to ask specific questions without being leading. For example, rather than asking, ‘What do you think about the interview training?’ we may ask ‘Did the interview training make you feel prepared for doing interviews? What helped you to feel prepared/ What would have helped you to feel more prepared?’
Consider other project tasks
If you know interviews will not be conducted regularly, talk to the commissioner of the evaluation to see if there are other project tasks co-researchers might be able to help with, such as analysing survey responses or setting up meeting agendas and Easy Read documents. Of course, it’s always important to check with the co-researchers to see if these are tasks that interest them!
WORKING TOGETHER PROVIDED VALUABLE INSIGHTS
This was a challenging project that gave us a lot of lessons on how we might do things better in future. The whole team got a lot out of this experience – we learned how to make our processes more accessible to people with intellectual disability, the co-researchers built valuable skills they can use in future roles, and we all made some new friends. Our approach also really enriched the experience for people with intellectual disability who participated in the interviews, and helped us interpret evaluation findings from the perspective of people with intellectual disability so they captured what is important to them.
We hope this blog provides you with some ideas about how you can involve people with disability in your evaluations so, together, we can all uphold the philosophy of ‘nothing about us without us’.