Let’s Get Digital
Earlier this year Ken attended the Evaluation Learning Sprint webinar on Digital Evaluation with Dr Samantha Abbato, Director of Visual Insights People. It covered the benefits and challenges of using digital film and other moving elements for everything from data collection to reporting. It sounded so appealing that Kate recently attended the two-day ‘Making it Digital: A practical introduction to using film for evaluation reporting’ two-day workshop with Dr Samantha Abbato and Rosie Pretorius.
This interactive workshop offered a hands-on approach to checking readiness for digital, planning and storyboarding a short film and an opportunity to practice technical and personal aspects of digital interviewing, with helpful workbooks to guide the process. It was great to meet and work with other evaluators from across Australia and learn from their experience too!
The Why
One of the activities we worked on in small groups was to reduce a page long document of information on the benefits of film in evaluation, to a short script for a video, and then to storyboard that out. Building on that experience, I developed a short video outlining some of the key reasons evaluators should consider film part of their toolkit.
The process of pulling this video together, compared to other videos I’ve created more ‘organically’ was so much more efficient, having done all the initial steps of planning the script (which I’ve added to and adapted from the one we worked on as a group) and storyboarding. By the time it came to make the video, I already had a very firm idea of what I wanted and was able to find some royalty-free footage to help get the message across.
With that said, it took me just under an hour to pull this one-minute clip. Developing films as evaluation resources does require effort and time, but it can be very worthwhile!
For more on the benefits of using film in your evaluation practice, you can check out Ken’s earlier blog on Digital Evaluation.
My Key Takeaways
- Recorded interviews can be used as a sense-making tool when watched together as an organisation. They can be used as a tool to reflect on organizational practices and processes, as well as service delivery. This can be particularly helpful when service delivery is happening across a number of locations. It struck me that while asking a number of staff to read an evaluation report and then come together to discuss involves both asynchronous and synchronous communication, watching a video together at the same time and then discussing could offer a much more immediate opportunity to engage in collaborative sense-making.
- I loved the idea of capturing film before and after implementation of a program to measure change, and of using the ‘before’ film as a prompt to reflect on what had changed since (a more in-depth version of photo-elicitation).
- It’s so logical but perhaps one of the most invisible parts of getting a fantastic final product is to plan all elements the final product of a film, including not only what interview footage you need to shoot, but also what images, data visualization, narration, music and other elements you might need.
- My group had some great insights on testing different visuals and other elements with different audiences to check engagement and also accessibility. Background music under narration was given as an example of something that might be extremely distracting for some, making it difficult to take in the actual information being provided.
- In the same vein, it’s possible to have too many images and quick changes in visuals, which distract from the meaning. Probably if I were to make the video above again, I’d think about pacing the narration out a bit better to ensure the visuals didn’t have to change quite so quickly.
- The personal aspects of digital interviewing are really important to get right! And this takes practice. Even for experienced interviewers in our group, there were aspects about interviewing digitally which were novel and threw them off their game. Not being able to give verbal encouragers was very difficult for some! There’s plenty of work to be done before you start rolling cameras in making sure the interviewee is comfortable in their surroundings, and with the process and what it will look like, to ensure you get the quality of footage – and data – you need.
- As with all data collection, using digital media has its own unique ethical considerations. For example, if you are collecting data by recording interviews, you will need to consider the risk of identifying participants in that footage. You will also need to ensure the footage is not used in any way other than that which the participants have consented. Secure digital storage of (often very large) files is, of course, paramount, particularly where there is sensitive information involved. You need to start out with the possible end in mind, with a clear view on how you will remove digital products if a participant withdraws their consent (this might mean editing post-release!).
I found the workshop very practical – it made me much more aware of the planning and ethics work required for high quality evaluation process and outputs and increased my awareness of the tools and applications available to facilitate digital evaluation work. I’ll certainly look out for any future workshops from the Visual Insights team on this topic!
