Collaborating to build evidence of program effectiveness
Helen Keller is quoted as saying that “Alone we can do so little; together we can do so much.” When it comes to evaluation consulting, never a truer word was spoken. Getting collaboration right is something we pride ourselves on, to the extent that it’s one of our organisational values.
How we collaborate looks different from client to client, and project to project. Sometimes, we deliver the entire project at arm’s length from the client to ensure complete independence. Other times, we work closely with the internal evaluation units of agencies and organisations to support their activities while providing an external viewpoint. In late 2018 we did this, working with the Australian Research Council’s evaluation unit to support the evaluation of the Industrial Transformation Research Program.
The Industrial Transformation Research Program (ITRP) supports collaboration between university researchers and industry to find solutions to industry problems and develop new products, processes and services to transform Australian industries. In 2018, The Australian Research Council evaluated the ITRP process and priorities. The evaluation assessed the ITRP’s alignment with the current government priority of supporting innovation and research collaboration, translation and commercialisation, particularly in the context of addressing industry issues.
We worked with the Australian Research Council to support this evaluation. We collaborated to develop and deliver surveys and interviews, collect and analyse quantitative and qualitative response data, and report to the ARC. Our ARC peers used their expert knowledge to help us tailor the surveys we delivered to research offices at universities, Research Hub and Training Centre directors and managers, Chief and Partner Investigators, and students and postdoctoral researchers who had done research training within a Research Hub or Training Centre. We also worked together to survey Industry Growth Centre representatives for their views on the program.
To complement the survey, we interviewed 33 key stakeholders from university research offices, Research Hub and Training Centres, the Australian Research Council and the Department of Industry, Innovation and Science (DIIS).
Our report analysed the results of the surveys and interviews and explored key themes which emerged from investigations. We then engaged with the evaluation team at the Australian Research Council to discuss our findings and to place them in the context of the broader evaluation.
Our collaborative involvement in designing and delivering the interviews and surveys enabled a degree of independence in the evaluation process, while still leveraging the internal expertise of the ARC’s evaluation team. We are glad to have been a part of the evaluation process and to support the Australian Research Council in their work to grow knowledge and innovation for the benefit of the Australian community. We are also supportive of the Australian Research Council’s decision to publicly release the findings of the evaluation, including our report, as this demonstrates a commitment to excellence and to supporting other organisations to conduct evaluations.
