Children and Families
Evaluation of evidence based models
Department of Children, Youth Justice and Multicultural Affairs, 2020
THE CHALLENGE
In 2013, the Queensland Child Protection Commission of Inquiry found the Queensland child protection system did not adequately provide for the safety, wellbeing, and best interests of children. The Queensland Government accepted all the report’s recommendations, and as part of its response, invested in a series of reforms to Child Protection and the family support system.
As part of these investments into family support and early intervention the Department of Children, Youth Justice and Multicultural Affairs trialled two evidence-based models developed in the United States of America, which have been shown to reduce the number of children requiring alternative care, and to provide more efficient and effective service delivery compared with standard models of care.
The Department funded two established non-profits to deliver these models: YFS implemented Functional Family Therapy – Child Welfare® (FFT–CW®) as a pilot in the Beaudesert and Browns Plains region and Act For Kids implemented a pilot of SafeCare® on the Gold Coast.
FFT–CW® is an intensive, in-home, strengths-based program that works with children and young people together with their family or kin to help them strengthen relationships, improve communication, and respond to challenging behaviours that could place them at risk of a statutory child safety intervention. Its overall objective is to enable children and young people to remain with their families where it is safe to do so.
SafeCare® is an evidence-based parenting program for families with children aged zero to five years, which aims to enhance early childhood development opportunities, strengthen parental capabilities, and support positive and safe parent–child/infant interactions essential to healthy early childhood development.
THE OBJECTIVE
ARTD, in partnership with Social Care Solutions, were engaged by the Department to undertake an evaluation to examine the implementation, appropriateness, effectiveness and efficiency of the first two years of these pilots.
OUR APPROACH
The evaluation used a mixed methods approach to assess implementation, appropriateness, effectiveness, as well as efficiency through a cost-effectiveness analysis. It drew on existing service provider case studies, outcomes and service data, participant and program staff interviews, and costs data. The evaluation did not seek to compare the models or pilots to each other: each was evaluated individually.
IMPACT
Details on the evaluation findings and the next stage the Department is taking to analyse these findings and application to the family support system is outlined at News articles – Department of Children, Youth Justice and Multicultural Affairs (cyjma.qld.gov.au).
You can also read more on the evaluation’s findings on the YFS website.
______
Resilient Families
NSW Office of Social Impact Investment, 2013-2019
THE CHALLENGE
In 2009 the NSW Government introduced a five-year action plan, Keep Them Safe, to reshape the way family and community services are delivered in the NSW and to improve the safety and wellbeing of children and young people. The plan has brought an expanded role for the non-government sector and stronger relationships between government and NGOs. It also reflects a commitment by the NSW Government to preventative and early intervention services to deliver better and more cost-effective social outcomes.
The Resilient Families service is an innovative and evidence-based, intensive family preservation service delivered by The Benevolent Society. The service is designed to address concerns about the safety and wellbeing of children that, if unaddressed, are likely to result in their entry into statutory out-of-home care.
The Benevolent Society has been delivering Resilient Families since October 2013, under one of the NSW Social Benefit Bond (SBB) pilots in which private investors provide up-front funding to service providers to deliver improved social outcomes. The funds provided under an SBB are intended to expand social investment into prevention and early intervention approaches that otherwise may not receive sufficient funding. If outcomes are delivered the cost-saving to the government can be used to pay back the investor’s principal and provide a return on investment, with the return on investment dependent on the degree of improvement in social outcomes and the structure of the SBB.
THE OBJECTIVE
ARTD Consultants were engaged by the NSW Office of Social Impact Investment to evaluate the implementation and outcomes of the RF service and to assess the appropriateness of the TBS SBB measures.
OUR APPROACH
The evaluation was undertaken in two stages, using a theory-based, mixed-methods design with the process, outcomes and economic components.
The process evaluation was the focus of Stage 1, examining the implementation and costs of the RF service and laying the groundwork for the outcomes evaluation; firstly, establishing the comparability of the Index and Control Cohorts, and then outlining ways in which outcomes would be measured.
In Stage 2 the focus of the evaluation shifted to the outcomes evaluation while continuing to
monitor key implementation factors including targeting, timeliness, family engagement, and service focus, duration and intensity. Stage 2 also included a case file review to understand the business-as-usual service being provided to Control families.
The evaluation delivered five reports over five years. The first four reports are now available
on the NSW Office of Social Impact Investment website: https://www.osii.nsw.gov.au/tools-
and-resources/evaluation-of-the-social-impact-investment.
IMPACT
The evaluation reports publicly available show the positive impact the program has had on children entering out-of-home-care. They also provide learnings and recommendations to refine the program delivery as well as the outcome measures used to calculate performance under the bond.
SECTOR IMPLICATIONS
The evaluation was a critical component of this social impact investment. It provided learnings about the design and delivery of Resilient Families that could be in future programs and captured important outcomes for children and families that were not measured within the bond calculation.
It also assessed the appropriateness of the bond measures themselves, providing learnings for future social impact investments in the sector. This is important giving the growing popularity of social impact investment in both in Australia and internationally.
Implications of the evaluation on practitioners and the social impact investment sector more broadly are also being shared at the Australian Evaluation Society conference in Sydney in September 2019.
