at a glance

Anglicare Victoria has developed an outcomes framework specific to Out of Home Care (OOHC). The framework is aligned to the Department of Health and Human Services (DHHS) Looking After Children (LAC) developmental domains (health, emotional and behavioural development, education, social presentation, identity, self-care skills, and family and social relationships).

Results from each client survey inform practice decisions about the individual client’s placement planning and other Interventions. The survey provides high quality data about the wellbeing of children and young people and subsequent Evidence-informed practice advice which is then used for continuous service improvement at individual, cohort-specific and service-wide levels.

The Challenge

Anglicare Victoria had previously developed and trialled an outcomes framework to measure the wellbeing of children and young people in its OOHC programs.

However, the initial framework tried to measure too much, took a long time for staff to complete, and led to problems with incomplete data.

As a result, they redesigned the framework with a focus on making it easy and efficient to use and embedding it within existing practice processes. David explained that they needed to make sure that relevant, Evidence-informed practice principles and advice are automatically included within results/reports when the survey indicates that individual children and young people are experiencing particular difficulties.

“It’s important to not to let the pursuit of perfect destroy the attainment of the good”

Lessons Learned

  • Embedding the new survey into existing processes — by aligning the delivery of the client survey with the child/young person’s placement review.
  • Taking the time — to determine what information was needed and asking the right questions to get it.
  • There is no such thing as an ideal framework — there is always a trade-off between scope of assessment and practical utility.
  • Understanding the benefits of using psychometric instruments — to validly and reliably measure client psychosocial well-being, while knowing when issue-specific questions need to be developed.
  • Making research and evidence accessible and fit for purpose — practice must be evidence-based, but evidence must also be practice-based!

The Journey

Start small and keep building

“For any outcomes framework to be successfully implemented within services, it must be both useful and convenient for workers.”

Throughout the design phase, Anglicare Victoria focused on key outcome areas as opposed to trying to measure every outcome that may possibly be of interest. The research team reduced the total number of questions from several hundred to more manageable 30-75 questions (depending on the age of the child or young person).

The revised survey includes a mix of Validated psychometric measures and specific questions relating to placement quality issues. Check out the measures Anglicare Victoria used at the bottom of the page.

Most importantly, the survey now only takes between 5 – 15 minutes (depending on the age of the child) and can be completed over the course of several input sessions – providing staff with the opportunity to gather additional information if required.

Reducing the administrative burden

A major consideration for the design team was to save staff time, which would increase the likelihood of survey completion and quality data.

By structuring the survey in accordance with the LAC developmental domains, workers can simply copy and paste straight from the results-report into the required fields for DHHS reporting, specifically the qualitative-input fields of the DHHS Review of Care and Placement Plans and 15+ Care and Transition Plans.

Survey scheduling is also built into each child or young person’s individual placement review cycle, enabling survey-related work to be staggered across the year, rather than all coming at once for all children and young people within a worker’s caseload.

Embedding a commitment to practice improvement

When designing the practice framework, Anglicare Victoria identified instruments and questions that could serve as a measure of client psychosocial wellbeing and developmental progress, as well as an assessment of placement and service quality. The survey results-report enables practitioners to reflect on client strengths and challenges, including in relation to the quality of placements.

This highlights useful areas of focus for the next six months, leading up to the next review. Additionally, Evidence-informed practice principles and advice is automatically included within the results-reports. This provides useful practice guidance around how to intervene regarding areas of difficulty highlighted in the data for an individual child.

Selecting IT systems to support success and foster confidence

Anglicare Victoria recognised the need for the survey interface to be easy for workers to access and use. Their Learning System Grant enabled the agency to use the services of Orima Research, which provide the interface and secure data-capturing systems for the survey. However, organisations with a smaller budget could potentially harness existing data analytics services offered by the government.

Having the right skills and expertise for the job

Anglicare Victoria’s research team dedicated significant thought to the development of the outcomes tool. The project drew on in-house expertise of staff in key positions, including executive management and principal practitioner roles, research and analytics, and direct service staff. This kind of multi-disciplinary collaboration helps ensure the finished product meets the needs of all stakeholders – including service staff, researchers, management, funders and – most importantly – clients.

“Your assessment/review measures and outcomes measures should be the same tools in a process sense…requiring workers to complete separate assessment/review and outcomes measures is just too much work. They need to be spending as much time as possible engaging with clients, not completing paperwork.”

The Outcome

The revised OOHC survey was officially launched in November 2018 and survey data analyses are occurring progressively over the course of the next 12 months.

Anglicare Victoria now has an outcomes measurement tool that they are actively using with clients in care. As a practice tool, the framework will assist with improving the services provided to individual clients by highlighting areas of strength and challenge at a client-cohort and service level, and so usefully inform continuous improvement processes. Additionally, outcomes data will be used for a variety of advocacy-oriented research and service-evaluation initiatives, including rigorous testing of service models to determine if these can be placed on the DHHS evidence–based service menu.

The Future

Over the next twelve months, Anglicare Victoria will develop a complete set of repeated measures data for each child/young person in their care. This information will allow for improvements in practice delivery so that services continue to provide optimal care, keeping children and young people at the centre of all efforts and planning.

Tools and resources used by Anglicare Victoria:

Quality of School Life – Adventure (Motivation) subscale (QSL)

Education

The QSL measures primary-school-aged students’ perceptions towards school against three dimensions, 1) general satisfaction with school 2) commitment to school work 3) attitudes towards teachers. Learn more about the QSL

School Attitudes Assessment Survey-revised

Education

This survey is a validated instrument used to measure the attitudes of adolescents toward school and teachers as well as their goal-valuation, motivation and academic self-perceptions. It is also used to explore below average academic achievement in high school students. Learn more