Last year, the quality network for inpatient child and adolescent mental health services launched a routine outcome measurement pilot study with six of its members.
Over four months, staff from the six units were interviewed about their thoughts and opinions on measuring outcomes within their service. All the staff were keen to measure outcomes and saw the evaluation as a necessary and important process.
Five of the six units already had a data collection system in place. However, collecting information was irregular and often data was collected on paper, filed away and never analysed.
For the study, data was collected at admission and discharge from multiple perspectives using three separate measures:
- the Health of the Nation Outcome Scales for Children and Adolescents - clinician, self and parent rated;
- the Children's Global Assessment Scale - clinician rated only;
- the Eating Disorders Examination - clinician and self rated.
The units then collated and submitted all data to the network.
The units that described the process as "challenging" were those that were facing most resistance from team members or that did not employ an assistant psychologist to do the work. Two of the units met with resistance from frontline staff, such as key workers, who felt they were already overworked. They highlighted a need for further funding for more computers and assistant psychologists.
The units that found the process simple employed assistant psychologists to conduct the work. They made sure measures were completed by working closely with staff members as well as young people and parents or carers who completed the forms.
Despite some challenges, the process of obtaining multiple perspectives from clinicians, young people and parents/carers was valued by all staff. The majority said they wanted to collect data "just in case" they were suddenly called on to provide evidence to commissioners. All units believed the evidence would be a good way to "prove" what a good service they provided.
Many units believed it would also be beneficial to feedback findings to parents/carers and young people, not only to further integrate them with the outcome measurement service but also to provide evidence on how their service was performing in meeting the needs of the young people.
The staff were keen on the notion of national benchmarking alongside other network members and on having the opportunity to share ideas among members. They also valued receiving regular feedback in the form of structured reports.
A key lesson from the study is that it is important to have one staff member who will take responsibility for the work. Administrative staff or assistant psychologists seem ideal to take ownership of this role.
There is also a need to engage key workers. One way this can be done is by feeding back findings to the team that illustrate the positive changes outcome measurement can bring about. There must also be an awareness of the need to answer commissioners' questions about outcomes.
The clinical team as a whole must be ready to take on outcome measurement, with managers and commissioners recognising the commitment to the work involved and providing the necessary support. A combination of motivation, resources, teamwork and realistic goals are crucial to ensure outcome measurement begins and continues successfully.
By starting small with a pilot project or by working with organisations such as the quality network, it can be easier to make outcome measurement a routine part of a service.