In September 2008, NHS Westminster commissioned Shared Intelligence and Gillian Granville Associates to develop an evaluation framework. The purpose was to shape the development and delivery of the health inequalities and health promotion programmes, and build the local evidence base of effectiveness.
The work developed in two parts:
- A programme evaluation framework for the health inequalities programme (incorporating the health promotion strategy) was developed which consisted of a programme logic model, an indicator set and accompanying guidance notes.
- A handbook and resource pack for use by local projects that are commissioned by the health inequalities unit was developed and tested with two local projects.
NHS Westminster commissions a wide range of projects and services aiming at reducing inequalities and promoting better health in its population. These are delivered through NHS Westminster’s provider arm – Central West London Community Services – and by a diverse range of community and voluntary organisations.
The challenge for NHS Westminster was to measure the effectiveness of these interventions in reducing health inequalities in order to inform world class commissioning.
The aim of the evaluation framework is to show what could be measured in the shorter term in order to track the progress towards the long-term goal of reducing health inequalities. It enables NHS Westminster to demonstrate across the primary care trust and to external stakeholders the effectiveness and achievements of their programme. Also, most importantly, it forms a basis for identifying areas for decommissioning and informs further commissioning by showing gaps in provision.
The programme framework has two parts: a logic model (figure 1) and an outcome indicator table (figure 2).
Developing the logic model:
The logic model were drawn up from the draft Health Inequalities Strategy and shows diagrammatically how the Health Inequalities Programme is intended to work – or the ‘logic’ behind it. It has six sections:
- Context – what are the key issues or problems the programme is trying to address?
- Rationale - why is a particular approach being developed? What are the underlying assumptions about how and why it will work?
- Activities and Inputs – what are the activities being developed and for whom? What approaches are being used?
- Outputs – What are the immediate results of the work? For example, number of people reached by the activities, numbers registering with a GP. Outputs are the first step in creating the longer-term change, although they are not enough by themselves to create change.
- Outcomes -What change will result from the activities of this intervention? These form measurable steps towards impact
- Impact - What are the ultimate aims and objectives the programme is hoping to achieve
Outcome Indicator Table:
This important part of the framework takes the outcomes from the logic model and defines what will be measured to know if the outcomes have been reached. It also asks how can those indicators be measured, when and by whom.
A forth column links the strategy to other NHS and partnership commitments.
Using the framework in practice
One of the values of the framework that emerged from this work was its use as a basis for discussion with colleagues and partners. The process of populating the framework seeks to clarify how activities may address the problem and lead to the desired outcomes. It allows constructive ‘challenge’ in a safe environment and enables shared learning across teams, and ensures the focus on outcomes.
We suggest that using the framework in this way would also enable it to become a live document that is continually refined, updated and debated and in doing so, create an opportunity for ownership of the strategy across the team in a very practical way.
To retain momentum, occasional focused sessions with external facilitators would be helpful in order to create legitimate ‘space’ to work with the framework in more depth, and allow time out from busy work schedules.
NHS Westminster has a well developed programme to tackle health inequalities and promote health and wellbeing but faces challenges, along with other PCTs, in evaluating the impact of preventative services. We recognised the need for a structured framework within which to do this.
The evaluation framework that has been developed has provided a tool that we can apply in the delivery of strategy, commissioning of programmes and building our local evidence base, and supports our work with provider agencies to capture the impact and effectiveness of specific programmes.
Have another idea or suggestion to share with HSJ readers? Send it to firstname.lastname@example.org
Dr Gillian Granville, Gillian Granville Associates
Madeleine Gabriel and Laura Jenkins, Shared Intelligence