World class commissioning: quality assurance

Nobody pretends that, overall, the English NHS is starting from a point of world class commissioning.

There are good bits and pieces in progress around the country, already making a difference to frontline services. But the route to world class commissioning needs a map, signposts and milestones. The Department of Health's world class commissioning team, working closely with the NHS, has developed an assurance system that can help clinicians and managers along this journey.

Sceptics have voiced concerns about the phrase "world class", fearing that it may lead critics to discuss the gap between world class rhetoric and present reality. But what should be the NHS's goal? "mediocre commissioning"? "faking it" commissioning?

There has been little, if any, serious challenge to the various elements that underpin commissioning - improving health outcomes, maximising practical use of data, assessing variations in clinical practice, and making finite NHS resources go further and do more.

The NHS needs to know how it is progressing. Echoing many key elements of the past decade's reforms, a quality assurance system has been developed with frontline staff, to help PCTs assess and understand their progress.

Just as care quality has the Healthcare Commission and foundation trusts have Monitor, the DH team has field-tested its newly launched world class commissioning assurance system to help PCTs assess their development, and understand what further steps they can take.

World class commissioning assurance is a national system for the English NHS, managed by strategic health authorities. Its core aim is a consistent system (manageable within an annual cycle) to help PCTs design, deliver and support world class commissioning.

Support and development is an important underpinning of the new assurance system. It is not "just another test", but a process in PCTs' own developmental interests.

The system will measure performance in three areas: health outcomes, competencies, and governance. For outcome measures, each PCT will select up to eight local indicators that are consistent with their strategic objectives. Data for these indicators will be used to produce a scorecard, which displays PCT outcomes in comparison with the national average and upper quartile for the NHS.

Outcomes can be defined as long term strategic objectives, referring to changes in population health which can rarely be made quickly. The options for outcomes will be based on the vital signs indicator set, which was published on the DH website in January.

Positive response

The core components of the assurance system were designed and tested with the PCTs in the NHS North West region. Reflections on the test were generally positive, and participating PCTs reported having found the panel day constructive, positively challenging and valuable. Participants particularly welcomed the inclusion of health outcomes, which was seen to support the strategic shift in commissioning focus towards delivering long-term, improved health outcomes.

Kate Dixon, who leads on the world class commissioning assurance system for the DH world class commissioning team, emphasised that the NHS North West PCTs that trialled the assurance system "found the assurance process - particularly the external review panel - a challenging, but very positive developmental process".

"The credibility and quality of the five people who made up the NHS North West panel was essential: participants felt that the quality and balance of skills on the review panel made the assurance process work overall," Ms Dixon says.

So assurance will not be "done to" PCTs: they will collect data to support the assurance process themselves.

The assurance process at each PCT will culminate in an assurance panel review day, where the review panel (chaired by an SHA director) uses structured interviews to make its assessment of the PCT's capabilities and offer feedback on areas for future development. These will take place in November and December this year.

These elements will then be summarised in a written report for the PCT, benchmarking them against their colleagues in the SHA and nationally. Current thinking is that the overall findings will not be publicised nationally in the first year of the system's operation (2008-09), but will be made publicly available in future years.

SHAs will implement the system in all PCTs by March 2009. A contract for consultancy to support this work has been awarded to a consortium of Ernst and Young, Kaiser Permanente, Humana, Dr Foster and Mental Health Strategies, led by McKinsey.

Measuring commissioning competencies

The world class commissioning assurance system has developed a range of measures for each of the competencies. World class commissioners will:

• locally lead the NHS;

• work with community partners;

• engage with public and patients;

• collaborate with clinicians;

• manage knowledge and assess needs;

• prioritise investment;

• stimulate the market;

• promote improvement and innovation;

• secure procurement skills;

• manage the local health system;

• make sound financial investments.

Each competency will be assessed on a combination of self-assessment, evidence and 360-degree feedback. Based on these assessments, the PCTs will be rated on levels 1-4 (where level 4 is world class). It is expected that most PCTs will be at level 1 or 2 this year. The governance element of the system focuses on whether the PCT board has taken ownership of commissioning. It examines whether the board has a meaningful strategic plan for commissioning, supported by a robust financial plan.

Assessment of board-level governance covers all the PCT's plans: strategic, long-term financial, annual operating, and organisation development plans, and board controls and processes. Each will be rated red (least good), amber (warning) or green (best).

The governance element draws on best practice from other approaches, in particular foundation trust regulator Monitor. The PCT board controls section also reflects assurance of the PCT's organisational capability, including ensuring information management and innovation.

The assurance programme also considers the organisation's potential for improvement. This is provided by the review panel, included in the final written report for each PCT. Ms Dixon notes: "This is a descriptive commentary - a snapshot of where the organisation is at the time of assessment. It's vital for SHAs to remember that they could get a similar sort of view from two very different PCTs: one could have innate strengths, but could still improve commissioning much further, whereas the other PCT may have made huge strides to get to a similar point, and might struggle to get much further on its own.

"Each PCT's potential for improvement would be very different, as could the likelihood that they will get to world class. One size will not fit all."

Incentives and consequences

Just as the principle of "earned autonomy" led foundation trusts to the financial freedoms they now enjoy, the world class commissioning assurance system is considering appropriate incentives and consequences in three broad categories: reward, regulation and recognition.

PCTs performing at the top level of success will achieve status as a World Class PCT and a package of complementary incentives.

The 2008-09 operating framework set out the requirement for all PCTs to produce a strategic plan by the autumn. The governance element will include an assessment of whether PCTs have ownership of and developed a meaningful strategic plan for commissioning. The DH's world class commissioning team offers help in this process with guidance on strategic planning, including information on content and format.

The world class commissioning team (with Dr Foster, Mental Health Strategies and McKinsey) has prepared data packs for all SHAs and PCTs, to support their strategic planning processes. These packs include trend and comparison data for every PCT, and focus on the key areas of commissioning spend and outcome measures:

• health needs and mapping;

• outcomes and vital signs;

• programme budgeting;

• primary care;

• prescribing;

• elective care;

• non-elective care;

• mental health and learning disabilities;

• community services.

Developed with NHS South East Coast PCTs, these data packs are now available via the electronic toolkit, which SHAs, PCTs and panel members can access via the website.

Private prescription

So once a PCT's commissioning health status is diagnosed, will there be a mandate or pressure to use consultants from the private sector? Ms Dixon answers: "It will be up to PCTs, with their SHAs, to decide how to develop any capacity and capability gaps they find. We will describe development tools and resources. PCTs can address gaps through local knowledge sharing.

"PCTs will be able to club together and collaborate, like the West Midlands data warehouse. Or they can choose to learn and train individually - building internal capacity by sourcing training courses (for example, on needs analysis). Or they can buy specific support, either from suppliers on the framework for procuring external support for commissioners or from others. We're value-neutral on how they fill capacity and capability gaps."

SHAs will implement the assurance system, and Ms Dixon emphasises that the new requirements "will ask a lot" of them. For 2008-09, SHAs will be supported by teams from Ernst and Young and McKinsey.

In the next phase, the DH team will help SHAs develop their assurance and review their capacity and capability. SHAs can also take part in a simulation in July run by Humana, to support their learning and development of tailored approaches to different PCTs' circumstances.

The world class commissioning team (in partnership with the DH system management team) is developing an SHA diagnostic tool to focus on capacity and capability building. This will help SHAs identify areas to strengthen, so they can play a critical role in assurance.

World class questions

Gary Belfield, DH director of commissioning, answers HSJ readers' questions on the new world class commissioning assurance system.

Will this new process mean more paperwork? Isn't it just another target in disguise?

The assurance system won't require extensive data collection, and uses documents the PCT will already have (for example their strategic plan for commissioning) as well as underpinning plans for delivery. PCTs will spend time on self-assessment and the ratings part of the panel review day. These will focus on supporting the PCT to reflect on its commissioning strengths and areas for development.

Will league tables be published as a result of the assurance system?

In the first year (2008-09), no formal ratings will be published at a national level. The results of the assurance system will form part of a conversation between PCTs and SHAs about developing commissioning capability. From 2009-10, ratings on outcomes, competencies and governance will be published.

How will commissioners and providers be assessed on their ability to work together and with wider stakeholders to achieve world class commissioning, especially where joint commissioning has been encouraged for a long time - for example, in mental health?

Commissioners need to operate as learning organisations, seeking and sharing knowledge and skills. The first four of the 11 organisational competencies that lead towards world class commissioning are about working together, with the PCT taking a leadership and facilitation role in strategic planning.

What additional data must I collect and interpret to enable world class commissioning to be successful in the PCT?

Competency 5 stresses the importance of data management and analysis, and all PCTs and SHAs will be provided with SHA-specific benchmarking documents to provide trend and comparison data for every PCT.

The data packs use national data sets, to which PCTs already have access, and will be accessible to PCTs and SHAs as part of an electronic toolkit to support commissioners with implementing the assurance system and helping them move towards world class.

The packs will analyse data for this year and provide a foundation for commissioners to understand health needs. With this experience, by next year commissioners will know whether there are any gaps in the data and so whether any more data should be collected.

Is there any additional resource associated with the assurance system?

No, there will not be any new resources for PCTs. Commissioning is a PCT's core business. By implementing world class commissioning, PCTs will ensure that health services are commissioned in the most effective way possible.

This will deliver better value and free additional resources to drive continuous improvement and additional investment in local health and care services.

How does the assurance system link to the operating framework and the vital signs?

The assurance system is focused on commissioning, and includes an assessment of whether locally prioritised outcomes are improving over time.

In the outcomes element of the system, PCTs choose up to 10 priority outcomes that reflect their strategic plan. These will have been agreed for the next five years with the PCT's partners and population, and reflect population needs. Two of these are nationally consistent - life expectancy and health inequalities.

The remaining outcomes are locally chosen. They may be drawn from the vital signs, or they may differ, depending on the local circumstances. They must be supported by robust data to measure improvement.

The vital signs reflect a combination of outcomes, and interim milestones to deliver outcomes. They reflect the priorities set out in the annual operating framework for 2008-09. Performance against the vital signs is assessed in the assurance system under governance.

The greater emphasis in the assurance system is on the assessment and improvement towards longer term, locally determined outcomes.

Will there be an element of 360-degree appraisal, so that we can reflect on our level of support from the SHA?

The assurance system is for PCTs, and so includes feedback on how SHAs are included in the PCT's commissioning function. It does not provide assurance of SHAs, and so will not include reflection by the PCT on the level of support from the SHA. The panel review day and follow-up with the SHA will provide the force for this discussion.

Assurance in practice: the assurance trial in the North West

Manchester PCT was one of five in NHS North West where the assurance system was trialled. Chief executive Laura Roberts confirms: "We found it really worthwhile. We'd have wanted to do it even if it weren't mandatory.

"It was an opportunity to look at our vital signs indicators, and choose which we should apply.

"Because our population has poor health outcomes, we could have chosen almost anything: this helped us to concentrate on where we could make the most difference quickly."

Manchester PCT was among those reconfigured 18 months ago, and Ms Roberts adds that the pilot "was a great opportunity to work as a team. It made us question: when we get into provision mode, do we adopt providers' problems rather than our commissioning role?"

Ms Roberts underlines the importance of the external assessors. "A high-quality panel is vital. We were very fortunate in that regard. One criticism of Fitness for Purpose [the 2006 PCT assessment programme] was the consistency of the assessors. This needs high-calibre assessors who understand your world."

What has Manchester PCT taken from the process? Ms Roberts concludes: "We approached assurance with a spirit of opportunity, and it gave us a chance to re-energise, re-commit, and focus our minds on a really good direction of travel."

NHS North West director of commissioning Joe Rafferty, who chaired the review panel, notes that his SHA "did not regard the assurance trial as 'Son of Fitness for Purpose' - which we'd actually found broadly positive. Our PCTs were up for trying it, having found Fitness for Purpose innovative and refreshing. We wanted to see the assurance framework as developmental, not as burdensome and threatening."

He adds: "I wanted to see how we could practically manage running this process across five of our 24 PCTs in one week. It proved arduous, but do-able. For most SHAs, it shouldn't be massively burdensome - London and ourselves have a big logistical challenge, with over 20 PCTs."

Mr Rafferty says that the review panel tried "to make it feel unlike heavy audit. The focus was on what PCTs were trying to achieve with commissioning.

"It felt positive and encouraging, rather than inspectorial."

He also warns that one problem with assurance systems "is that people get good at jumping the hurdles, when we need to be running the race and looking to the finishing line of a world class system".

PCTs should look for the positives, to create a "dynamo for change" effect, he concludes.

www.dh.gov.uk/worldclasscommissioning