The need for clarity in evidence based commissioning

The need for unprecedented improvements in quality and productivity in the NHS means that using evidence for commissioning has become more important than ever. While some believe that better use of evidence is a matter of producing rigorous evidence and of creating search portals, others point out that the health service does not have but needs urgently a methodology for using evidence effectively.

A multidisciplinary team from Warwick University believes that to develop such a methodology we first need to understand how evidence is used on the ground. Over the past 18 months the team has been conducting a large study funded by the National Institute for Health Research Service Delivery and Organisation programme.

This research focuses on how senior NHS staff, mainly from finance, commissioning, information, clinical governance and public health departments, use evidence in practice for commissioning and, more specifically, for improving health and healthcare services, mainly through service redesign.

Two key themes for commissioning emerged: the need to use different forms of evidence; and bringing everything together – timing and expertise.

Preliminary findings from a survey of more than 300 NHS commissioning staff suggest that multiple kinds of evidence are useful in commissioning decisions. Local public health intelligence, expert advice and examples of best practice appear the most sought after types of evidence (see image 1, above).

However, established hierarchies of evidence (see image 2), although valuable, appear to be inadequate for commissioning services and whole pathways if used alone.

Although it may be tempting to explain preferences for using one kind of evidence over another on the basis of different staff background (such as clinical or managerial), more than 130 real time observations of commissioning meetings and in-depth interviews lead the Warwick research team to resist this.

The case for

“We’ve observed that redesigning services in the NHS can be an incredibly complex task, with differential demands for information and evidence originating from the nature of the task itself. Regardless of background, many types of evidence are required due to the different task elements,” says Professor Jacky Swan, chief investigator of the project.

Emerging research suggests that, for redesigning and commissioning or recommissioning services, equal attention may need to be paid to both clinical and non-clinical evidence.

Using different forms of evidence depends on the task. For most redesign projects that were observed by the team, the task of justifying investment in a large redesign initiative, for example improving long term conditions or diabetes management, was crucial. This involved the use of authoritative evidence such as quality and outcomes framework figures or benchmarking data and/or locally produced evidence such as a strategic plan.

For example, in one project, scrutiny of the better care, better value indicators had suggested there was room for reducing ambulatory care emergency admissions. For another project it was realised that, through reference to the diabetes national service framework, there was an opportunity for improving the service quality.

For most redesign projects which aim at improving management of long term conditions or developing a new community service, developing a strategy or delivery model required:

• the review and understanding of existing models of care (for example from Department of Health strategies or international healthcare organisations);
• important clinical evidence produced from the National Institute for Health and Clinical Excellence and other sources;
• commissioning toolkits.

Such a task involves appreciating the requirements of a good service locally.

Read a case study on reducing length of stay through pathway redesign

The transformation of these abstract models into concrete commissioning implications was another crucial task. In terms of evidence, this required the creation of new intelligence and costing information with the intensive use of:

• local demographic and public health data;
• activity data, contract performance data;
• providers’ feedback and views;
• activity and financial modelling.

Common questions for engaging with this task included:

• How much does this new pathway cost?
• Will the new service actually reduce emergency admissions significantly?
• Does the type of contract or costing method provide the levers for making a cost saving?
• How can providers pragmatically deliver what we aspire to?

Answering these questions required leveraging and creating local knowledge in order to give abstract models or NICE guidelines local meaning and to create realisable plans.

Most importantly, asking other people with similar experiences can yield evidence of high value for the task of fitting everything together.

Bringing everything together

The implications seem to be that commissioning may benefit greatly from the careful consideration of a plurality, rather than hierarchy, of evidence. While the availability and use of robust clinical evidence is absolutely necessary, it may need to be complemented with the creation and use of multiple kinds of non-clinical evidence (see image 3). This can vary from rich narratives of implementation, local intelligence and demographic analyses to information about contracting options, costing, and the deliverability of specifications.

So what does it take to bring different important forms of evidence together?

Professor Davide Nicolini at Warwick Business School says: “Some important challenges for using evidence effectively seem to be related to the way different commissioning tasks are perceived and organised. It was not an uncommon observation that tasks were seen as distinct, linear, separated, if not isolated, from each other.”

In some cases of service redesign this resulted in serious implementation problems.

For example, on one project that was observed, there was great emphasis on using cutting edge clinical evidence and on developing a “gold standard” service specification without considering procurement options and contracting models well in advance.

As a result, when there was a rushed handover from service designers to commissioners and the procurement team, roadblocks emerged: the specification lacked robust performance measures, contractual monitoring requirements were fuzzy, and the proposed pathway, costed at such a late stage, proved to be very expensive. A detailed breakdown of redesigned pathways was elusive.

With hindsight, it was realised that evidence regarding contractual issues, as well as different procurement routes, had to be considered at the very beginning. The different groups – finance, information, commissioners, public health and providers – needed to work more collaboratively throughout the pathway development and commissioning process.

“At the root of all these challenges,” Professor Nicolini adds, “appear to be two very basic and problematic ideas: the belief that commissioning practicalities should be considered and dealt with at the stage of implementation, and the conviction that service redesign is the job of ‘designers’ only, while procurement and commissioning is the job of ‘implementers’.”

Using evidence for redesigning services effectively requires looking at commissioning tasks in a holistic and coordinated fashion.

For example, developing a local model of care or delivery requires a unique form of evidence (NICE guidelines, for example). Yet service redesign and commissioning as a whole might benefit from considering at the very beginning the interface of that task with other tasks, and especially procurement, commissioning and contract monitoring – commonly seen as “implementation tasks”.

Find out more

The emerging results of this research, which is ongoing, are due to be published in the following months. To find out more, visit www.warwick.ac.uk/go/emd or www.sdo.nihr.ac.uk

The views and opinions expressed in this article are those of the authors and do not necessarily reflect those of the NIHR SDO programme or the Department of Health.