CCG managers must make sense of evidence

When we go to the doctor, we expect a straightforward approach where the treatment we get is based on the best evidence of what works. In the same way we all need our local NHS commissioning managers to make good decisions based on the best evidence when they plan and organise our healthcare. I have been involved in a study looking at commissioning managers making decisions and found an interesting picture.

Primary care trusts used to receive the bulk of the NHS budget to allow them to commission healthcare from hospitals and community services, but the NHS reforms have changed that.

Clinical commissioning groups will now be doing the commissioning against a background of a cut in the budget, so it will be imperative for the new teams to get it right and to make sure patients get care based on the best evidence.

‘Consistency and fairness were important, but the managers also wanted strong, authoritative, peer-reviewed evidence to help them and this was not always easy to get hold of’

To understand the process, we need to understand how commissioning managers use the evidence to make decisions. The two-year research project investigated 15 NHS commissioning organisations – four were looked at in depth and the remaining 11 surveyed.

We observed 75 management meetings and interviewed 52 managers. Nearly 350 commissioning managers completed the survey, explaining how they use evidence in their everyday decisions.

We looked at different levels and sizes of decisions. One sort of decision managers take regularly is whether to fund patients with exceptional cases (for example, unusual, highly expensive cancer drugs, or long-term hospitalisation for complex forms of addiction treatment).

On the evidence

These are called individual funding requests. We found that many types of “evidence” were used and that this evidence did not seem to follow the standard model that doctors use in patients’ care. Three kinds of activities took place: following bureaucratic procedures, making sense of cases, deliberating over the merits of the request.

The most crucial of these activities was making sense of patients’ cases. It allowed managers to imagine the effects of their decisions: “Is it going to work? How much will it cost? Will the patient benefit? Are there alternative treatments?”

Consistency and fairness were important, but the managers also wanted strong, authoritative, peer-reviewed evidence to help them and this was not always easy to access.

Even when managers found the evidence it did not “speak for itself” but needed translation to be useful. Translation happened by collaboration, co-production and sense making between the groups. In our survey, commissioners often felt most satisfied with decisions based on this sense making, reframing and co-producing of evidence.

For all decisions, collaboration, co-production and sense making worked best when the aim was to develop a mutually agreed solution with existing providers but during the process the involvement of experts with vested interests had to be accounted for.

Political weapon

Problems needed to be faced when financial or other (dis)incentives came into play in order to proactively manage diverse perspectives and potential conflicts of interest. And evidence was sometimes reinterpreted in the context of a specific activity.

For example, National Institute for Health and Clinical Excellence guidelines were used in one context as a “political weapon”, when commissioners used the guidelines to convince clinicians about the necessity for changing existing commissioning arrangements for transient ischemic attack services.

Conversely, in another context (in diabetes), NICE guidelines were used as a “technical instrument”, enabling project members to identify the most effective and safest drugs for the diabetic patient pathway.

Different managers (public health, finance, contracts and so on) clearly brought very different kinds of expertise to the decisions. In the survey, the commissioning managers said that local budget, national guidelines and fit with the local strategic plan were some of the most important drivers of decisions. All these considerations had to be integrated to incorporate evidence into commissioning decision making.

As a result of our research we designed a toolkit to help managers with decision making.

‘Managers will need experience, understanding and a real mix of expertise to make sense of the evidence’

Making decisions in NHS commissioning is a complex process that does not seem to follow the standard model. There is a risk that the NHS changes will result in difficulties for new NHS commissioning managers.

Working in new organisations with new colleagues these managers will need to make good decisions. To do that they will need experience, understanding and a real mix of expertise to make sense of the evidence in the local context – and these qualities cannot be built up quickly.

Aileen Clarke is professor of public health and health services research at the University of Warwick