Performance watch: Opaque A&E trial won’t wash

A perceived lack of transparency around NHS England’s high-speed trial of new accident and emergency targets has prompted fresh concerns and gives the look of an organisation apparently trying to avoid scrutiny.

Concerns a trial of alternative A&E metrics represented a ‘fait accompli’ designed to get rid of the NHS’s four-hour waiting time standard were first raised to HSJ (and rejected by NHS England) even before proposals were published in March.

The controversial proposals were set out in NHSE’s clinical review of access standards.

The timing of the review’s publication and contents had been so closely guarded that they took many senior NHS figures by surprise, including, HSJ was told, some who were on the clinical review group. And concerns about a lack of consultation had already been raised by the Royal College of Emergency Medicine.

The review then set out a very short implementation timetable for piloting potential new A&E standards, targeting a potential full roll-out of a new performance regime by April 2020.

This, coupled with strong and repeated criticism of the four-hour target by senior NHSE figures including chief Simon Stevens and chair Lord Prior in the run-up to the review’s publication only further raised suspicions among some senior NHS sources that “a process was following a decision”.

And when NHSE revealed it was excusing the 14 trusts piloting the new A&E targets from reporting their four-hour performance, as reported by HSJ last week, it prompted fresh concerns, This time, from the healthcare think tanks.

NHSE said taking the trusts out of the national four-hour data would ensure the study design was “not contaminated”.

Removing the target, it argued, was an attempt to change behaviours and ensure the trusts focused on the new targets which are: an average time for patients in A&E, time before a patient is clinically assessed, and how long the most critically ill patients waited.

But this didn’t wash with several senior figures and independent experts contacted by HSJ. The Nuffield Trust and the King’s Fund were both prepared to go on record to say so (see piece for their quotes) and I expect others will, in time, also go public with misgivings.

The think-tanks both said omitting the trusts from the four hour data was unnecessary. They also raised concerns from an analytical point of view and broader questions about a lack of transparency around the process.

The third of the three major health think-tanks, the Health Foundation said the move was “understandable”. But it also urged NHSE to set out more details about how the pilot sites are being evaluated.

HSJ asked for the criteria. NHSE said it had already published it and pointed me to section 6.2 of the review (see p. 37): a section around one and half pages long which sets out some broad-brush principles.

In my view this cannot be considered a full evaluation criteria by any stretch of the imagination. You can make up your own mind by reading it here – and it won’t take you long.

Publishing the full evaluation criteria would most likely draw some criticism. Developing such a criteria is a fiendishly complex task, and could no doubt be done in different ways.

But surely this kind of scrutiny and debate should be an important part of a good process to make what would be a hugely significant change.

One document which was, however, released last week was the memorandum of understanding between the 14 trusts involved in the trial (see box) and NHSE.

Lines of note included what it said about performance against the new metrics, which, it stresses, should not be publicly reported or discussed in public forums.

“If forums in which performance is discussed are subject to release of minutes under Freedom of Information legislation, then minutes are to be redacted,” it says.

A reluctance to release the results in live time of performance against the new metrics does seem valid. The misinterpretation of the experimental data, willful or otherwise, could easily spiral out of control.

But the tone of document seems surprisingly aggressive and controlling. This is after all a trial of new targets for a taxpayer funded public service, rather than Apple’s next iPhone.

The level of secrecy seems unnecessary and it further fuels concern that NHSE is rushing, with a predetermined result in mind.

There is still time for it to open up the process and rebuild trust, though on such a short timetable, that will soon run out.

An NHS England spokesman said: “Professor Powis’ interim report clearly sets out the intention to field test the proposals to understand the extent to which they address the identified issues with current measures, as well as the overarching principles against which all service area proposals would be considered.

“We are working with the field test sites and other stakeholders to finalise the approach within each service area.”