Fourteen trusts to be excused from reporting A&E target during controversial trial

The Nuffield Trust and the King’s Fund both questioned the analytical approach being taken by NHSE, and queried an apparent lack of transparency, after it emerged the 14 trusts piloting the new standards will not publicly report their four-hour target performance during the trial.

NHSE said in a note published on its website that the 14 pilot trusts (see full list below), which began testing new metrics last month, were being omitted from the four-hour data for May published today, “as not to contaminate the study design”.

The hugely significant trial follows a major review of NHS targets and represents the foundations on which NHSE will potentially build the case for scrapping the four-hour A&E target.

However, the Nuffield Trust told HSJ it would “be useful to continue publishing [the four-hour data of the trial trusts] for transparency, but also from an analytical point of view”.

Deputy director of research Sarah Scobie said: “There is no reason not to publish the four-hour target data from the trial sites that we can see.

“It would be useful to compare performance against the four-hour target as part of assessing and interpreting the new standards and if there is any correlation with the existing standard. It would also be interesting to see how the metrics worked alongside each other.”

King’s Fund chief analyst Siva Anandaciva also raised questions over the NHSE move. He said: “It’s not an easy one to judge but why not err on the side of openness?

“National air cover [for local organisations] is clearly important when you are piloting changes to something as high-profile as the four-hour A&E target…But national air cover is not necessarily the same thing as pausing the national reporting of A&E performance in fourteen major NHS hospital [trusts].”

He added: “It would have been helpful if the NHS England review had publicly set out from the outset what its plans were for operating the pilots, and what data will be collected and how this will be presented to the public. There is still time for them to do this.”

NHSE’s decision was, however, supported by Health Foundation senior policy fellow Tim Gardner. He told HSJ the move was “understandable”. 

He said: “What gets measured is what gets done, so requiring those trusts to continue publicly reporting against the four-hour target could have risked distracting their attention from testing the new measures and therefore potentially skewing the results.”

He added: “It would be helpful for NHSE to set out more detail about the evaluation of the pilot sites and whether data from the 14 trusts will be added retrospectively.”

NHSE said in its briefing note, when the 14 trusts were excluded from the national data, it led to a national performance that was 0.1-0.2 per cent better because the trial group’s collective performance was slightly below the average of the whole sector.

The official data release revealed the overall performance for May against the four-hour target was 86.6 per cent, compared to 90.3 per cent in May 2018.

NHS Providers, meanwhile, raised concerns the process could result in the loss of “valuable national comparable data” and urged for a “transparent debate” to ensure “credibility with the sector, clinicians and the public”.

NHSP chief executive Chris Hopson said: “One potential consequence of changing the standards could be the loss of valuable national comparable time series data. It is important that we do all we can to preserve this and can continue to compare national performance like for like.”

NHSE also published the memorandum of understanding trusts involved in the study had signed. It confirmed the urgent and emergency care review of standards “field tests” would take place in two six-week phases.

The trial sites will report performance on the new metrics to their local commissioners and other key stakeholders, but have been banned from publicly reporting, the MoU said.

It said trusts should not mention the results at public board and refuse to release the data if requested under the Freedom of Information Act.

Ms Scobie said it was understandable “why [NHSE] does not want to publish data from the pilot metrics because they may not be sure how to interpret the data. But that is all the more reason why they should continue to publish the four-hour data, which is after all still the national target”.

The 14 trusts field testing the new A&E metrics