Performance watch: NHS has 'addressed' its A&E data problems, concludes watchdog

The statistics watchdog which raised high profile concerns last winter about the integrity of the NHS’ accident and emergency data this week told HSJ that they “have been addressed”.

In an exclusive interview with HSJ, the UK Statistics Authority director general for regulation Ed Humpherson said changes to guidelines and how different A&E attendance categories were published by NHS England had “met expectations”.

Mr Humpherson also said he was satisfied with the robustness of NHS England’s investigation in April, which concluded while there were inaccuracies in some trusts’ data returns, it only shifted the overall national figure by under 0.2 per cent.

The positive verdict is a major boost for NHS system leaders who have faced serious allegations they were deliberately trying to “fiddle the figures” after some NHS Improvement guidance sent to trusts in October 2017.

The accusation was that the guidance in a letter sent by then NHSI chief Jim Mackey had told hospitals to include statistics from local walk-in centres – even if they were not run by the hospital or on their site – to improve the overall performance.

Urgent cases seen in urgent care centres, so-called type three attendances, are far more easily dealt with inside the four-hour window than the life-threatening or critical cases, type one cases, dealt with in major A&E departments.

Mr Humpherson said: “We were unsure about the changes [to collection guidelines which were introduced by NHSI] during autumn 2017. But the problems that we were concerned about have been addressed.

“The thing that changed for us was that NHS England, under the leadership of its chief statistician, was very clear in revising the guidance to make a much clearer [separation] [the type 1 and type 3 attendance] in the report.

“We were concerned that the changes were challenging the integrity of the statistics. We expected more work to be done of a quality assurance nature and NHSE did that work.”

The UKSA’s intervention prompted NHS England to change the way it presents the monthly A&E publications. It now details what proportion of cases make up each trust’s overall attendances, including how many came from off site UCCs.

The new presentation and the UKSA’s subsequent seal of approval received the backing of both NHS Providers and the Nuffield Trust, which has often taken umbrage with the NHS for its spinning of statistics.

So, are all the A&E data problems addressed? Not quite.

A number of senior figures told HSJ that the issue of how the statistics were presented had been addressed thanks to the UKSA’s intervention… But how the data was used operationally remained opaque.

Concerns remain among senior provider sources contacted by HSJ this week that hospitals with onsite UCCs are still at an advantage in being able to make their “overall” performance data look better than those who do not run their local UCC.

How so? Despite the data now being presented in a more transparent way, sources said it was not clear which standard trusts were being held to: the trust’s overall performance figure, or the figure calculated once activity from other UCCs in the trust’s footprint had been taken into account.

Moreover, Nuffield Trust deputy director of research Sarah Scobie argued “locally, four-hour figures will never give a perfect representation of how well individual trusts are doing, because other services have such a big impact on them”.

The UKSA’s Mr Humpherson was also keen to stress the NHS’ success in addressing these issues “does not represent a clean bill of health for NHS statistics by any means”.

The expectations may have been met on this occasion, but the watchdog pledged to keep a close eye on how the NHS used and presented its performance data over the coming winter. 

And with the winter daily Sitrep reporting season beginning next month, this will be more important than ever.