Performance Watch: No more sticking plasters – have a transparent debate about A&E data

This week’s Performance Watch sets out some of the main issues which need addressing to bring some short term rigour back to the collection of accident and emergency waiting time data amid justifiable concerns about its validity.

First, a quick recap of events which ultimately resulted in the UK Statistics Authority launching an investigation last week into the validity of the NHS’s A&E data.

Jim Mackey – shortly before stepping down as NHS Improvement chief executive – wrote to trusts in October encouraging them to bundle as much urgent care work as possible into their reported activity. This move to pick some low hanging fruit ahead of winter appears to have exacerbated a figure fiddling culture that stretches back as far as the targets themselves.

It would be wrong to suggest that Mr Mackey’s intervention was simply about massaging the data, however. Longstanding inconsistencies in counting of activity really do need to be addressed. There are three main problem areas:

• inconsistency about how “type three” activity (minor injuries units, walk in centres and urgent care centres) is counted;
• classification of activity coming through newer clinical pathways like ambulatory care and primary care streaming models; and
• unintended consequences, which mean trusts can be penalised when they are rightly rerouting less urgent cases away from core A&E.

NHSI’s intervention and the reaction in the NHS moved the UK Statistics Authority to raise its concerns directly with NHS bosses – in turn prompting widespread accusations that the NHS was deliberately fiddling its figures.

The resulting media coverage, including from the BBC, which first reported the authority’s intervention, has concentrated minds.

How bad is it?

A well placed senior NHS source confirmed to HSJ this week there were genuine concerns at the top echelons of NHS England and NHSI about the validity of the A&E data from October onwards in particular. They suggested NHSI and NHS England had “got off lightly” from the coverage. So, be under no illusions: this is a genuine mess.

Did Mr Mackey’s letter cause the national NHS performance figure to be wrongly inflated? We genuinely don’t know, the source said. What we do know is that several trusts were able to boost their own performance by including additional data from walk in centres that were not onsite.

Type three services and unintended consequences

Type three activity makes up a significant chunk of the overall A&E caseload – around a third of the 6.1 million attendees in the third quarter of 2017-18 (here are the full official definitions).

The rules on what constitutes type three were set out by NHS England in November 2015. They say a trust can include walk in centre figures only if it has clinical responsibility for it, or if it is co-located.

But as emergency care improvement lead at Isle of Wight Trust, Rick Strang, noted, in response to one of our stories: “It is silly that some systems are allowed to include type one [major emergency department activity] and type three as their [overall] performance, and others not.

“It is also silly that by sensibly rerouting emergent patients with a less urgent need away from [the ED], this system response… is likely to penalise the ED as the patients most likely to take less than four hours to complete their care are taken out of their denominator. You do the right things and performance worsens. It is right, therefore, that all of this is considered.”

New clinical pathways 

Mr Mackey’s October letter said trusts should include in their overall figures data from a set of “very effective new services for the elderly and/or ambulant patient”.

But HSJ understands there is a live debate about which pathways constitute genuine substitutes for A&E admissions, and which are simply being bundled into the overall data to massage the numbers. Pathways up for debate include activity from minor assessment units, ambulatory care pathways, some eye patients, some urgent treatment centre activity and some early pregnancy units.

A ‘very transparent’ debate is needed

I spoke to King’s Fund policy director Richard Murray – a former chief analyst at the Department of Health and Social Care – about the issue. He warned that changes need to be made in “a very transparent way to avoid any suspicion that this is an attempt to change the standards rather than just changing the way the data is counted. They may need to double run the systems for a period of time.”

Meanwhile, Nuffield Trust chief executive Nigel Edwards pointed out that the sector must not “lose sight” of the fact that the four hour target was always supposed to be about patient experience and quality.

Mr Edwards said too much focus on a single measure was having too many perverse, unintended consequences, and called for a wider range of metrics to be used. This idea makes sense, and is supported by many, but may be a debate for another day.

NHSI’s new chief executive, Ian Dalton, who only started in December, has the advantage of a clean slate. His swiftness to announce interim arrangements and kick off a review this month was commendable.

But he will ultimately be judged by whether the review he is overseeing can repair the badly damaged credibility of the A&E data and those in charge of it.