Performance data questioned for 51 trusts

Trusts are required to get an independent external opinion on their annual quality accounts. Auditors carry out a limited assurance exercise, looking at information available on a small sample of cases, typically patients referred for a procedure and those waiting in accident and emergency.

HSJ examined the 2017-18 quality accounts for 205 trusts. In 51 cases, the auditors found problems sufficient for the accounts to be qualified, given an adverse report or include a disclaimer.

Elective referral to treatment activity accounted for 37 cases, including four where accident and emergency data was also suspect. Two contained problems just with A&E data and 12 were questioned for a variety of other reasons involving different performance measures which not all trusts were assessed on.

The most common concern, accounting for issues in as much as two-thirds of RTT activity at some trusts, was inconsistent data for clock starts and stops. In some instances, the inconsistent data may have concealed breaches of the 18 week standard.

Auditors warn the results of the small sample should not be extrapolated to cover all of the trust’s data; however, if the discrepancies were replicated more widely, it could mean some trusts’ performance data was significantly inaccurate. There is no suggestion that trusts intended the reported data to be deliberately misleading.

Sarah Scobie, deputy director of research at the Nuffield Trust, said: “A lot of providers don’t have a streamlined system where all the data they need is managed. It could be that in different clinical departments they have different systems for managing data and there could be variations in how systematically data is validated.

“It can reflect how high priority this is for the clinical teams which are involved in recording some of this.”

Rob Findlay, waiting times expert at Gooroo Limited, said the problems highlighted in quality accounts may just represent a wider issue with trusts’ data. “Data quality is always problematic,” he said. “This has given rise to a whole waiting list validation industry.”

He added the way RTT waiting lists were measured did not align with how trusts organised themselves, where the patient journey was broken down into several stages. Many hospitals’ patient administration systems were based on designs which predate the current elective activity system, making it harder to collect the data.

Medway Foundation Trust had discrepancies in two thirds of the A&E cases examined, which could have affected the breach status of 17 per cent of the sample.

Lesley Dwyer, chief executive at Medway FT, said: “The samples used by our auditors were very small, and utilised a specific, limited methodology in their analysis of our data. Although this was very useful in helping us to identify areas where we could strengthen the trust’s overall data quality, the methodology did not take into account all of the aspects of how the data is generated and recorded, and so we do not believe that the findings reflect the wider quality of the data for these indicators.”

North West Anglia had adverse opinions for both its four hour and elective activity figures because the start or stop time recorded was not adequately backed up. In nearly half the sample looked at for elective activity performance, the data and what the trust recorded elsewhere did not tally.

The trust said differences between the recorded information was caused by a mix of paper and IT systems. The trust added its IT replacement programme was due to finish in July and would help remove the variation caused by paper data capture.

HSJ looked at the quality accounts of 205 trusts that were publicly available. The remaining providers had not published their quality accounts.