The Download: Trust digital maturity revealed at last

More than a year after they were collated, HSJ can reveal the second round of digital maturity assessments for 233 trusts across the country.

In theory, the NHS England-run survey provides the first opportunity to track, trust by trust, how digital maturity in the NHS has improved over time. The first assessments, conducted in late 2015 and published in April 2016, revealed wide variation in digital maturity across NHS providers. This second set should show how far each provider has come, whether the hundreds of millions of pounds invested in NHS IT since has been well spent, and where future investment is needed.

Unfortunately, it is not that straightforward.

First off, the assessments are now about 18 months old. Trusts completed the second round of survey in Autumn 2017 and the result has been available in a piecemeal form within the NHS since early 2018. NHS England only released the assessments at all after HSJ appealed to the Information Commissioner’s Office. A third round is meant to start this summer but it’s anyone’s guess when it will be published.

Secondly, they are self-assessments, with trusts essentially marking their own homework. This creates both a benchmarking problem (not every trust answers the same questions the same way) and the risk that trusts will inflate/deflate their score in the hope of gaining access to central IT funding.

The Nuffield Trust examined the second-round digital maturity assessments when compiling its report on NHS IT, published last month. Researchers concluded structured interviews with senior trust IT managers were a more reliable gauge of progress on digital maturity. Other researchers have expressed similar reservations about its reliability.

In short, the assessment should be taken with more than a pinch of salt.

Nevertheless, the assessments are still the clearest and most up-to-date measure of trust digital maturity the NHS has. They have also played a major role in deciding where hundreds of millions of pounds in central IT funding have been spent and so, even though deeply flawed, remain important.

This is what they tell us.

A rising tide 

Like the 2015-16 assessment, trusts were asked to rate their digital maturity out of 100 against three measures: readiness, existing capabilities, and enabling infrastructure. HSJ has taken an average on these three measures to create an overall digital maturity score out of 100 for each trust (see the full lists of all trusts here).

The figures show that between autumn 2015 when the first survey was run and autumn 2017 when the second survey was run, the average digital maturity score for trusts rose from 60 to 70. This direct comparison should be treated with caution, especially at the trust level. Some of the wilder swings in digital maturity scores at a trust level could be down to a change in the person filling in the survey as much as a change in circumstances.

Overall then, trusts rated themselves relatively well for “readiness” to go digital (81.6), that is the ability to plan and strategise for new technology, and enabling infrastructure (76.4). Trusts rated themselves less well on existing tech capability (53.3), although this was a marked improvement from 2015 (40.4).

There were a few exceptions. Twenty-four trusts self-reported lower digital maturity in 2017 than two years prior. Among them were two global digital exemplar trusts (Royal Free London FT, and Newcastle Upon Tyne FT), trusts that, at the time and since, received millions in additional central funding for digital technology to become “exemplars” for the rest of the NHS to follow. Northern Lincolnshire and Goole FT reported the biggest slide, judging itself to be 30 per cent less mature. The trust is in financial and quality special measures and in 2016 was infected with ransomware, leading to cancellations of thousands of appointments.

The head and tail

Top 10 digital trusts

Cambridge University Hospitals NHS Foundation Trust	99	92	97	288	96.00
Salford Royal NHS Foundation Trust	99	88	98	285	95
Wirral University Teaching Hospital NHS Foundation Trust	98	88	98	284	94.7
North Tees and Hartlepool NHS Foundation Trust	96	89	98	283	94.3
University Hospitals Birmingham NHS Foundation Trust	98	81	97	276	92
Royal Cornwall Hospitals NHS Trust	91	81	98	270	90
Bradford Teaching Hospitals NHS Foundation Trust	99	77	93	269	89.7
Royal Liverpool and Broadgreen University Hospitals NHS Trust	95	82	90	267	89
City Hospitals Sunderland NHS Foundation Trust	97	79	88	264	88
Liverpool Women’s NHS Foundation Trust	97	69	97	263	87.7

Cambridge University Hospitals FT, with its hundreds of millions invested in digital technology, has pipped Salford Royal FT as the most digitally mature trust in the country (at least in its own estimation).

Five of the top ten trusts in 2017 were there in 2015, and many of those that dropped out are lurking not far below on the list. Six of the trusts are global digital exemplars, and two are fast follower trusts (Liverpool Women’s and North Tees and Hartlepool). Only Bradford Teaching and Royal Cornwall are not part of the GDE programme. Coincidentally both have reported a major increase in digital maturity of 30 and 41 points respectfully taking them from middling trusts to among the top of the pack.

Bottom 10 digital trusts

Staffordshire and Stoke on Trent Partnership NHS Trust	51	35	73	159	53.00
Norfolk and Norwich University Hospitals NHS Foundation Trust	64	34	57	155	51.7
North Cumbria University Hospitals NHS Trust	66	29	59	154	51.3
Avon and Wiltshire Mental Health Partnership NHS Trust	65	31	54	150	50
West Hertfordshire Hospitals NHS Trust	64	31	54	149	49.7
Barts Health NHS Trust	58	44	42	144	48
North Middlesex University Hospital NHS Trust	58	31	48	137	45.7
Queen Elizabeth Hospital Kings Lynn NHS Foundation Trust	50	25	60	135	45
Buckinghamshire Healthcare NHS Trust	50	42	42	134	44.7
London Ambulance Service NHS Trust	40	30	58	128	42.7

At the lower end, there has been a bigger shift, with only three trusts (London Ambulance Service, Queen Elizabeth Hospital King’s Lynn FT and Barts Health Trust) appearing in 2015 and 2017. Both Barts Health and London Ambulance have reported severe problems with basic IT in recent years, with the former the worst affected by WannaCry cyber attack and the latter struggling with its ageing dispatch system at the time of assessment.

However there also some signs of progress at the bottom end of digital maturity, with all but two of these trusts (North Middlesex and Buckinghamshire Healthcare) reporting improvements in digital maturity since 2015.

The have-nots get some

One of the major criticisms of NHS England’s global digital exemplar programme, launched in 2016 to kick-start the latest attempt at trust digitisation, was that it financially rewarded the digitally advanced and did nothing for half of trusts that remained heavily dependent of ageing IT systems and warehouses full of paper records.

However, if the latest self-assessments are to be taken on face value the opposite has occurred. An HSJ analysis shows that overall trusts with relatively poor digital maturity in 2015 reported improving significantly more, on average than trusts with relatively highly digital maturity in 2015. For instance, the bottom 50 rated trusts in 2017 had, on average, a digital maturity score of 55, 12 points higher than in 2015. In contrast, the top 50 trusts increased their average digital maturity score to 85.2 over the same period and improve 7.9 points. Even the chosen 23 global digital exemplars trusts reported less improvement over the period (8.4 points) than the bottom 50 trusts. All this suggests that, rather than a handful of “exemplar” trusts streaking ahead of their digitally impoverished peers, the gap has narrowed.

 

Why might this be?

It could partly be down to gaming. The way the exemplar funding model favours more digitally advanced trusts first itself could have skewed scores. It’s impossible to prove one way or another but some less advanced trusts may have boosted their scores in the hope of becoming an exemplar or fast follower.

Another possibility is the timing. The less digitally advanced trusts received extra cyber security funding in the wake of the 2016 WannaCry attack (Barts Health, for instance, received £3.5m in 2016-17, more than any other trust), boosting their digital maturity score from a low base. While the majority of digital exemplar trusts had received some central funding by autumn 2017, for most the programme was still at an early stage. Another argument: it is simply easier for less digitally advanced trusts to make substantial improvement by introducing basic IT systems. Further up the digital maturity ladder improvements may become more incremental, difficult and risky.