The truth behind NHS Digital's take on the 'weekend effect'

NHS Digital has just published experimental statistics on three indicators to support the seven-day services experiment. Annual data is provided by trusts over three years from 2013/14 to 2015/16. The aim of this publication is to “provide a starting point for discussions on how to effectively measure improvement and variation in care provision across the week”.

The first indicator relates to mortality within 30 days of admission. This indicator compares the odds of mortality for patients admitted at weekends to that of patients admitted midweek, better known as the “weekend effect”. Figures are also provided comparing the newly conceived ‘transition period’ (Monday and Friday) to midweek (Tuesday to Thursday).

These indicators measure outcomes, not inputs. They tell us nothing directly about care provision.

We related NHS Digital’s measures of the weekend effect to trusts’ performance against the four priority clinical standards for seven-day services. There is no significant correlation between the magnitude of trusts’ weekend effects and their performance against any of the four clinical standards (see attached file at bottom).

There is also no significant correlation between the changes in trusts’ weekend effects between 2013/14 and 2015/16 and their achievement of any of the clinical standards. So what does this tell us about the seven-day services policy?

Well nothing, given numerous problems with how the indicator has been calculated. NHS Digital used the methodology published in the BMJ in September 2015. This has subsequently been shown to suffer from several flaws.

The elevation of mortality among weekend admissions has been shown to be driven by a reduction in admission volumes at the weekend, and not an increase in the number of deaths. When examining the whole population of patients attending accident and emergency (A&E), there is no difference in the mortality rates between patients attending on weekdays compared to those attending on weekends.

Although the seven-day services initiative is focused on emergency care only, NHS Digital include both elective and emergency patients in their indicator. Later research has not only shown the need to separate these distinct patient groups, but also that further significant differences exist within the emergency patient population.

Mortality rates for those admitted on Mondays and Fridays are not significantly different to Wednesday admissions

Elevated mortality rates have been shown to be concentrated in the small group of patients directly admitted to hospital in an emergency by a GP or other healthcare professional. This is in the context of a 61 per cent reduction in these direct admission volumes at weekends, suggesting a clear difference in the composition of admissions compared to midweek.

There is a costless way to reduce these weekend effect figures – better severity adjustment during their estimation. NHS Digital note that there may be differences in the case-mix of patients that they were unable to control for, and that these differences could explain the observed weekend effect.

This is indeed the case. Arrival by ambulance is a proxy for severity of presentation available in national hospital episode statistics, but not used by NHS Digital in their calculations. Patients arriving by ambulance experience a 30-day mortality rate of 5.5 per cent, compared to 0.8 per cent for those arriving by other means. The higher proportion of weekend patients arriving by ambulance explains much of the weekend effect.

The latest research also shows that there is no “transition period”. Mortality rates for those admitted on Mondays and Fridays are not significantly different to Wednesday admissions.

To make the statistics more useful, NHS Digital should provide figures on the variation across the week in:

• The death rate among the whole population of patients attending A&E
• The admission rate of patients attending A&E
• Average daily numbers of deaths, attendances and admissions

They should focus on emergency admissions only and present results for admissions via A&E and direct admissions separately.

Provision of these metrics would avoid the selection bias introduced into the current indicator by the more stringent admission threshold operated by some hospitals at weekends.

With the mortality indicator in its current form, trusts could reduce elevated weekend mortality by increasing admissions at weekends or reducing admissions during the week, without affecting the numbers of deaths. This would be a statistical fix rather than a clinically meaningful improvement.

There is a need for the provision of statistics tracking seven day services and the impact this has on patient outcomes for those admitted both at the weekend and during the week. These should link outcomes to care provision, and be calculated based on the latest research rather than a single study favoured by government.

Rachel Meacock and Matt Sutton, University of Manchester.