Published: 01/07/2004, Volume II3, No. 5912 Page 16 17 18
The importance of patients'perceptions in measuring health service performance is not in dispute, but many of the key factors that impact on their thinking lie outside NHS control. How can these be taken into account in measuring how well trusts are faring?
How much weight should we give to patient perceptions when measuring the performance of the NHS? This is one of the issues facing the Healthcare Commission as it determines the exact nature of the replacement for the old starratings system.
The Frontiers of Performance in the NHS is the latest analysis from MORI and highlights the key factors that determine overall patient perceptions.
The importance of patient perceptions compared to financial or clinical measures is something we have been debating at the Quality Measures group, formed by healthcare data analysts Dr Foster, along with leading chief executives and medical directors.
The analysis shows that in terms of patient perceptions, there are some very clear key drivers of patient experience that individual managers and clinicians can affect.
There are other factors that are beyond the control of managers or, even, the Department of Health.
By being sensitive to these factors, however, we can put patient perceptions in context, and understand that high performance means different things in different places, as well as being very different from what we see reflected in simple league tables.
In my last piece ('What they really, really want', pages 16-19, 8 April) we examined what factors - both within and without the NHS's control - influenced patient perception.
Looking first at the factors within NHS control, being treated with dignity and respect was found to be key for patients.
Other vital factors were cleanliness, effective communication with doctors, successful pain control, a wellorganised accident and emergency department and privacy.
Those factors which had relatively little impact on patient perceptions included waiting time for admission, standardised mortality ratios, average lengths of stay or readmission rates.
We might also expect to see a relationship between financial and manpower resources and perception ratings recorded in the latest NHS patient surveys.
However, the amount spent per patient seems to have no impact on patient perceptions.
For example, the unit cost of providing health services, either at hospital or primary care trust level, has very little or no bearing on what patients think of the service they receive (see graphs 1 and 2). Neither does per capita income show any relationship with patient ratings, for either PCTs or acute trusts.
But what about the factors that influence perceptions lying outside NHS control? Local area characteristics matter a lot and because of this there are real dangers in comparing performance (however it is measured) without taking into account the local context in which trusts operate.
In particular, our analysis shows that trusts serving patients drawn from a relatively ethnically diverse population attract lower ratings from patients than those operating in relatively ethnically homogeneous areas like the North East or South West.
It is important not to leap to sweeping conclusions on the basis of this finding. Ethnic diversity can simply reflect other local area characteristics such as local population turnover, deprivation or inequality and will correlate with age.
However, our latest analysis shows ethnic diversity to be a key 'driver' of patient satisfaction, even after taking account of these potentially related factors.
The age profile of the local population is also linked to patient ratings of health services.
Where there is a relatively high prevalence of over-65s, patients appear easier to please than where the population is, relatively, younger.
This is a common finding in public sector research - older people are more likely to express satisfaction with a range of services for any given level of service quality.
Conversely, trusts that serve areas with a relatively large number of dependent children are more likely to have lower patient perception scores. These age-related factors have more impact than deprivation, which is a significant, but less important influence.
So what? Having observed these relationships we have now created a model which accounts for external factors to assess how well individual trusts are performing in terms of patient satisfaction. Key factors that are important and are incorporated, are all highlighted below, as either positive or negative factors, for both PCTs and acute trusts.
Alongside these we have factored in spending, which - even if not correlated with perception - it is still a fairly key factor.
We have used these models to 'predict' the patient ratings that can be expected for individual PCTs and acute trusts, given prevailing local conditions.
Adopting a technique called data envelopment analysis, we have made more meaningful comparisons of the performance of individual trusts, taking account of the relative constraints under which they are operating, including area profile, spending and so on. The results show that certain trusts that appear to be underperforming on the basis of their patient rating score are often performing at least as well as might be expected in the context of prevailing local conditions.
In contrast, others serving less 'demanding' populations, on the other hand, could and should be performing better than they are currently, even if they are achieving respectable scores - a fairer method of comparison than looking at raw scores.
Our model highlights how each trust can be expected to perform, based on satisfaction/patient ratings other similar trusts have already achieved, taking on board its available resources and population profile.We can then measure how efficient each trust is at delivering patient perception.
Results enable us to identify which trusts are performing below what would be expected, given prevailing local conditions and available resources.
Looking first at acute trust results, it is important to compare like with like. Specialist hospitals and orthopaedic trusts, for example, outperform others by some margin, but do have different contributory conditions.
Comparing multi-service, large acute, medium acute and small acute trusts separately, the results are very useful in identifying both under-achieving trusts and those that are performing very well considering the constraints placed upon them.
For example, Hinchingbrooke trust is performing at 100 per cent efficiency, despite scoring the second lowest patient ratings of all multi-service trusts.
Other trusts performing well in spite of difficult local conditions and/or resource constraints include North West London Hospitals trust (among the large acute cluster), Mayday Healthcare trust (among the medium acute cluster) and Newham Healthcare trust (among the small acute cluster).
Conversely, examples of trusts with relatively 'easier' populations to serve (ie comparatively homogeneous and older populations, or with greater resources on which to draw) that are shown to be under-performing, despite relatively high patient ratings, include: Isle of Wight Healthcare (multi-service); Royal Cornwall Hospitals (large acute); Worthing and Southlands Hospitals (medium acute); and Royal West Sussex (small acute).
Similarly for PCTs, the results highlight some interesting findings.
For example, while PCTs such as Newham and Redbridge are at the bottom of the scale in terms of patient perceptions, the analysis suggests they are performing well, given the nature of their local population (see table 1).
All are 100 per cent efficient - although a key point to note is that being rated as 100 per cent efficient does not mean there is no room for improvement in patient perceptions.
It only indicates that there are no other PCTs or similar acute trusts that are performing better at the moment. In contrast, some trusts in the South West, plus Eden Valley PCT in Cumbria, while attracting relatively high patient ratings, might be expected to perform even better, given their populations (see table 2).
Overall, the analysis throws new light on patient perceptions and the factors that are - and are not - under the NHS's control.While there is always room for improvement, it highlights the need for any new system of assessing overall performance to reflect local conditions rather than assuming trusts all face exactly the same issues. l Ben Page is the director of MORI's social research institute.To order The Frontiers of Performance in the NHS, e-mail ben. page@mori. com