World Health Organisation figures support critics who said the Health System 2000 report was flawed: the relative responsiveness of healthcare systems around the world has now been updated, says John Appleby

Published: 26/09/2002, Volume II2, No. 5824 Page 31

Since the interesting - but much criticised - publication of the Health System 2000 report by the World Health Organisation, more work has been carried out to refine various measures the report used to rank the performance of healthcare systems around the world.

In the 2000 report, the UK came 24th - a relatively poor position in the eyes of many, and largely the result of poor performance on one element of the overall measure: patient responsiveness. For critics of the NHS, the UK’s ranking - particularly on patient responsiveness - reinforced their view of a monolithic, bureaucratic, indeed, Stalinist, system run by and for the benefit of pen pushers, politicians and providers.

For others who looked more closely at the way WHO had measured and quantified responsiveness, and then drawn up the rankings, it was clear there were significant problems with the methodology.

As such, drawing any conclusions as to why, for example, the UK performed poorly on responsiveness was pointless. In fact, as one of the figures shows, perhaps the best explanation for the UK’s ranking (on what was admittedly a poor measure in the first place) is per capita healthcare spending: more money tends to equal better responsiveness.

A key criticism of the WHO work was that data used in the original responsiveness index was derived from surveys of around 1,800 ‘public health experts’ in 35 countries and then generalised to all other countries around the world (including the UK). The relative importance of different aspects of responsiveness (respect for dignity, confidentiality, prompt attention and so on) was ascertained from a further survey of around 1,000 public health experts (half of whom were WHO employees). No patients or members of the public contributed their opinions on responsiveness.

Since the 2000 report, WHO has carried out further survey work to address such criticisms and also to address a fundamental problem in this area: how to compare responses in different countries.

This latter problem is particularly acute when trying to measure responsiveness, as many of the domains that one would wish to measure are subjective and will vary from country to country.

So, for example, in rating the promptness of attention using a scale such as ‘very prompt’, ‘prompt’, ‘slow’, ‘very slow’, some adjustment needs to be made to deal with systematic variations in the way these elements of the scale are interpreted in different countries.

Adjustments for these ‘cut point’ differences can produce important changes in the ranking of the ‘raw’ subjective assessments. Recent work from WHO suggests that one element of their responsiveness index - the domain of prompt attention - shows that for men, the UK (in fact, Great Britain) did rather well in terms of inpatients, while countries such as France and Germany did rather badly.

REFERENCE

1Evans DB. ‘The evolution of WHO’s approach to health system performance assessment’. Smith P (ed). Measuring Up: Improving Health System Performance in OECD Countries. Organisation for Economic Cooperation and Development, Paris, 2002.

John Appleby is director of the health systems programme at the King’s Fund.