Published: 07/10/2004, Volume II4, No. 5926 Page 28
A new report allows comparison of surgeons' performance, but is cautious about how it is presented. John Appleby explains
A significant step forward in the reporting of surgical data has taken place. It is now possible to compare the performance of individual, named cardiothoracic surgeons.
The latest report from the Society of Cardiothoracic Surgeons builds on its previous four publications and contains a wealth of information on cardiac-related activity and surgical outcomes.
1But it has been a long slog to get this far, as the society acknowledges, and there is still some way to go before the data becomes really useful for patients.
The society emphasises that any commentary on the data should bear in mind that reporting is voluntary and built on a fragile professional consensus. Such feelings are understandable; public disclosure of surgeons' performance could have disastrous consequences for them, so the effort to collect the right information and present it in the right way has been considerable.
A key issue for surgeons, and one repeatedly emphasised by the society, is the need to adjust the outputs of surgery according to the quality of the inputs. Poor outcomes can in part be explained by factors beyond the surgeon's control, such as age of the patient and co-morbidities (see table 1). Of course, many other factors will affect the chances of a patient's survival, not least the quality of post-operative care in the hospital (and at home) and the competence of the whole surgical team.
Given enough data, crude mortality rates can be adjusted in order to isolate what impact the surgeons themselves have on survival.
However, the society has been unable to amass enough data to allow for the case-mix of patients, so instead presents crude mortality rates for each surgeon averaged across all operations over two or three years (see tables 2 and 3 for overall figures).
For the patients, making sense of the resultant figures is difficult. Across all 215 surgeons covered by the society's analysis, crude average mortality rates for coronary artery bypass grafting (CABG) vary from 0 to about 6 per cent (with a national average of about 2 per cent). Although volumes of operations are shown for each surgeon (which in any case are largely unrelated to mortality, the society states), no individual figures on mortality are presented and no data is presented for the variation in outcomes for individual surgeons across all their cases.
In order to make some judgement about the data, the society has chosen a statistical benchmark or limit to indicate a level of performance that may be considered poor or unacceptable. The limits chosen are confidence intervals set at 99.9 and 99.99 per cent.
On these criteria, for three years' averaged results none of the 215 surgeons fell into the 0.01 per cent level of unacceptable performance and just two were above the lower limit of 0.1 per cent. Of course, given these statistical limits, such results are perhaps not unexpected: the chances of falling above the 99.99 per cent limit are one in 10,000. l Professor John Appleby is chief economist at the King's Fund.
1 The Society of Cardiothoracic Surgeons of Great Britain and Ireland (2004). Fifth National Adult Cardiac Surgical Database Report: 2003. All graphs are taken from this report.