A health information expert has claimed official consultant level quality data is flawed because of the ‘conflicts of interest’ of those who oversee national clinical audits.
Roger Taylor, co-founder of the health data company Dr Foster Intelligence, told HSJ that information published from audits of individual consultants’ outcomes did not reflect the “true variation” in surgeons’ performance.
He said the clinical societies responsible for managing the audits had a vested interest in not exposing significant performance differences.
In December 2010 the government commissioned the charity Healthcare Quality Improvement Partnership to coordinate and publish outcomes data for 10 national clinical audits at consultant level. The first figures were published in autumn last year.
Mr Taylor said only one of the 10 audits identified any individuals whose performance made them “outliers” and this suggested the others did not “accurately identify the degree of variation”.
He said: “There are real conflicts of interest… If you have a group of surgeons who are trying to agree a way of comparing themselves… the analysis that is going to most likely satisfy a large number of people and get consensus is the one that shows they’re all the same.”
Mr Taylor also said the audits covered a relatively narrow range of procedures and reported outcomes which were not the most appropriate measures of performance.
“The selection of mortality as an outcome for hip and knee operations is selecting an indicator that is very unlikely to show any difference between consultants,” he said.
“If you select an outcome such as complication rates or revision rates, you are much more likely to see real differences.”
He recommended improving the usefulness of clinical audits by opening them to independent scrutiny and having the data “externally validated” by linking and comparing it to administrative data.
HQIP director of outcomes publication Ben Bridgewater told HSJ he agreed there was a “potential” for conflicts of interest to compromise audits but said checks were in place to mitigate this.
Audits had “clinical representation from the professional societies, [but also] methodological representation from the audit provider [and] project management from the audit provider”, he said.
“Certainly [for] the ones I’m involved with, all the stuff is out in the public domain so the idea that these are purely professional society led audits that are looking after their membership just isn’t the situation at all.”
Professor Bridgewater, a consultant cardiac surgeon at the University Hospital of South Manchester, said the fact most audits were not identifying outliers could be evidence clinicians were using the data effectively to deliver good outcomes.
However he said he “absolutely” agreed with Mr Taylor that indicators should be refined further and said clinicians were working to “evolve” the audits to make them more discriminating.