Published: 15/01/2004, Volume II4, No. 5888 Page 4 5

E-mail correspondence seen by HSJ has revealed that Department of Health officials urged former health secretary Alan Milburn not to publish mental health trust performance ratings in 2002 because the data was of poor quality.

Despite pressure from officials and trust chief executives to defer publication, Mr Milburn insisted that community and acute mental health trusts should be rated on an indicative basis in the performance tables, published in July 2002.

On 12 July, two weeks before the performance ratings were published, Department of Health performance development unit head Giles Wilmore warned Mr Milburn that publishing full, unvalidated ratings for community trusts was 'technically possible but extremely difficult and high risk'.

He said none of the community trust data had been seen or validated by the service and that the DoH had never before published unvalidated data in performance indicators. 'There is therefore a serious risk of publishing data errors, either because the material is unchecked by the service or through human error in rushing the publication. This could lead to community trusts getting the wrong rating.'

Mr Wilmore told Mr Milburn that 'mental health colleagues' recommended publication of mental health trust ratings in 2003, when more specific national service framework indicators would be available. 'For now, publish a set of indicators in the same way as we are doing for primary care organisations, for mental health and community trusts.'

Primary care trusts were not given an overall star-rating, but their performance was assessed against a range of indicators.

Chief executives had also been urging officials to defer publication in 2002. One chief executive who, wanted to remain anonymous, said. 'The feedback we got was basically 'forget it, there is a row between Milburn and his officials and he was going to win'. On the Friday morning before the Monday the data was due, we did not know whether we were going to be involved. When we were told, we said the data was not very good - so we had about three hours to improve it before the deadline.'

On 17 July, Mr Milburn's aide e-mailed Mr Wilmore: 'The SofS [secretary of state] definitely wants to rate both specialist mental health health trusts and the 55 community trusts that were providing mental services in 2001-02.

It was agreed that we would validate the specialist mental health trust's data at a minimum and the mental health team do the best we can to clean the rest of the data... SofS is very clear that the department, especially the mental health team and Louis Appleby as mental health czar, must give full backing to the mental health ratings.'

But Professor Appleby insisted he had always supported publication of ratings for 2002, and was pleased ministers shared his view, despite being given 'more cautious advice'.

Acknowledging concerns about the relevance of the data and its validity, he told HSJ: 'I was always very keen that we shouldn't be left out, as too often mental health is the Cinderella service.'

He said that despite plans to improve the quality and relevance of indicators for mental health, 'at that stage the indicators available - for partly historical reasons - were not as close to the NSF or issues on patient care as we would have liked'.

'So there were doubts about whether the indicators were the right indicators and some concerns about the validity of the data.'

Professor Appleby said the decision to publish ratings on an 'indicative' basis 'seemed to us better than withdrawing from the system'. Asked whether he was aware of a last-minute 'scrabble' among trusts to improve the quality of their data, he said: 'I am sure there probably was, but with a laudable aim. The process did go on till the eleventh hour, so the information could be got in as good a shape as possible.'

Crisp says mental health ratings went to deadline The issue of the mental health indicators was raised by NHS chief executive Sir Nigel Crisp last week in a letter responding to questions from Conservative Party co-chair Dr Liam Fox over the star-ratings process.

Sir Nigel brought up the mental health ratings as an example of an area in which ministers and officials continued to work until close to publication because of concerns around data.But he did not mention that the decision to publish data was counter to officials' recommendations.

Sir Nigel said the validation of star-ratings indicators highlighted a problem: many of the services provided by community trusts in 2001-02 had transferred to other NHS bodies.He said only a fairly limited number of specific mental health indicators were available.

'It was therefore decided, again late in the process, to publish the results for mental health trusts (and community trusts which provide mental health services) on an indicative basis only. I stress this to emphasise the importance we attached to assuring the information we were going to publish and to illustrate that a range of questions - going well beyond the impact of the new system on individual acute trusts - was under consideration by ministers and officials until close to the publication date.'