Consultants' traditional resistance to appraisal is being broken down by a few trusts which are taking the first steps towards assessment. Alison Moore reports

To those working in industry, higher education and certain parts of the NHS the idea of a yearly appraisal is commonplace. But for hospital consultants the idea that their work should be scrutinised in this way has historically been an anathema and their power within hospitals has meant they have been excluded from appraisal schemes.

But times have changed. The Bristol children's heart operations scandal and a government with a quality driven agenda have made consultant appraisal a priority.

National negotiations are now well under way and the British Medical Association has said it accepts the principle of a national scheme - although that won't stop it arguing over the details. BMA consultants committee chair Dr Peter Hawker says: 'If this is done properly it will get us out of the trenches. The potential is there to make a genuine contribution to the quality agenda. The anxiety is that if we get one bad scheme it will give it all a bad name.'

By this time next year consultants could see their NHS work being appraised on an annual basis, with their clinical practice put at the heart of the process, and appraisal linking into the General Medical Council's planned revalidation process. Meanwhile, a handful of trusts have introduced some form of appraisal of senior clinical staff (see boxes).

Many other trusts are interested in appraisals, although they may now be waiting for the outcome of the national negotiations before taking matters forward. Professor Jack Sanger, head of the centre for educational policy and leadership at Anglia Polytechnic University, worked with the hospital's consultants to develop the Norfolk and Norwich Health Care trust scheme and has since been approached by about 10 other trusts. A course on appraisal skills, run by the British Association of Medical Managers, was over-subscribed, despite having 85 places.

BAMM chief executive Dr Jenny Simpson believes that appraisal will come to have a central place in improving quality: 'We are talking about a process that will be at the core of clinical governance and revalidation. It has to be more than a snapshot of how people are performing on a particular day - it has to be robust and look at the effectiveness of an individual's clinical practice.' BAMM has produced a guide to appraisals.

1So how easy is it to introduce appraisal schemes? Despite initial concerns those trusts that have introduced appraisal have managed to allay many consultants' fears.

At Norfolk and Norwich there were several stormy meetings when the idea was first unveiled in 1997, before the Bristol scandal broke.

But many doctors were talked round and their involvement was crucial, not only in producing a scheme which could be fine-tuned to the different specialties, but also in ensuring they were signed up to the principle of yearly appraisals. 'Most of the consultants have now come to the view that the scheme is not tough enough, ' says chief executive Malcolm Stamp.

South Tees Acute Hospitals trust medical director Professor Ian Haslock, who wants to extend appraisals to cover all consultants, rather than just clinical directors, says: 'It was predicted that doctors would have nothing to do with 360degree feedback and every clinical director did it.

Doubtless we will have more opposition as we spread it wider - but so far it has been widely accepted.

'I think the lesson for other trusts is that if you introduce things sensitively, doctors are less conservative than people think they are. You have to offer people opportunities to make constructive change.'

Undoubtedly the most controversial and problematic side of appraisals is assessment of a consultant's clinical performance. 'It is extremely difficult to adequately assess a person in terms of clinical care, ' says Dr Allan Cole, medical director at Glenfield Hospital trust in Leicester. 'It is impossible to measure how good a doctor you are - there are some people who are good at one thing but not at another.'

There is also concern that suitable data to assess and compare clinical work is not available: Mr Stamp says investment is needed in outcome data if it is to assist the appraisal process. 'Doctors are not worried about being measured, but they are worried about inappropriate measurements, 'he points out.

This approach is criticised by Alan Maynard, professor of health economics at York University.

He argues that, in the main, adequate data on consultants' activity is available and this can be adjusted for case-mix. But more information on patients' quality of life after treatment would make it more meaningful.

'I think the managers are being very conservative and it is the doctors leading the way, 'he says. 'Over time, appraisals will give us a much better picture of what individual consultants are doing. 'But until such data is used, appraisals will be 'breast-beating and nice little chats', he warns.

Marianne Rigge, director of the College of Health, argues: 'I don't think consultants are any different from anyone else . It is too easy an answer to say that we are not comparing like with like - we should be able to do that.'

She points to the vast amount of work going on around methods of revalidation and the Commission for Health Improvement as one area where suitable models for assessing clinical work may emerge.

Another issue has been the time appraisals take for both the appraisee and appraiser. As well as finding protected time for the appraisal interview, there is a considerable evidence-gathering burden - although existing material such as audit data is usually available. 'I think the consultants are very worried about the time and effort it will take to have a robust appraisal system, 'warns Dr Iain Brooksby, medical director at the Norfolk and Norwich.

But what difference do appraisals make? Proponents are reluctant to make great claims for these relatively new schemes and instead point to subtle changes in attitudes and approaches. Professor Sanger says: 'I think one of the great values to the trust is the much richer dialogue that there is between consultants on matters of appraisals.'

But he also suggests that appraisals can help the medical staff feel more valued - many consultants have said they don't otherwise get praised for their work. Appraisals may make doctors more aware of the needs and aims of a trust.

Dr Cole says : 'What we are trying to do is get people tied in. We don't want doctors to be isolated islands any more.

The whole process has to take into account hospital needs as well.' Appraisals can be an opportunity to talk about what a trust wants out of a doctor and to set objectives for the next year, as well as assessing current performance.

Some doctors also highlight lengthy one-to-one discussions with their clinical director as being valuable.

But the BMA is keen to point out how doctors' performance may be affected by facilities and resources - points which can also come out in appraisals.

One potential benefit is increased team-working, which Professor Haslock believes he is already seeing in South Tees, and many trusts would like to see team appraisals carried out as well as individual ones.

'I actually think that is where the biggest gains will be, ' says Dr Brooksby, suggesting it will enable a wider view than individual appraisals, with consultants encouraged to concentrate on areas where they are strong. Trusts should also be able to focus professional development where it is needed most, and develop the skills they need in the future.

At South Tees, developmental programmes are now informed by an assessment of the skills clinical directors will need later on. But appraisals are not panaceas and there is some concern that they won't pick up seriously underperforming doctors. Dr Brooksby suggests it can be a good way to deal with minor issues: 'Problems can be flagged up earlier in a less confrontational manner. I think appraisal, to some extent, should be a touch on the tiller.'

A particular failing of voluntary systems is that only 'good' doctors may choose to get involved. 'There is a concern that those who need to be appraised probably aren't and those who don't need to be are, ' says rheumatologist Professor David G I Scott, former chair of the consultants' staff committee at the Norfolk and Norwich. 'There is always a concern about how quickly would we pick up Bristol. Would an appraisal process really do it?'

Accident and emergency consultant Keith Walters chose to put his treatment of cardiac patients - which he found one of the more difficult parts of his work - under the microscope when he agreed to take part in the voluntary appraisal scheme at the Norfolk and Norwich.

'In A&E you rarely get feedback unless something has gone wrong - this gave me the opportunity to look at what had happened, 'he says. He traced patients he had seen with suspected cardiology problems and a colleague reviewed his treatment of them, informed by the eventual outcome.

Mr Walters says it was a useful exercise - not only because his clinical practice was examined and discussed, but because it set him thinking about what more could be done for some of the difficult cases. Having identified cardiology as an area he found difficult, he also went on a four-day course and feels his ability to interpret ECGs has improved.

As the second part of the appraisal, a questionnaire about his interaction with other staff was sent to a crosssection of people he worked with. He was, he says, pleased to find 'I have a bit of a fan club out there'.

His experience has converted him to the idea of appraisal. 'I'm not half as negative as I was the first time round, 'he says. 'I'm not getting marks out of 10 - it is meant to be allowing me to say this is what I'm good at and this is where I can improve.'

One drawback he found was the huge amount of time involved, not only for him and the colleague reviewing his work, but for his secretary and the medical records division. He also feels that being able to pick someone he knows and trusts as his appraiser made him more comfortable with the idea of appraisal.

Professor Scott is halfway through being appraised at the Norfolk and Norwich - collecting evidence about his work and getting the views of colleagues but not yet completing the process with an appraisal interview.

'It is an exercise in finding out things about myself that I already know - although I think that can be very useful. I have certainly found it very helpful, 'he says. 'My problem is time management - I take on a lot of things that I don't have time for.

'But although I know my time management is bad, this makes me face up to it and what I can do about it.'

As well as assessing his clinical practice, he is concentrating on his management skills - one of four areas the Norfolk system examines in rotation. His appraiser talked him out of looking at research, which is a major part of his work, because she felt it would be a 'soft option'.

The appraisal involves collecting anonymised views about his management abilities from other people within the department, including secretaries and receptionists as well as medical staff.

'It was no surprise to me that my information technology skills scored badly. But it was a surprise when one person scored me badly for not treating people with equal respect - that has made me think, 'he admits.

'One thing about being appraised is that it has made me think about my future. It's important to think about where you want to be in five years' time. Overall, I'm very positive about appraisals.'

Voluntary service: Norfolk and Norwich Health Care trust Around 40 of the Norfolk and Norwich's 148 consultants have undergone voluntary appraisals in the past year, and 20 others have started but not completed the process. The scheme was introduced after a series of meetings with consultants, and some consultants were involved in drawing up a scheme, together with Professor Jack Sanger, head of the centre for educational policy and leadership at Anglia Polytechnic University.

Appraisals cover clinical performance and one other area - either management/administration;

interpersonal communications/effectiveness; training/teaching; or research/professional development - in what is intended as a four-yearly rotation. A clinical reviewer observes and discusses clinical practice.

The appraiser is chosen by the appraisee from a bank of 20 trained by the trust - but potentially from outside the trust - and there are safeguards to ensure this is not abused.

There is a considerable evidence-gathering burden on appraisees, who present data on their clinical work. This can include audit and outcome review as well as softer data such as colleagues' and patients' perceptions.

They can also circulate questionnaires looking at the secondary focus of their appraisal - for example, asking medical staff, nurses and receptionists about their communication skills. One-to-one interviews with line managers, peers and juniors can also be organised and presented as part of the portfolio of evidence which forms the basis of the appraisal evidence.

The appraisals, which can be lengthy, and the evidence collected for them are seen as confidential, but a summary form is lodged with the human resources department to contribute to trust policies on professional development. Department teams are also encouraged to draw up team development goals.

The scheme is explicitly not linked to disciplinary processes and so far no major performance problems have been uncovered through it.

Par t work: South Tees Acute Hospitals trust The South Tees scheme is part appraisal, part management development tool. Unlike the two other schemes, it concentrates on management skills and currently only involves clinical directors.

It started by looking at what would be the likely role of clinical directors in the future and the leadership qualities and behaviour traits they would need. The trust's existing clinical directors then used a multi-choice questionnaire to get a profile of their own behaviour.

Comparing the two showed areas where clinical directors needed development to meet the demands of the role in the future.

In addition, clinical directors carried out 360-degree appraisals, seeking information about how colleagues perceived their management skills.

According to medical director Professor Ian Haslock, the clinical directors were happy to do this and were also willing to share the results of the appraisals with colleagues and other members of their teams - although they were all given the option of keeping the outcomes confidential.

The appraisal of clinical directors is likely to be repeated every 18 months and the trust is now planning to extend the system to other consultants. So far it has mainly been used to inform development plans - for example, clinical directors felt they would need greater negotiating skills in the future and that their feedback skills needed improvement.

Upping the ante: Glenfield Hospital trust

The Glenfield scheme was introduced over three years, beginning with simply reviewing consultants' job plans and growing into a relatively informal appraisal system. Medical director Dr Allan Cole says he introduced it 'by stealth - each year I have upped the ante'.

Consultants are appraised annually by their clinical director. The appraisal interview usually takes between an hour and an hour-and-a-half and covers reviewing the components of the job; personal 'needs' required to do the job such as additional equipment or staff; personal and professional training plan; individual performance assessment; and objective setting, which can be in any field but has to be capable of producing measurable achievements. It covers all aspects of a consultant's work. All consultants are meant to take part - new consultants have annual appraisals included in their job plans - but in practice probably only 80 to 85 per cent do.

Consultants present evidence about their work, but the detail of this varies. Some are keen on 360-degree assessment with views being sought from peers, line managers and juniors. Dr Cole describes the individual performance assessment as 'challenged self-assessment' where consultants are encouraged to collect the information themselves and then discuss it with their appraiser. Evidence on clinical work can come from a variety of sources - audit, information about volume of work done, outcomes and so on - and the appraisal can also cover softer subjects such as how medical students view teaching, communication with patients and so forth.

There is no formal appraisal form but those involved are encouraged to keep notes of what is discussed. Dr Cole says some parts of these are frequently shared with him or with other clinicians - such as training needs and objectives for the following year - but other parts are kept as confidential and may not even be recorded.

Dr Cole says any major problems with consultants' work should be picked up and tackled as they arise rather than waiting for an annual appraisal. The system is not linked to disciplinary action, but some minor problems with consultants' work have been picked up and addressed as a result of the appraisal system.