APPRAISE BE

Published: 20/10/2005 Volume 115 No. 5978 Page 6 7 8 11

Last month, HSJ arranged a historic first meeting of the heads of the three UK bodies charged with treatment appraisals. They joined others from medicine, industry and academia to debate the challenges ahead Delivering robust and timely assessments on the effectiveness and cost effectiveness of treatments is a tough challenge. Approaches to technology appraisals vary across the UK. The strengths and weaknesses of the different approaches and what can be learned from this collective experience were among the issues debated by our expert panel.

Different UK approaches DW The Scottish Medicines Consortium looks at all new drugs, new formulations and new indications for existing drugs. We make our assessment as close to launch as possible, accepting submissions before launch if possible.

That is probably our strength. If you can get the message out before prescribing has really taken off it is easier to manage prescribing patterns.

Because we are dealing with new drugs we work in partnership with industry, who have to make a case for their drug. A weakness is that we cannot do our own modelling like the National Institute for Health and Clinical Excellence. If we do not like the case made, we have to reject the case and let the company know why they failed to convince us.

We involve practising NHS clinicians and pharmacists and other relevant health professionals. We have three lay members out of a team of 30 and a patient and public involvement group to back them up and help patient-related organisations in making submissions. A critical appraisal of the evidence by the new drugs committee results in recommendations to the main SMC, where the lay members are represented. At this point, and not before, the patient submissions come in.

PH Do you include equipment?

DW We only look at medicines. Some of the same medicines are picked up by NICE, perhaps a year down the line, sometimes more. NICE reviews draw on a larger evidence base than was available to us so our decision will be reviewed.

RW In Wales, we appraise new high-cost drugs, but we also deal with prescribing strategy. We only meet quarterly and do two appraisals at each meeting. We focus on drugs that will have an impact on NHS Wales... and we avoid, at all cost, duplication with NICE.

Like the SMC we work in partnership with the industry, we involve the public and all our meetings are held in public, including the appraisal itself. The appraisal is carried out by a consortium called the Welsh Medicines Partnership, but the decision is made in public by the All Wales Medicines Strategy Group, a mix of lay members and professionals.

PH Do you appraise technological kit?

RW No, we rely on other bodies to do that. The cost threshold we use for drugs to be appraised is£2,000 per patient per year, but other factors such as volume of use may be taken into account.

AD In England, the technology appraisal programme is one of six NICE programmes and in six years has produced 95 or so pieces of guidance. This includes any intervention the NHS might use - ranging from newly licensed pharmaceuticals to, for example, an educational model for helping parents deal with disturbed adolescents. Most have been on pharmaceuticals in response to concerns that built up through the 1980s and 90s around both variability and access to effective treatment and quality of care. We have about 64 topics at various stages.

A major difference is that NICE is commissioned by the Department of Health to do the topics, we do not independently set our own agenda. Also, most appraisals are of more than one product, so we may consider a whole class of drug.

The evidence base varies enormously from two significant clinical trials for a new entity or a new indication for an existing drug, through to more than 900 in the systematic review for the atypical anti-psychotics.

When we first set up the programme in 1999, appraisals took around a year. But It is now more like 14-15 months, or longer if there is an appeal.

In the assessment stage we gather evidence from a range of sources, inviting stakeholders to make submissions. We also commission a systematic independent review through NHS research and development, which takes six to seven months, described as a short technology appraisal. Quality reviewers for this are at a premium.

The rest of the time is mostly taken up with the process of appraisal. This is where it is exposed to public scrutiny and comment, is interpreted by the standing appraisal committee and goes out to public consultation. It then comes back for second review and final interpretation of the evidence, at which point there is an opportunity for appeal.

NICE is subject to judicial review. About 35 per cent of final appraisal determinations are appealed against, mostly by consultees effectively saying they do not agree.

So the strengths are that we have developed a very robust process. We have tried to build in multiple opportunities for those who are going to be affected by the decision to take part, understand what's happening and influence it at fixed points. We have tried to make it transparent, to publish everything. The downside is the process is very long. How quickly we are commissioned, and our workload at the time, determines how rapidly we can start and the time to the issue of NICE guidance.

Every few years we ask for views on our methods and processes, which is what we are doing at the moment. But the responses always have the effect of lengthening the process because everyone wants more time to do things.

We have always tried to strike a balance between acknowledging all the facts and reasonable expectations for better engagement with keeping the process as snappy as possible, if a year can be regarded as snappy.

Now we are drawing on our experience, and that of others in the UK and internationally, to try and speed up the process, while retaining the reliability of the evidence base, comprehensiveness, transparency and the inclusiveness. That is a pretty tall order, but nothing is impossible.

Following our board meeting in September we announced that we had put proposals to the DoH for a new, shorter process designed to deliver guidance close to the point of launch of drugs and other technologies into the NHS.

The DoH is also reviewing topic selection with the aim of making it more slick and efficient, while still producing referrals that reflect NHS priorities. That is hugely important.

PH The introduction of new drugs is different from other health technologies such as equipment, which does not have the same regulatory structure.

PC There are different licensing requirements, but there may be similar access issues. It was originally envisaged that NICE would potentially look at drugs, diagnostics and devices. But pharmaceuticals have always had more reviews than their fair share: there are probably 10 times as many drugs looked at. This may result in a different access inequity as more drugs compared with other interventions become the subject of national mandatory guidance.

RW We were set up to look at drug appraisals in Wales because it counts for such a big visible percentage of NHS expenditure.

DP But That is a paradox because an appraisal usually results in more expenditure, whereas the things you do not evaluate probably will get forgotten or money will not be found for them.

PH It is a concern. The irony is that we are appraising drugs which already undergo rigorous assessment before coming to market and yet many other technologies are introduced with minimal evidence base.

PC Drugs are often very cost effective compared to inpatient stays, for example, but often get greater attention when it comes to budget scrutiny because they are externally purchased.

DW In Scotland there was a recognition that not all drugs provided good value for money. There was also a concern about postcode prescribing.

We try to ensure consistency with the NICE process, so have naturally gone towards similarcost quality adjusted life year ranges. If a drug is above a threshold of£20,000 cost/QALY, we are substantially more likely to say no. We have appraised 14 of the same drugs as NICE; only once did we come to a different decision and here the capability for NICE to model the data probably made the difference.

PC I think it is reasonable that the different processes give different answers and outcomes;

the nature of the processes and the comprehensiveness of the evidence bases differ.

So NICE has the benefit of the Rolls Royce of review processes, but it takes a long time to do it.

DW It is coming later in the life cycle, so it is quite plausible that new information would be available at that point.

RW We also have an appeals process and independent review option. Wales is signed up to NICE, but the AWMSG looks at issues relevant to NHS Wales that NICE has not picked up in a timely way.

DW NICE guidance is also adopted in Scotland and we try to avoid undertaking appraisals of drugs where the timing is too close.

RW We refuse to undertake an appraisal if we are not going to have a minimum of 12 months between our appraisal and publication of the final NICE guidance.

Topic selection AD The DoH controls the selection of topics for appraisal by NICE. The process is not hugely transparent, although there are published criteria.

These are around trying to identify significant interventions that are not treatments in the disease or condition area they are designed for, or are likely to push resource expenditure significantly in one direction or another, or are consistent with an aspect of clinical practice that is an NHS priority.

But even then judgement has to be exercised.

This is initially by the advisory committee on topic selection, a DoH committee on which NICE is represented that combines advice from the NHS, independent academic sources and DoH policy leads. The judgement is also influenced by NICE's available capacity. The final decision on which topics should go forward to ministers is by the joint planning group, a joint committee of NICE and the DoH.

It is important that what goes through is close to what any local health community would give as one of their major priorities. One of the aims of the DoH review is to ensure the structure consistently achieves this.

PC The referral time for getting topics into the appraisal system is absolutely key as it directly affects patients' access to medicines. The concern for England is that the referral process has started to become quite long and unwieldy.

It can take as long as the appraisal process itself, which is quite unacceptable. The UK is behind many other European countries in providing access to new life-saving medicines.

The process needs to be sped up, but it also needs to involve the right stakeholders, the evaluation criteria must be appropriate and it has to be very transparent, so it is clear how decisions are made. This is something the NICE appraisal process does incredibly well but which has slipped a bit in terms of the referral process.

Implementation PH There is giving the advice in the first instance, and then leading a horse to water to see if they will actually take it up. The evidence indicates not everybody reads the advice.

AD We know from various reports that implementation is variable across the country.

The NHS is directed to fund interventions recommended by NICE, but not clinicians to use them - that wouldn't be tenable or appropriate.

But while funding is inevitably a problem however much money is shuffled into the system, leadership is also important. If the message is not coming down from the top of the organisation that this is important to the quality of care [then] implementation will be slow.

PC The NHS needs to develop more robust systems to allow local organisations to monitor implementation and uptake of national guidance.

The great promise of the national IT programme is that it will allow a quantum leap in monitoring and measuring things like national guidance. But the NHS can't wait for this, it needs to exploit existing systems and solutions now.

Industry has taken a lead in this. Work by the Association of the British Pharmaceutical Industry and Roche on uptake of guidance shows that in secondary care the availability of national guidance is starting to have a positive impact on some cancer drugs, which more patients are now able to receive. But there still remain unacceptable variations across the country. And for some other medicines there has been little impact in terms of uptake. A lot more work is required to understand these differences and tackle them.

PH One of the ways round it in my specialty is to make it a condition of professional training that people are conversant with national guidelines in their area of practice, because you can tell the trainee what to do but not the consultant. For equipment we have done this with some success - not without turbulence - but I am sure a similar thing could be done for drugs.

AD Yes, to be successful in ensuring implementation we have to look at every touch point in terms of access and engagement on the healthcare chain. NICE's implementation systems directorate is currently trying to reach out into medical education and training. It is also about looking for the points at which incentives are most appropriate and useful. An example is the quality and outcomes framework which is having a significant impact on GP remuneration.

AD There is also the issue of how how you get the guidance through to the people who need it when they need it. UK health technologies appraisal is leading the world after a relatively slow start.

Much of this is due to its use of electronic technology and the internet, not only in making available the final guidance, but in giving access to the whole of the business processes and evidence as it is emerging.

Regional evaluation groups AD There are also regional evaluation groups.

These were set up prior to the national groups and continue to provide advice on new drugs, but they do not usually have the full skill sets to do a full evaluation on cost effectiveness.

DW There are still grounds for regional groups not only in implementation, but in obtaining local ownership for decisions coming down nationally, particularly with regional differences in access to technology and disease prevalence as in Scotland, with its Highland and island populations.

AD I very much agree with that, particularly in England where organisations often have to wait a long time for NICE guidance. NHS organisations are obliged to make their own decisions locally about new technologies and innovations in the absence of NICE guidance or while guidance is in the pipeline.

But some have hidden behind this and have not done that work, hence the term 'NICE blight'.

There is much more learning now about how to do good health technology appraisals and the 9- regional new drugs groups could learn quite a lot from what has been established at a national level. One difficulty is that groups like NICE and the SMC may have access to information that is commercial in confidence and may not be so easy to share on a wider basis.

But if NICE or others are going to do more rapid reviews and appraisals closer to the time of launch, we need to recognise that different levels of evidence are available at different times.

Orphan drugs AD Another issue is how we deal with expensive orphan drugs used to treat rare conditions.

Traditionally there has been no standard approach to their use and parts of the NHS have very different funding policies.

NICE has a citizens' council - 30 people brought together to represent the demographics of England and Wales as a sounding board for these tricky issues. We asked them: 'Should the NHS be prepared to pay premium prices for drugs for very rare conditions?' Some people believed this was a no-brainer. They said why shouldn't people with a rare condition have access to treatment.

Others though that to spend£100,000 on one patient for a treatment that often will not cure them is not a good use of limited resources. We presented to ministers on this earlier this year.

PC It clearly requires a separate approach, using different thresholds and some different measures.

DP I am not entirely in agreement with that. I do not think the arguments hold up.

PH It is not just orphan drugs, It is orphan technologies. We need to look at the overall impact, for example freeing up bed days, not just the cost of treating that individual.

DP But all economic evaluations do that. The difference is that where you have got a net high cost, you have to ask what would be the alternative to providing this treatment.

DW I am against having different cost thresholds, but if the treatment served as a bridge to another technology that might have a curative role, you could maybe argue the case.

RW It is very important we involve the public, so that justice is seen to be done.

Involving stakeholders DP Do appraisal committees actually use the evidence or arguments that patients bring?

AD It is important that patient groups get the chance to take part, but that like any other single group they do not determine the outcome. The question is what can patient organisations supply into the evidence base that you can't get from elsewhere? I find that understanding what has been lost in terms of quality of life and what is valued in restoring it is often quite useful.

DP But if that was true you would want to do it systematically and not every disease has a patient group. I am sorry if this would delay things further.

AD You are quite right. We have commissioned Birmingham University to pilot a methodology for a more systematic assessment of patient evidence.

This has some potential but is very expensive. I suppose we can say we should pursue it, but inevitably we have to make a judgement about where to spend limited resources.

AD There is also the issue of clinician involvement. When it comes to practising clinicians this is not so easy because you might be talking about a clinical community with 10,000 members. The pragmatic route has been to construct the advisory committee largely out of practising clinicians. Unlike the guidelines group, the appraisal committee is generic; it is not practical to make it representative of every clinical discipline. But our meetings include clinical experts and the relevant professional bodies are involved throughout.

PC There is a responsibility on all stakeholders to consider how best to engage with their members and colleagues so the case they put forward is representative. That can sometimes be difficult in larger organisations like the royal colleges.

PH I agree. One potential problem is that trusts are increasingly unwilling to release people to do this type of work. We need to remind trusts of their responsibility to the wider NHS. With the newer contracts in some of the bigger trusts where things are a bit tight, there is perhaps less freedom to engage outside the trust than before.

With foundation hospitals, especially, It is easy to see that responsibility being eroded. l ROUND TABLE PARTICIPANTS Chaired by Ann Dix Assistant editor, HSJ AD Andrew Dillon Chief executive, National Institute for Health and Clinical Excellence PH Professor Peter Hutton Past chair of the Academy of Medical Colleges and professor of anaesthesia, Birmingham University DW Professor David Webb Scottish Medicines Consortium chair RW Professor Roger Walker All Wales Medicines Strategy Group chair PC Paul Catchpole Chair of the Association of British Pharmaceutical Industry's national health technology appraisals/clinical guidelines user group and healthcare management director, Roche DP Professor David Parkin Head of the City University Health Economics Centre, London