For years, there has been no explicit law guarding medical records against casual or careless disclosure. Some of the most private and sensitive information in existence about UK citizens was protected by a mishmash of doctors' professional etiquette, the 1984 Data Protection Act, and the civil 'duty of confidentiality'.
The first of these has the drawback that it does not bind administrative staff, while the second suffered from the act's lack of any serious sanctions.
The third - the duty of confidentiality - rested on the Department of Health's bare assertion of it. It was not tested in court until 1997, when the DoH clashed with a company called Source Informatics.
Source was trying to persuade community pharmacists to give it copies of all prescriptions they dispensed. The prescription data - stripped of patients' names - was to be aggregated and analysed into reports of individual doctors' prescribing habits, under the brand name Xponent. Source hoped to sell Xponent reports to the pharmaceutical industry, which would then be able to aim its sales staff straight at GPs who were under-prescribing their most profitable drugs (a practice called 'targeted marketing').
In return the company offered pharmacists free equipment and software. Thousands of pharmacists, backed by the National Pharmaceutical Association, joined the scheme.
The DoH objected - ostensibly on the grounds that the plan risked a breach of confidentiality, even though the data was anonymous. So in July 1997, the NHS Executive issued a circular advising pharmacists (and GPs) not to participate.
1The circular naturally damaged Source's plans, and in 1998 it brought a judicial review case challenging the DoH's actions. Source claimed that it had proven its process was secure against any loss of confidentiality.
Source filed its request for review in 1998, and pressed on with its programme. But when the full judicial review was heard in the High Court in May 1999, the DoH won: the judge accepted that patients who did not consent to the use of their records should not have to face even the slightest risk of lost confidentiality.
Source - which by then had itself been bought by its main competitor, IMS Health - had to stop its programme immediately. But it appealed. According to Source/IMS spokesman Peter Stephens, the DoH seemed to be basing its argument solely on a division between public bodies, like the NHS, and private commercial ones, like drug companies. 'They seemed to be saying that collecting even anonymised patient data requires the patient's consent unless it is done in the public interest, and that the DoH's activities are automatically in the public interest while a commercial company's are not. We disagree.'
Source received the public backing of the Association of the British Pharmaceutical Industry and the National Pharmaceutical Association.
More unexpected was the intervention, on Source's side, of the General Medical Council and the Medical Research Council. All these bodies filed court statements to the effect that anonymous medical records were widely used without patient consent in many types of medical research, and a judgement that the practice was illegal would severely handicap science. In November, a group of leading epidemiologists published a letter in the British Medical Journal protesting at the decision, and demanded new, clearer, guidelines.
2Then, in December, the Court of Appeal heard the case. Unexpectedly, it agreed with Source and its supporters, and overturned the earlier High Court judgement. The DoH did not have the statutory power to control the use of anonymous data, said the court.
It also refused the DoH's application to appeal, with the three judges saying they saw no possible grounds. But the DoH is not best pleased. A spokesman says: 'The secretary of state remains concerned about the impact of targeted marketing on the drugs bill.
We have petitioned the House of Lords directly for leave to appeal to them.'
Was confidentiality really the issue, or does the remark about the drugs bill indicate the DoH's real concerns?
After all, it, too, collects and distributes prescription data collected by the Prescription Pricing Authority, and via the General Practice Research Database, for research purposes.
In the meantime, IMS/Source says it will resume its pharmacy data collection operations as soon as it can.
There is a further complication: the law changed on 1 March, with the coming into force of the 1998 Data Protection Act. The act contains specific provisions about using anonymised medical data for research, but it is still not clear exactly what it will and will not allow.
Perhaps the House of Lords will tell us.
1 Bid to ban scrip data sales. HSJ 1998; 108 (5602): 6.
2 Walton J, Doll R, Asscher Wetal. Consequences for research if use of anonymised patient data breaches confidentiality. BMJ (letter) 1999; 319: 7221, 1366.