As we enter the new political season, the outlines of the debates affecting health and social care are becoming clearer. Many old friends - rationing, waiting lists, winter pressures - are still with us. But they are joined this year by a newcomer: freedom of information.
Why freedom of information?
There are several arguments in favour of greater freedom of information. One is based on a general principle that the state should act to protect and enhance the interests of a society. For over a century, following Gladstone's reforms of the civil service in the 1860s, the design of the civil service and other public institutions has been based on notions of trust - the institutions and the individuals who worked in them were guided by an ethical framework that would lead them to take society's needs into account.
Recognising that no person or institution is perfect, Gladstone built in accountability mechanisms such as scrutiny of public expenditure by the legislature, supported by the Exchequer and the Audit Department (later the National Audit Office).
But a reliance on trust, codes of conduct or scrutiny by quasi-state organisations may not always be enough to ensure good government. Many countries have found that additional lines of accountability are needed, as experience has shown that no institution is beyond reproach (see box, right). So, in addition to the usual performance and probity checks, accountability can be enhanced by a more direct route between the state and the public.
In the current debate on the Freedom of Information Bill, this principle is often replaced by a more modern one - that the actions of the state should be open to scrutiny by the rest of us as a matter of right. We think this redefinition of individual rights is reasonable, and captures the idea that our own perceptions of acceptable behaviour from public officials has changed.
The second argument starts with the observation that trust in public servants has been severely dented in recent times. There have been health scandals such as BSE, tobacco sponsorship and GM foods.
1These seem to be part of a wider set of problems, ranging from the Pergau Dam affair to sleaze at Westminster.
Some of these were unearthed by auditors and regulators, but we seem to have been alarmingly reliant on persistent journalists to identify problems and pursue them until someone in authority acts. When the story finally emerges, it transpires that in almost all cases there has been a serious conflict of interest.
It seems perfectly reasonable to argue that if early discussions had been exposed to wider scrutiny, then different decisions, taking account of the public interest, would have been taken.
Whichever argument one uses, the conclusion is that existing structures do not provide individuals with sufficient assurance that the state necessarily has our interests at heart.
In retrospect, policy changes in the first half of the 1990s did not help. The drive to split responsibility for policymaking and service delivery, and to allow 'managers to manage', focused attention on creating structures which, ministers believed, would lead to more efficient and effective public services.
But the hiving off and greater freedoms were not accompanied by serious strengthening of accountability mechanisms. There has certainly been an increase in audit and regulation activity, but there are many gaps and overlaps, many bodies do not have appropriate powers to root out problems, and there are serious questions about whether all these bodies are having a positive effect on the use of resources or quality of services. Even recently overhauled structures such as the NHS complaints procedure do not appear to be working well.
2The third argument for greater public access to information is that - in this increasingly cynical age - the public could become more engaged with government and the state as active citizens. For example, it is clear that when the public learns more about the complexity and difficulty of the decisions health authorities face they become much more sympathetic to managers' and clinicians' rationing dilemmas, and more supportive of public bodies' attempts to grapple with resource allocation decisions.
Your right to know The Labour Party swept to power in May 1997 promising 'a Freedom of Information Act, leading to more open government, and an independent National Statistical Service'.
For people interested in greater access to information held by civil servants and other public institutions, the early prospects were encouraging. David Clark - then Chancellor of the Duchy of Lancaster - published the white paper, Your Right to Know, which proposed that information held by public bodies should be made available to the public if requested unless it could be shown that it would in some way endanger individuals or the state.3An ombudsman would oversee decisions.
. . . and their right to say 'no' But David Clark lost his government post in 1998 and responsibility for freedom of information passed to the Home Office. It seems that home secretary Jack Straw is not a supporter of greater freedom of information, and when the bill was published in the summer it contained few of the progressive concepts in the white paper. Indeed, it was difficult to determine whether or not it was an advance on an earlier initiative, the code of practice on access to government information introduced in 1995 under the Conservatives (see box, page 26).
The tone of the bill suggested that access to the workings of the civil service and other public bodies would be very limited, and many issues were to be hidden from view altogether, such as large areas of social security and aspects of the work of the police.
Crucially, the power to determine whether information would be released was effectively returned to civil servants. These points were picked up in pre-legislative scrutiny of the bill in the summer. The home secretary appeared to make concessions on some points. But as the bill starts its passage through the legislature it seems likely that the resulting act will be closer to the original published bill than to David Clark's white paper.
Health policy Why does this matter in the field of health policy? One answer is that there are areas where the instinct of the parties involved seems to be to shield discussions from view, even though the result is that they simply store up problems. Eventually they come to the attention of journalists, and are then handled in such a way that nobody benefits.
The private finance initiative may be one such problem in the future, and (on past experience) aspects of information technology policy seem certain to be another.
It is not obvious why discussion of these policies remains off limits to the rest of us, not least because the NHS Executive is good at consulting in other areas of policy.
Openness would lead to more sensible debate and, quite possibly, more sensible policy outcomes.
Another answer is that there are powerful private vested interests in health and healthcare, and further conflicts of interest are bound to occur.
It is unclear how far NHS bodies such as the National Institute for Clinical Excellence and the Commission for Health Improvement will themselves be open to scrutiny.
The work of these bodies may matter a great deal, and there will be many parties, including commercial firms and consumer groups, wanting to know how decisions are arrived at.
But it is clear that health authorities and primary care groups will be obliged to disclose information about their decisions and actions. For example, even as the draft bill currently stands, HAs and PCGs will have to respond to requests for information about the way they allocate resources between services. Are all HAs and PCGs confident that they could supply coherent and defensible information on this?
And are trusts confident that they have access to their consultants' records of performance if they are asked to provide details of complication and death rates? And what about the way consultants prioritise their waiting lists?
But Jack Straw's desire to keep policy advice secret means that, as the bill stands, details of options for alternative funding of the NHS, the expansion in scope of charges, the reasons for the introduction of NHS Direct before evaluative research has been completed and a host of other national policy issues and decisions could remain undisclosed.
Accountability based on upwards reporting through audit, regulation and performance management does half the job we need it to do - at best. Most of the bodies in healthcare and elsewhere do not have powers to question government policy - and one way of introducing greater scrutiny is to open up policy discussions that occur mainly behind closed doors.
Freedom of information seems unlikely to be a panacea - at least in the short term. The history of NHS reforms is that they take time. A culture based on withholding information seems unlikely to change very fast.
Experience in other countries shows that freedom of information changes the balance of power between officials and individuals, but not to the extent that the role of officials is diminished.
The paradox in the UK is that the same clashes of interest that led us into BSE and other problems are manifested in the battle over freedom of information itself.
Ultimately, as Thomas Jefferson said: 'Whenever the people are well informed, they can be trusted with their government, for whenever things go so far wrong to attract their notice, they can be relied on to set things right.'
Setting limits: leading the way Stateside Several countries have put in place legislation that gives their citizens rights to information held by government. Most notable, perhaps, is the US which, since president Lyndon Johnson reluctantly put his name to the 1966 federal Freedom of Information Bill, has wrestled with the difficulties of freedom of information in practice. The Federal Bureau of Investigation currently has 15,000 outstanding information requests with a minimum wait of more than two years.
Other countries have been slower off the mark, but Ireland, Australia, Israel and Japan have all introduced similar acts in the past few years.
New Zealand took progressive steps to open up the workings of government and the civil service in the mid-1980s. The Official Information Act of 1982 was part of a set of initiatives aimed at encouraging greater accountability and public participation in government. The act stated that 'information is to be made available unless there is a good reason to withhold it'. This is in marked contrast to the prevailing attitude of British civil servants and ministers.
Official information is defined widely, and not only includes tangible records but extends to the collective memories of officials and ministers: officials can be quizzed as to their recollections of events and actions.
No freedom of information legislation anywhere in the world provides for unlimited access to all information. In New Zealand efforts have been made to define as small a set of restrictions as possible while still maintaining the essential spirit of the act. The grounds for refusing to disclose information are carefully worded.
In essence, information may be refused if it would prejudice national security, relations with another state, or maintenance of the law; seriously damage the economy; be contrary to the privacy of persons alive or dead; or endanger public health and safety. There are also some catch-all reasons to protect the maintenance of constitutional conventions, the conduct of public affairs and legal professional privilege.
While these reasons sound rather restrictive, a watertight case for non-disclosure needs to be given and appeals can be made to an ombudsman who has the right to see the information withheld and to ask the government to make it available if he or she disagrees with the reasons for withholding it. Not since 1987 has an ombudsman's ruling been overturned by the ultimate arbiter, the Cabinet (using an Order in Council).
The most common reason given for non-disclosure is that the proper conduct of public affairs would be jeopardised if ministerial or official advice were made public.
It has also been felt that the 'free and frank exchange of views' would be stunted if officials thought such views could be disclosed. In fact, much of this advice does become available under the act once it has ceased to be current .
Legislation does not guarantee freedom of information, of course - in the US the Clinton administration fought hard to keep the deliberations of the national healthcare taskforce closed to public scrutiny, and 'personal privacy' (a non-disclosure reason all freedom of information acts contain) is often used to withhold information.
Restricted access: the code of practice The forerunner of Your Right to Know - the code of practice on access to government - came into force in April 1995. The code made provision for anyone to request information held by the Department of Health and the NHS (as well as other government departments). The code places various restrictions on the information the government will provide and is subject to previous legislation, such as the Data Protection Act .
Most departments will charge for information and aim to respond to requests within 20 working days - but our experience suggests this is optimistic. Even a request for simple information on the number and type of requests the DoH has handled since the introduction of the code has remained unacknowledged for more than five months. Complaints about the quality of the information service are dealt with by the same unit that is responsible for handling requests in the first place.
It is hard to avoid the conclusion that the code is a superficial response to calls for a penetrating and effective freedom of information act, and that it has done little to change the attitude that government information should remain secret unless there is a good reason to reveal it.
Act now: the arguments in favour Enhance probity Statutory bodies such as the National Audit Office are limited in the number of investigations they can carry out and cannot scrutinise government policy.
Improve policy-making The know ledge that policy advice could be made public should improve the quality of advice and increase the chances of spotting bad advice.
Improve accountability The accountability of elected politicians is limited (and that of unelected officials even more so). Increased access to government information adds to existing accountability mechanisms and can reduce the political 'distance' between the state and its citizenry.
Increase public participation Access to the workings of government and the evidence underlying policy decisions should erode the growing feeling of 'us and them'. It could also make clear to the public the difficult issues government has to grapple with and the real constraints under which politicians and public servants have to operate.
The forthcoming legislation on freedom of information is likely to be far less radical than originally envisaged.
Decisions over what information to release will rest with civil servants.
The legislation is likely to require primary care groups to respond to requests for information about allocation of resources between services.
Access to the workings of government and the evidence underlying policy decisions would enable people to become active citizens. It would also stimulate sympathy for managers' and clinicians' rationing dilemmas.
Current restrictions are at odds with government policy to modernise the NHS.
1 McKee M. Secret government revisited. Br Med J 1999; 318 (7200):171213.
2 Wallace H, Mulcahy L. Cause for Complaint? London: Public Law Project, 1999.
3 Chancellor of the Duchy of Lancaster. Your Right to Know. Cm 3818. Stationery Office, 1997.
4 Hood C, Scott T, James O, Jones G, Travers T. Regulation Inside Government . Oxford University Press, 1999.
Justin Keen is fellow and John Appleby director, health systems programme, King's Fund