The Act of Parliament that established the Healthcare Commission required us to assess on an annual basis the performance of every NHS organisation, taking account of the standards issued by the Department of Health. Out of this requirement grew the annual health check.

In this piece, I want to describe the thinking that led us to the annual health check and the lessons we have learned.

When I was asked to chair the Healthcare Commission, there was barely any tradition of regulation in healthcare, except in the case of professionals. Moreover, the word regulation was not commonly used; we were thought of as being in the inspection business. And inspection was taken to be synonymous with visiting. So, the assumption was that we would create an organisation with a large force of inspectors who would wander the land visiting places and looking for whatever might serve as evidence that the government's standards and targets were, or were not, being met.

I had a problem with this line of thinking. Not only is visiting resource intensive and therefore costly, it is also unlikely to provide patients and the public with the level of reassurance that they deserve (despite the popular assumption that it will). For example, the organisation could prepare for the visit, and visits would necessarily be infrequent.

There had to be a better way to enquire into the performance of the NHS that could penetrate the complexities and multiple manifestations of the beast. That better way was to use the one commodity that makes the NHS unique in the world - data. The NHS collects, and has collected for the past 60 years, a huge range of data. So have other bodies interacting with the NHS, such as the Litigation Authority, the Audit Commission, the Royal Colleges, and various patients' groups. All this data could be put together and analysed. Gaps could be identified and filled over time. A picture could be developed of what is going on. Superimposed on that picture could be a model of what good practice should amount to. Variations from the model could be identified and, from those variations, the risk posed to patients and the public by any particular organisation could be revealed.

Harnessing data

This was the genesis of what came to be called our "information-led, risk based" system of regulation. It had a number of things going for it. The commission could be a lean organisation - a third the size of other comparable bodies while regulating a much larger sector. It meant the idea of knowing what you are doing, particularly as regards the quality and outcome of care and the experience patients have of it, became an imperative for all NHS organisations (and a novelty for many!). It meant judgements could be based on evidence and that benchmarks of performance could begin to be developed. The notion of benchmarks is important because it suggests that what you are looking for is not some abstract standard of performance, but one that is reasonable in the circumstances, while never being allowed to fall below a given level. The system also allows you to compare like with like, and explore why organisations in similar situations can perform differently. In this way, good practice can be shared and poor practice identified and avoided.

But being information-led is not the same as having a regulatory model based wholly on information. Inspections (or visits) are a crucial part of the system. And they come in two forms. There are the ones triggered by our information - where we think there may be a risk that performance is below where it should be - and there are the unannounced visits, to take the temperature of organisations. This was an idea I borrowed from Terry Leahy, CEO of Tesco, who recommended using "mystery shoppers".

The final part of the jigsaw was the decision to turn the system on its head. The commission was not going to do the first assessment of performance, the boards of trusts were getting that job. This was not decided on lightly. Some feared that self-declaration of performance would be like most forms of self-regulation: self-serving and unreliable. Nor was it introduced because it was a cheaper way of doing regulation. No - boards are legally responsible for the performance of their organisations. This was the best way of holding them to account. Of course, they could try it on, as some feared. And some did in the first year. But then the penny dropped. We had thousands of pieces of data against which to test what they claimed. Significant variations between their accounts and our evidence meant a visit from the commission and an often painful reassessment. The system came to be respected and a new model of regulation began to take shape.

Scope for improvement

If the commission were not being abolished, what developments would I like to have seen? First, it has always been a regret that the standards that we must take account of only partly capture what is important to patients. I would like to see standards focusing on three things - the safety and quality of care, the outcomes of care and the experiences of patients. And all of these would be measured not by reference to processes ("Do you have a system in place?") but by outcomes ("What is actually being delivered?"). And, as I have said, the standards would reflect what patients and the clinicians who look after them think is important in improving health and healthcare. They should be grown organically from the bottom up, rather than stipulated by central government. This is a challenge that the Care Quality Commission now has to take on, so that the things it measures are those that matter to patients and those who care for them.

Second, it would be good to see the annual health check supplemented by an increasing ability to assess performance on a regular basis during the year - a movement from post-hoc annual audit to real-time surveillance of what is happening. We are getting there, but now it will be the task of others. Why is this important? There are at least two reasons. The first is to spot and head off another Bristol or Maidstone and Tunbridge Wells in the making. The second is so the regulator can provide everyone with an up-to-date picture of performance. And, I have no doubt that this is a crucial role of an independent regulator - to report publicly what is being delivered so as to hold those delivering healthcare and public health to account on behalf of those funding the NHS (taxpayers) and those receiving care.

Third, I would like the annual health check and our developing surveillance to enable us to tell the public what they really want to know. They are less interested in whether an organisation receives a score of "excellent" than whether they or a family member will be well looked after by this GP or that unit in this hospital. And increasingly, they want to know whether their care will be managed well as they travel along what is called the pathway of care. Currently, the annual health check does not take us far enough. This is a great regret.

The annual health check could yet provide it. Though our hands were slightly tied by the legislation that established the commission and limited us to assessing the performance of organisations rather than pathways of care, GPs' practices, units and services, we are slowly finding ways around this handicap. Our only real enemy has been time: the time needed to agree on the metrics, draw in the data, analyse it and make the results known.

Working with government

Last, I would have liked the annual health check and the broader role of the regulator to have been embraced more warmly by government. Regulation is a crucial element in the government's strategy to devolve responsibility from Whitehall. The regulator is the public's independent agent in making sure that all is well and, where it is not, that something is being done.

But, given the highly politicised nature of any discussion of the NHS, government both saw the need for the regulator and at the same time felt uncomfortable about it, particularly when it brought bad news. Regulation was sometimes seen as part of the problem rather than part of the solution. Now, of course, those days when regulation was seen as a fetter to progress and innovation have vanished before our eyes as the economy reels, and calls for greater regulation echo through the City. Let us hope the Care Quality Commission will be able to take advantage of this new understanding and cement the role of regulation.

For my part, I have no doubt that the case for information-led, risk-based regulation has been made and that, in one form or another, it is here to stay.

See Healthcare Commission longed for government's embrace