Sir Derek Wanless has declared the NHS has made lacklustre progress since its 2002 cash injection. Nigel Edwards looks at his findings in detail and considers the key lessons

Sir Derek Wanless's latest return to examine the NHS has some encouraging messages, but generally makes depressing reading. He finds the NHS is making lacklustre progress and that a number of key opportunities have been missed or not fully exploited.

Much of the rich analysis looks back at the last five years and provides a balanced judgement which acknowledges progress as well as problems (see Money Matters, below). The history is important because lessons need to be learned to avoid repeating the mistakes of the last few years.

The first strong lesson from the report is the failure of the system to collect data that would give a real picture of what is happening.

The most obvious example of this relates to changes in productivity. The standard method of measuring this uses cost-weighted activity - largely in the hospital sector - and says almost nothing about quality.

The original Wanless review made the important point that productivity needs to measure both quality and volume. But this data is not collected because there was not an agreed way of measuring the product of health spending or even, in a number of cases, what the product is.

Where it can be identified and measures can be defined, it is far from obvious how the data from different types of healthcare activity can be added together to provide an aggregate picture.

Looking forward, the system needs to be much clearer about what success looks like and what measures are required to show that progress is being made. Traditional measures that assume more admissions are equivalent to increased productivity are particularly unhelpful, especially where the direction of policy has been to reduce non-elective admissions and shift care into the community.

Other investment such as new drugs, for example statins, have increased spending and reduced admissions. Yet although they will have produced significant health gain this paradoxically appears to reduce productivity.

There is a related lesson make sure the baseline is measured before you make changes. Trying to reconstruct a baseline from data that happens to be lying around is generally very unsatisfactory, locally as well as nationally.

A further problem here is that the baseline from which changes in productivity have been measured was a period of high activity combined with financial austerity. The consequence of this was that the system looked fairly productive, even if it was close to breaking point, and the first tranche of new money was soaked up by the backlog of problems that needed to be put right.

What this implies is that productivity and efficiency improvement remains a key strategic priority for all parts of the system and, as Sir Derek points out, with a return to historical levels of growth - around 3 per cent in real terms - there will be need to be an even more significant push on this.

The second lesson is about policy-making and the management of change. There is a lesson about information here too. Very little evaluation of policy seems to have taken place, except by external agencies.

The report is damning about the way policy-making has failed to come up to the government's own gold standard of being forward-looking, innovative and creative, evidence-based, inclusive, joined up and properly evaluated.

The lessons here are important but will come as little surprise, particularly in relation to overly rapid implementation and a failure to understand how policy changes in one area have an impact elsewhere.

The report gives the example of the impact of care-closer-to-home policies on the finance of acute trusts, the roll-out of NHS Direct and the implementation of management policy on long-term conditions.

But it could also have mentioned trying to renegotiate consultant contracts and reduce waiting lists at the same time, or getting uptake of practice-based commissioning while primary care trusts are in turnaround.

The extent, frequency and poor execution of reorganisation also comes in for some fierce criticism and is blamed for a number of the difficulties with the development of commissioning.

The report makes a number of recommendations and stresses the need for measurement, data collection, experiments, strategic modelling and forecasting and the evaluation of a range of current policies including the national IT programme.

He calls for national service frameworks to be revised. The report contains many wise words and ideas about improving the policy-making process and I am attracted to the idea of experimentation and much more evaluation. While this is not the main thrust of the report, it is where some of the most interesting ideas are to be found.

The report is rightly more optimistic about the current policy framework, but cautious as it is early days and because sir Derek focuses his concerns on the absence of a national demand-management approach and problems with clinical engagement. His recommendation that we should be concentrating on making the current system work rather than indulging in further fundamental reform will certainly be very welcome in the NHS.

Sir Derek's view that the system as a whole needs to get much better at implementation, particularly in terms of financial control, workforce planning and information management also has resonance. Although at least in the first of these there is evidence of significant improvement, particularly in foundation trusts.

Like many in the NHS he is unsure how far some of the mechanisms of choice, competition, standards and targets will be effective and what the balance of central direction versus local discretion should be. More clarity on this would be helpful but we may need to allow time for experience and learning.

The report is very critical about the missed opportunities in workforce planning, pay and contracts. These arguments are well rehearsed and the reasons understood, but the report is not so clear about what can be done, partly because it is deliberately focused on reviewing the past. NHS Employers is supporting NHS organisations to realise the benefits from pay reform, but this is going to take time to produce results.

He is similarly critical about the lack of progress in getting the public engaged with health improvement and the failure to contain the growth in obesity. This is even more an issue for the future as every year lost in achieving progress compounds problems for coming years.

When Sir Derek discussed this issue at the King's Fund last week he argued that there is a need to look at incentives for getting the public, commissioners, local authorities and providers prioritising long-term investment in this area. This has always been a challenge as immediate service pressures tend to trump long-term expenditure pay-offs.

Ringfencing is a tempting way of dealing with this but Sir Derek, as an economist, is opposed to it in principle. If we agreed to ringfence one area a queue of other deserving causes will form and those with the sharpest elbows would win - and this may not be public health.

Sir Derek regrets, as may we, that his 2004 recommendations, which included radical changes in primary care, investments in workforce and a much more rigorous approach to policy-making, were not adopted.

The 2002 Wanless review took a long forward look at demand, financing requirements, risks and other expenditure drivers. His latest review was not able to do this, which is a pity.

Inside the Department of Health and the Treasury a similar exercise is presumably taking place. We are now asking NHS organisations to take a much more strategic view of the future and, in the absence of a revised set of Wanless projections, it will be very important that the DoH and Treasury share their assumptions in as much detail as and as soon as possible after publication of the spending review.

Our work with members and hints emerging from Whitehall suggest some of the assumptions being made will, as Sir Derek states, require demanding efficiency improvements.

Nigel Edwards is policy director at the NHS Confederation.

Money matters: the main findings

  • Funding will need to increase substantially to deliver the aspirations of the original review, even with improved productivity and self-management by patients.

  • The NHS will fall short in funding by£15.2bn,£9.2bn or£7.2bn by April 2011 depending on progress defined by Sir Derek as slow uptake, solid progress or fully engaged.

  • The NHS is in better shape than in 2002 to deliver improvements but this will be a huge challenge.

  • New pay contracts accounted for£18.9bn of the extra£43.2bn given to the NHS since 2002 - but there is 'very little robust evidence' to demonstrate 'significant benefits'.

  • The government has exceeded staffing targets - but more will be needed.

  • IT delays could undermine envisaged productivity gains.

  • Poor out-of-hours services are partly to blame for the increase in emergency admissions, the biggest acute sector rise.

  • Prescriptions rose by more than a fifth from 2002-06 but cost increased by 0.3 per cent.

  • Some 'clear and notable improvements' with more staff and equipment, improved infrastructure, reduced waiting times and improvements in coronary heart disease, cancer, stroke and mental health.

  • Mortality has fallen and life expectancy increased - but 'relatively low priority' has been given to public health, particularly an 'unforeseen' rise in obesity.

  • The NHS has failed on 'relatively modest' unit cost reductions. Broadly, unit costs have increased for all hospital services.

  • Policy implementation has been 'slow and uncertain' and pressure for results has led to policies being introduced without adequate preparation. The government failed to take full account of the impact of new policies on the system as a whole.

  • Questions remain over evidence for hospital reconfigurations.