Already bruised by MTAS, junior doctors are now feeling the effects of market competition - but did the DoH bungle its workforce planning, asks Noel Plumridge, or did it know what it was doing all along?

It was entirely predictable. Over 29,000 applicants in pursuit of 18,400 posts was always going to descend into an unseemly scrum this summer. Come the end of the month, when their guaranteed NHS employment comes to an end, around 10,000 doctors will be without training posts.

The junior doctors chaos appears, on the surface, to reinforce two widely held beliefs about the Department of Health. The first is that implementing big new computer systems is not exactly its strongest suit. The second is that they are pretty dismal at workforce planning, too; prone to blow large sums of money on ill-designed and virtually uncosted 'reforms'.

'Where did all the money go?' people ask as the years of major growth come to an end and Sir Derek Wanless's latest report confirms that pay inflation accounts for a goodly share, with hospital consultants taking a 25 per cent pay rise and the GP sector 23 per cent.

'There is very little robust evidence so far to demonstrate significant benefits arising from the new pay deals,' asserts Sir Derek.

More doctors than we need, it would seem, being paid somewhat more than necessary.

Inevitably the British Medical Association has called for an independent inquiry. 'We know that thousands of doctors have had their careers messed up,' says Dr Andrew Rowland, vice-chair of its junior doctors' committee. 'What we don't yet know is how much public money has been wasted on this failed experiment.'

But the shortage of junior doctor posts and the new system for allocating them are entirely separate issues, although this year they have combined to cause doctors dismay. Disentangle them and a rather different picture emerges. Maybe centralised DoH workforce planning has not been so naive.

Danger of failure

Let us first acknowledge that introducing the Medical Training Application Service software has not been the NHS success story of 2007. Simon Stevens wrote recently in these pages of the debacle of its 'cack-handed' implementation (to read Simon Stevens' column, click here). The story includes breaches of data security, shortfalls in the feedback process to unsuccessful applicants and countless computer crashes.

It is not unknown for new software to experience glitches, especially when the changeover is not universally popular among users. However, good project management predicts areas of high risk and attempts to reduce the danger of failure. Questions remain about the way MTAS has been implemented.

More fundamental are criticisms of the selection criteria used within MTAS. Sceptics draw attention to the scale of values used for selection: too much weight being given to creativity, too little to academic achievement and clinical experience. As one doctor friend pithily puts it, 'it was weighted in favour of those schooled in "self-reflective learning" (lying about your life experiences) rather than those who had been out there doing some cutting'. Far better, it is implied, to dispense with new-fangled trendy managerialism and leave selection to the old hands in the medical profession.

But even if MTAS had worked sweetly, surely that 10,000-doctor gap between applicants and vacancies smacks of bungling on a heroic scale? In March, the Commons health select committee report Workforce Planning was severely critical of a lack of joined-up thinking within the NHS.

Predictably, the committee wanted better alignment of workforce, financial and service planning in NHS organisations, including more accurate joint forecasting of future supply and demand, notwithstanding the competitive nature of the NHS that has emerged from the reforms in England.

In the case of medical staffing, the committee's judgement came hot on the heels of a DoH pay and workforce strategy for 2008-11 projecting a surplus of 3,200 whole-time equivalent consultants by 2010-11 (but a shortage of 1,100 junior and staff-grade doctors).

The solution proposed by the DoH was to encourage the creation, by foundation trusts, of new sub-consultant roles for doctors, such roles being 'more cost-effective than traditional consultant posts'.

So, having dramatically increased the number of doctors (and other groups of clinicians) in its labour pool and very significantly increased their pay, we are now witnessing the NHS forcing large numbers of newly trained doctors (and other groups of clinicians) to reconsider their career options. The labour market pendulum has swung. Go overseas to gain experience, pursue sub-consultant jobs with limited career prospects, or leave medicine altogether. And meanwhile the pay screw is being tightened, nationally, as if to demonstrate the old wisdom that a pay rise never kept anyone happy for long.

High stakes

But why? Cock-up or conspiracy? The answer is to be found in the original context of the NHS plan, which generated all those staff increases. As far back as 1999, the big challenge for the DoH came to be defined as increasing NHS capacity. This, it was believed, was the key to reducing waiting times, especially for hospital surgery, which in turn might improve flagging public perceptions of NHS performance. And politically the stakes were high. On it rested not mere economic gain but a considerable slice of New Labour credibility.

In practice, the DoH believed, more capacity meant more clinical staff. The modernisers' science of capacity and demand management was still in its infancy. A medium-term strategy took shape:

  • Improve NHS pay, both as a short-term boost to recruitment and retention and as a structural tool for squeezing extra productivity out of the service. Via the consultant contract, the general medical services contract and Agenda for Change, a pay system that had served the NHS tolerably well for 50 years was shaken to its foundations.

  • Encourage workforce flexibility and destabilise the NHS culture of professional silos, focusing especially on using the skilled but not registered labour pool to fill staffing gaps. The Changing Workforce Programme, established in 2001 as the tool for workforce re-engineering, had a predictably short lifespan: it disappeared along with the rest of the Modernisation Agency in 2005.

  • Overseas recruitment, in two forms. Initially we saw large-scale recruitment of trained foreign health professionals, most visibly nurses from the Philippines and Spain and doctors from Germany. Then came measures to open up NHS-funded provision to overseas competition, especially via the new multinationally owned independent sector treatment centres.

Charm offensive

Meanwhile, the market was quietly flooded with trainees. This was the crude but effective intervention that has transformed the labour market over five years and made the three previous measures largely superfluous. The new workforce development confederations, created to protect training funds from raids by cash-strapped local health economies, commissioned unprecedented numbers of training places.

Their job done, former health secretary Alan Milburn quickly sent them the way of other transitory NHS quangos. And by the summer of 2005, as trainees matured into qualified health professionals, unemployment stories were emerging. Two years ago, the BMA was already complaining of 'hundreds of doctors who have been turned down for hundreds of jobs and now have no idea what they are going to do'.

Far from being uncosted, economic aspects of the pay reforms were argued in some detail with the Treasury, which was, after all, putting up the money. The DoH business case for Agenda for Change presented in 2002, for instance, included among its key assumptions:

  • a more rational pay system would save at least 0.5 per cent per year in pay review body awards, which historically had been manipulated to address perceived inequities and perceived low pay;

  • each 0.5 per cent invested in pay produces a 0.25 per cent improvement in recruitment and retention;

  • NHS exposure to equal pay claims, on the basis of the then recent speech and language therapy judgements and the huge equal pay settlement by women health workers in Carlisle, was potentially£9bn.

All of which still appear sound. In the case of the consultant contract, much has been written about the overspend; less about the huge reduction in waiting-list initiative payments that it arguably made possible.

Overall the DoH strategy has been remarkably successful, which is why Gordon Brown can now rein in NHS pay with confidence. BMA chairman Dr Hamish Meldrum recently called for a fresh start with the prime minister aimed at improving engagement with the NHS reforms. And there may be one, but after so much government effort to rebalance its relationship with the medical profession it seems unlikely to take the form of a charm offensive. The chilly wind of market competition is new to junior doctors, but it is unlikely to go away.