CCGs lack skills to commission research

As a life long health service researcher and consultant, I still sometimes bid for evaluation projects in the NHS, of a scale that I can do alone or with a colleague. It is very encouraging that, in spite of being strapped for cash, the NHS is still at least prepared to invest in evaluations to find out if initiatives are working.

If we do not know what works, we are hardly likely to increase the efficiency and effectiveness of the NHS and following the “New” without evaluation may mean wasting resources.

The impact of major changes in public health programmes or mental health services cannot be assessed after a few months and plans for impact assessment should include scope for follow on work 

It is, therefore, something of a shame that several of the evaluations I have looked at recently are substantially under scoped and under planned. The budgets are simply not adequate for the work to be carried out and the timing of some of these proposed evaluations is simply not right.

The impact of major changes in public health programmes or mental health services cannot be assessed after a few months. These are programmes which might be expected to have an impact over many years and plans for impact assessment should include scope for follow on work. 

Extent of cooperation

Those commissioning such research also tend to underestimate very substantially the extent of cooperation required from other parts of the NHS and other public agencies. I have experienced, in the not too distant past, one NHS provider refusing to part with data that might show whether another NHS agency was actually achieving its goal of keeping people out of hospital.

Those commissioning such research also tend to underestimate very substantially the extent of cooperation needed from other parts of the NHS and other public agencies

The cooperation issue is particularly important for mental health evaluations as there are a multitude of providers, including voluntary and private sector bodies, who might be helping patients outside what the NHS is doing. Not all of these providers will use the same, or indeed any, measures of patient outcome and so assessing their contribution will inevitably be difficult. Without their cooperation, it will not be possible to evaluate an NHS initiative in isolation.

In the public health sphere, the problems are potentially even more complex. There is relatively little measurement of public health, as opposed to public use of services, and so measuring a broad change in, say, local consumption of sugar, is likely to be particularly difficult. Even a potentially simple issue such as smoking cessation is complicated by private services, moves to vaping and gaps in our knowledge of local tobacco sales in an area.

Lack of resources

Lastly, there is the question of resources. Sometimes, a little funding is not really better than none at all and this can be particularly important for good quality research.

Half a course of antibiotics will not cure a patient of a disease but may encourage the bacteria to adapt to the treatment, potentially increasing the risks it poses quite substantially.

Sometimes, a little funding is not really better than none at all and this can be particularly important for good quality research

Providing a bit of money, often a very small amount compared to the money and resources tied up in the programme to be evaluated, will not generate a sound assessment of the effectiveness and efficiency of a new NHS service.

Lack of research skills

In trying to decide how this situation has come about, the most likely explanation is that new services and their evaluation are being introduced by bodies such as clinical commissioning groups and wider groupings of NHS commissioners without the necessary skills to commission research.

While there are national bodies dedicated to taking forward research or more direct, short term service evaluations, I doubt that the expertise to commission research exists in many CCGs. One possible solution would be for CCGs to hire in some research advice.

This could help strengthen research design and put the resources on a reasonable footing. Without it, there is a real danger that the NHS will compound misuse of funds on schemes that do not work by spending money on research that cannot tell whether this is happening or not!