Published: 26/05/2005, Volume II5, No. 5957 Page 31
I have been guilty of squandering NHS resources. In the late 1980s, as a single-handed GP, I was awarded funding to undertake an evaluation of a cholesterol management pilot in general practice. The response rate to the patient questionnaire was impressive, but it was only when I attempted to write the report some months later that I realised my failure to collect any data on the ages or sexes of the respondents.
Without this information it was impossible to place the findings in any meaningful context. To this day the questionnaire occupies a 'pending' box in my university office.
Unfortunately, many colleagues have admitted to encountering similar problems. The difficulty is that when we undertake an evaluation of a new service, we often start by setting objectives for the pilot. We should instead work out conclusions or results we want and then work backwards.
Thinking backwards is about imagining a service has been piloted and the results of the evaluation produced. It involves writing down the conclusion you expect to reach from the pilot and running through the following questions: is getting this conclusion worth the effort? Has someone else done similar work? Has the planned pilot the capability to lead to this conclusion? Has the planned pilot the capability to account for alternative explanations for the results (for example, bias, confounding or chance)?
Recently I was asked to comment on a pilot of a diabetes management service in pharmacies in comparison with general practice. Patients saw the GP for initial assessment and were given a choice between continuing their follow-up in general practice or to attend a pharmacy.
The objectives of the evaluation were to compare glucose and HbA1C results (a measure of longer term diabetic control) between the two groups. At first glance the outcome measures seemed reasonably robust and a sample size calculation had even been undertaken.
But the conclusion the evaluation was designed to reach was that diabetic management in pharmacies was as good (if not better) than general practice in terms of sugar control. Unfortunately, by involving GPs in the selection process, it was likely the two patient populations would be dissimilar in terms of age, co-morbidities and cholesterol control. So the pilot did not have the capability of reaching the desired conclusion without some major investment in data manipulation.
Thinking backwards is also relevant to clinical practice. For example, in undertaking any diagnostic investigation it is important to consider the end result and whether the test will make any difference to that result.
In this context I am perplexed about why some GPs insist on measuring cardiac enzymes in patients with chest pain. If the clinical suspicion is sufficient to warrant undertaking a test, it is more sensible to admit the patient to the coronary care unit as soon as possible. As these tests are not 100 per cent accurate, performing them appears to represent an unwarranted and somewhat risky delay.
Unfortunately thinking backwards requires an acceptance that it is worth thinking in the first place.
Government targets and politically correct initiatives often seem to take priority. I have all too often encountered instances in which outcomes and objectives for a service evaluation are non-existent. When questioned, the response is 'It is only a pilot, not research'. Let's stop wasting money in this way. l Nick Summerton is a GP and reader in public health and primary care at Hull University.