When I first began talking to primary care staff about the Croydon virtual wards project I had to explain to them that patients would only be offered admission if they ranked highly on predictions from a computer.
It is fair to say that this was met with a fair degree of scepticism - GPs were clear that they wanted admitting rights to their local virtual ward. I still work in A&E myself each month, so I could understand exactly why the GPs found it difficult to accept that computer predictions would be more accurate than their gut feelings about their own patients.
Last weekend, for example, I met two or three patients that I am convinced will be coming back to our A&E department many more times this year. Nevertheless, the literature is quite clear - predictions from clinicians are nothing like as accurate as the forecasts from predictive risk algorithms.
Over Christmas, the precision of predictive risk modelling was illustrated vividly to me in a conversation I had with my uncle Wyn who was visiting from North America. Wyn has spent his career working in the banking software industry. He told me that up until a few years ago, banks in the US used to stock each of their cash machines with $300,000 at the beginning of every week. At the end of the week, the remaining notes would be removed from the ATM, to be replaced with another $300,000.
These days, apparently, the banks use predictive risk algorithms that tell them to within ten bank notes exactly how much cash will be withdrawn from any of their ATMs across the country each week. So now they only need to stock each ATM with that specific amount, and no longer have to tie up billions of dollars unnecessarily. They also use similar techniques to predict how many customers will come into any branch on any working day.
But the banks did not stop there. They were interested in exploring the possibility of tapping into the wisdom of their staff in order to improve the accuracy of these predictions. So they began asking their employees to estimate how many customers they thought would come into the branch each day. Initially this information was simply recorded alongside the forecasts of the predictive risk algorithm. The number of customers who actually did come in was also recorded, and this information fed back to the employees.
Over time, comparisons were made between the algorithm output, the staff predictions, and the true number of customers. In this way, accuracy and bias weightings were established for each employee. Once the predictions from a particular staff member reached the point where they added predictive accuracy to the algorithm, their forecasts were incorporated into the algorithm in real time.
When it comes to selecting the case loads for community matrons, a number of commentators have called for the opinions of clinicians not to be completely dismissed. Predicting admissions is far more complex and nuanced than estimating numbers of bank customers. And the costs of errors in healthcare planning are infinitely more serious than just leaving too much money sitting in a hole-in-the wall.
However, using some of the techniques employed by the banks might offer a way to incorporate clinical opinion into predictive algorithms in a rational, evidence-based way.
A system could potentially be developed that enabled front-line staff in primary care, A&E departments and hospital wards to obtain consent from patients to record clinical forecasts in databases that were linked to predictive algorithms. This would make staff feel that their opinions were valued and influential, and add predictive value to the algorithm outputs.
On a cautionary note, the inverse care law warns that clinical staff in deprived areas tend to be more overburdened, and would therefore probably have less time to record their predictions in this way. To an extent, predictive algorithms could adjust for this phenomenon by adding extra weight to deprivation scores when calculating risk. However there would also be a strong case for offering special incentives to encourage staff in poor areas to record their views.
Geraint Lewis is a public health registrar, a visiting fellow at the King's Fund and a policy advisor at the Cabinet Office. In August he will begin a Harkness Fellowship in New York.