• Emails show regulators have been reviewing safety concerns about Babylon’s symptom checker software for months
  • CQC says 11 concerns have been received about Babylon’s service, including three about the symptom checker
  • Babylon says “vested interests” trying to stifle innovation “with anonymous criticism and mudslinging”.

Safety regulators are reviewing concerns raised about Babylon Healthcare’s symptom checker “chatbot” following multiple complaints, HSJ has learned.

Emails with one complainant, seen by HSJ, show safety concerns were raised with the Care Quality Commission in March last year and subsequently passed on to the Medicines Health Regulatory Agency, which has been leading the review.

The concerns relate to the clinical safety of the Babylon app’s triage advice given to patients in response to some reported symptoms, such as chest pain.

But in a statement Babylon suggested bogus doctors were trying to destroy the product.

It said: “A number of vested interests would like to see us fail, and often resort to anonymous attacks and false allegations to do so. Some claim to be doctors and are deliberately misrepresenting information, which raises serious probity and professional standards issues.”

It comes as the London-based digital health company, headed by former Circle chief executive Ali Parsa, continues its rapid expansion. Late last month, the company secured a major deal with Samsung to pre-install its video GP appointment and symptom-checker software on all new devices in the UK.

Babylon’s symptom-checker software is also an element in several NHS services, including GP at Hand in London and the NHS 111 Online service.

In one email to a complainant in September 2017, a CQC official said: “Concerns you raised with us about the Babylon symptom checker app are still under review between CQC and the MHRA as a matter of priority. I hope you will appreciate that we cannot report on the detail while our review and related investigations are ongoing. However, we will let you know the outcome of our deliberations when they are concluded.”

In April this year, the complainant was told “our deliberations are ongoing but we should be able to let you know the outcome shortly,” an email shows.

In another email in May, MHRA’s device software manager told the complainant that Babylon had been approached about the concerns raised and the regulator was awaiting a response.

The complainant told HSJ they had not been informed on any outcome of the review by either CQC or MHRA.

A CQC spokesman told HSJ that, in total, there had been 11 instances of concerns raised directly with CQC about the Babylon app, including three specifically about the symptom checker.

He said the CQC was “reviewing the information provided”, as it did with tens of thousands on concerns raised every year, but was not conducting a “formal review” of the Babylon app.

He said the CQC is not responsible for directly regulating clinical software, and concerns about the Babylon chatbot had been passed on to MHRA.

MHRA would not confirm a review of the Babylon chatbot was under way. 

In a statement, Babylon Healthcare said: “Babylon’s triage chatbot is fully CE marked and we work closely with the MHRA and other regulators.

“Every day, some two million people across Rwanda and a million in the UK can make use of world-leading AI for symptom checking and information services because they are registered with Babylon. 

“A number of vested interests would like to see us fail, and often resort to anonymous attacks and false allegations to do so. Some claim to be doctors and are deliberately misrepresenting information, which raises serious probity and professional standards issues.

“HSJ readers know that a patient safety culture means rigorous clinical testing, transparently showing results, and continuous audit and learning - exactly the approach with our AI. They also know how important innovation is for the NHS, and how a few people will always try to stifle it with anonymous criticism and mudslinging.

“We will stay focused on providing exceptional care – that why thousands are joining us every day, and an affordable and accessible health service is being put into people’s hands across the world through combining clinical expertise with Babylon’s technology.”

Regulatory jigsaw for digital health

At least five different public bodies, from national regulators to local commissioners, have been involved in assessing the safety and impact of Babylon Healthcare’s digital service over the past two years.

Some of these assessments are in the public domain, others are not. Some involved independent inspections, while others rely on self-assessment.

MHRA

Records show Babylon registered its symptom checker software with MHRA as a class 1 medical device in July last year, a category that includes devices deemed low risk, such as spectacles or bandages.

This was about 12 months after the company launched the first version of its symptom checker tool, pitting it against the advice of human clinicians in an in-house “live challenge”.

Babylon’s app also displays a CE marking, which is essentially declaration that the software meets European standard for that device.

Registration for class 1 medical devices and CE marking are both based on self-assessment by the manufacturer, against safety, reliability, and usability standards. MHRA guidelines state that registration does not imply ”any form of accreditation, certification or approval for the device”.

Post-market surveillance is also considered primarily the responsibility of the manufacturer, although users can raise concerns directly with MHRA.

Care Quality Commission

The CQC is responsible for regulating the health service offered by Babylon’s GP delivered through its app, but not the software itself.

In December last year, the regulator published its second inspection report of Babylon’s health service. The report stated that Babylon was not providing safe care in some areas (you can read the report in full here).

Part of the reports, mainly the summary, were removed before publication after Babylon challenged its accuracy through the High Court.

NHS Digital

Babylon was one of the first apps to appear on the NHS “digital tool” library, essentially a list of digital applications endorsed as appropriate for NHS patients to use, in March 2017.

To get on the library, an expert group within NHS Digital carried out review in early 2017 to ensure the app meets standards focused on making sure the company has systems for detecting, recording, and escalating any safety or security concerns about its software.

A more rigorous assessment for the app library has since been introduced. Babylon was undergoing assessment against this new standard, but this was halted when its app was withdrawn from the library altogether. NHS Digital has said the app was removed after it was decided the library would no longer promote paid services, not because of any faults in the app.

NHS England

Babylon is involved in two of NHS England’s national programmes, NHS 111 Online and the NHS app.

Babylon’s symptom checker forms part of NHS 111 Online pilot with the London Central & West Unscheduled Care Collaborative, and has been declared “safe” by the national commissioning body and LCW.

An assessment of the service was leaked to HSJ in January, but no other assessment has been published. NHS England said ”internal and external clinical testing, and independent review” was performed on all NHS 111 Online products.

The Babylon NHS app is still at an early stage of development.

NHS England has also intervened in Babylon’s GP at Hand partnership (see below), requiring the service be scaled back and not treat complex patients.

Hammersmith and Fulham CCG

Last year Babylon created a partnership, called GP at Hand, with a Hammersmith GP practice to deliver video consultation and symptom checkers service free as an NHS service.

The new practice has been enormously successfully in attracting new patients (growing more than four-fold in six months) but also drawn accusations, particularly from GPs, of cherry picking patients, fragmenting care, and undermining the NHS model for primary care.

For the CCG, which approved the sub-contracting arrangement that allows GP at Hand to operate, the surge in patients had also caused tens of millions in stranded costs.

The CCG, with the support of NHS England, is commissioning a £250,000 independent valuation of GP at Hand and its impact on patients and the NHS.