• UK medicines and devices regulator reveals “concerns” over Babylon tool
  • Two senior MHRA figures say issues raised by consultant “valid and ones that we share”
  • New regulations for medical devices under development

A regulator has admitted “concerns” over the software Babylon Healthcare uses in one of its digital health solutions and is exploring how to address this.

The Medicines and Healthcare products Regulatory Authority’s concerns relate to Babylon’s symptom checker “chatbot” tool. This is used by thousands of patients, including those registered with digital primary care practice GP at Hand.

Two senior figures within the agency set out the MHRA’s concerns about the tool in a letter, seen by HSJ, which was sent to consultant oncologist David Watkins following a meeting between the parties last October.

Dr Watkins has raised doubts over the tool’s safety for several years, including repeatedly documenting alleged flaws in the chatbot through videos posted online. However, last year, Babylon said only 20 of Dr Watkins’ 2,400 tests resulted in “genuine errors” being identified in the software.

In the letter, dated 4 December, the MHRA’s clinical director for devices Duncan McPherson and head of software related device technologies Johan Ordish said Dr Watkins’ “concerns are all valid and ones that we share”. 

This marks the first time the MHRA has admitted concerns about the tool after being alerted to the issue more than two years ago, as the regulator never comments publicly on individual cases.

In the letter to Dr Watkins, the two MHRA directors also said the regulator is further exploring some of the issues highlighted and the work could “be important as we develop a new regulatory framework for medical devices in the UK”.

New regulations on medical devices, which would include Babylon’s chatbot, were initially planned to be imposed last May. According to Dr McPherson and Mr Ordish, this would have had the “benefit” of raising the classification for the tool, which would have increased “regulatory scrutiny”.

However, the introduction of new regulations was postponed, and they are still being designed by the MHRA.

An MHRA spokeswoman told HSJ the new regulations would consider “all aspects of medical devices regulation, including the risk classification rules that apply to software as a medical device”.

She did not say when the new framework would be published, but she added its requirements would need to be met by July 2023.

In their letter, the MHRA leaders stressed Dr Watkins’ “concerns are being taken seriously and if there is action to be taken, then we will”.

Asked for his thoughts on the MHRA’s letter, Dr Watkins told HSJ: “It is refreshingly candid.”

He said based on the discussions that had taken place and the response from the MHRA, the agency had acknowledged both the concerns being raised and the limitations of the existing regulations in addressing those concerns.

He added there was “no quick fix to resolve these issues”, and until a system of “regulatory approval” is in place “we’re reliant on healthtech companies behaving responsibly”.

A Babylon spokeswoman said the company meets all the local requirements of regulatory bodies for the countries it operates in, adding: ”Babylon is committed to upholding the highest of standards when it comes to patient safety.”

She continued: “We have supported a number of regulatory bodies across the industry by offering our…insight to help shape the future of the digital health regulatory landscape so it strikes the right balance between oversight and innovation.”

The company has partnerships with three NHS trusts, and hosts seven clinics across London and Birmingham under the GP at Hand service.

Updated 7.25pm 7 March 2021 to include full text of the letter to Dr Watkins, with contact details removed.

Text of letter to Dr Watkins in full

Dear Dr Watkins

Thank you for meeting us and writing the notes and your follow up email. Your notes are a good reflection of the meeting. Your concerns are all valid and ones that we share.

I would like to introduce Johan Ordish, who recently took up post as Group Manager (Medical Device Software and Digital Health) here in the Devices Division of the MHRA. He and I have discussed the issues that you raised about Babylon and he has now read the details of your concerns and the other documents that we have on file. As you may recall from the discussion that we had recently, the regulatory changes that we were expecting to see implemented in May are now not coming into force in the UK. This would have had the benefit of raising the classification for many types of software medical device products including the Babylon chat bot and with it the regulatory scrutiny. You have raised a complex set of issues and there are several aspects that fall outside of our existing remit. However, this highlights some issues which we are exploring further, and which may be important as we develop a new regulatory framework for medical devices in the UK. As we have previously advised you, confidentiality unfortunately binds us from saying more on any specific investigation, but please be assured that your concerns are being taken seriously and if there is action to be taken, then we will.

It is a matter of immense frustration that I am not allowed to be as open with you as I would like. We are shifting the law in this area, so that in future we will have much more ability to discuss issues with people outside MHRA. I’d like again to thank you for raising these issues publicly at considerable risk to yourself and will continue to offer whatever support I can.

Your sincerely

Dr Duncan McPherson MBBS FRCA, Clinical Director Devices

Johan Ordish, Head of Software Related Device Technologies

Source

Information obtained by HSJ

Source Date

March 2021

Topics