How North West London’s Health Secure Data Environment originated

In 2019, a crucial shift began to take place in the way NHS data was being utilised to support research and innovation in healthcare. Previously, the focus had been on sharing data to support research and development. However, there was a growing need to provide a secure, controlled environment for researchers and practitioners to access data while ensuring public trust. This change was in line with the broader national direction, particularly within the framework of Health Data Research UK (HDR UK) and the “Five Safes” approach to data security.

It was during this period that Ben Glampson, deputy director of iCARE, Dr Erik Mayer, director of iCARE and clinical reader at Imperial College London, and Kavitha Saravanakumar, director of business intelligence NHS in North West London, started exploring the potential of cloud-based infrastructure to support this new approach. The objective was not just to create a secure data-sharing access model but also to demonstrate direct patient benefits through real-world use cases.

One of the earliest projects involved using machine learning techniques to enhance the completeness and accuracy of smoking status for patients as recorded in the electronic health record, without the need for additional coding by healthcare professionals. Through natural language processing, it improved structured data capture, increasing smoking status recording rates from around 40 per cent to 96 per cent. More importantly, this initiative helped identify active smokers and support them with nicotine replacement therapy and referral to community smoking cessation programmes. This project funded by the National Institute of Health and Care Research Imperial Biomedical Research Centre had just gone live in a respiratory ward at Imperial College Healthcare Trust when the covid-19 pandemic struck, reshaping priorities overnight.

When covid-19 hit, the need for rapid access to sector-wide data became more urgent than ever. In a matter of days, a “gold command” was set up, bringing together key North West London stakeholders, including those from Imperial College Healthcare Trust and the North West London Business Intelligence function. The focus was on leveraging data assets across the region to deliver real-time insights that could inform the pandemic response. Questions needed immediate answers: How many patients had covid? Where were they in the hospital system? How could we alleviate pressure on intensive care units by predicting covid-19 positive hospital bed occupancy? The crisis required an agile, cross-sector approach, and it became clear that existing data silos were impeding progress.

This urgency led to the integration of Whole Systems Integrated Care (WSIC) data with granular acute hospital data held within the iCARE Secure Data Environment hosted by the trust, a critical milestone. WSIC had already been established as a linked longitudinal dataset, linking data from primary care, secondary care, mental health services, community providers, and social care. Originally designed to support population health management, WSIC had gradually expanded to encompass clinical analytics, operational insights, and research. During the pandemic, it became a key data asset for tracking covid-19 cases, predicting future outbreaks, and managing vital resources like intensive care unit beds and oxygen supplies.

At the height of the crisis, many business intelligence teams were redirected to pandemic-related duties, with some even running incident rooms. Ms Saravanakumar personally coordinated analytics support, working closely with academics to develop predictive models to aid management of hospital resources during times of increased pressure. These models helped answer critical questions: Who was at risk today? Who would be at risk tomorrow? There was limited real-time operation data available to support the analysis. The WSIC team set up situational reports (SITREP) to get the covid-positive patient list, bed occupancy, and patient outcomes from providers, which were ingested into the data warehouse and linked with the wider linked dataset to support the advanced analytics.

As the pandemic unfolded, the need for a shared, scalable platform that could streamline data analytics across the sector was realised. The digital innovation hub status awarded to our system by Health Data Research UK provided an opportunity to re-platform, aligning our efforts with Imperial’s ongoing infrastructure transformation. By joining forces, we could achieve both financial efficiencies and intellectual synergies. This collaboration laid the foundation for iCARE Digital Collaboration Space – a platform designed to support translational clinical analytics, research, innovation and real-world evidence by hosting multiple databases across tenancies within secure data environments. The project was solidified as a Digital Health theme priority within the National Institute for Health and Care Research funded Imperial Biomedical Research Centre.

Selecting the right technological architecture was critical to supporting not only the short-term requirements but also ensuring it was “future-proofed” for the rapidly evolving digital transformation of healthcare and interdependent digital health research and innovation. After a thorough options appraisal, a Microsoft Azure and Snowflake-based infrastructure was selected, which offered greater flexibility and scalability. This decision marked a turning point, providing a robust foundation for integrated data access and analytics.

One of the most impactful early applications of iCARE SDE was in addressing vaccine hesitancy. Using insights from WSIC and acute care data, we identified communities that were less likely to accept the covid-19 vaccine. This led to targeted engagement efforts, including town hall meetings where community members voiced concerns and sought reassurances. These discussions were not always easy – some community members questioned why healthcare authorities were suddenly reaching out. However, this initiative underscored the power of data-driven public health interventions and reinforced the need for transparent, inclusive dialogue.

Since then, the iCARE SDE has continued to evolve. Processes that initially required extensive manual effort, such as identifying patients eligible for covid-19 vaccinations, have become largely automated. The availability of curated, standardised data has significantly accelerated direct patient care interventions, operational efficiencies, and research projects across north west London. Today, iCARE and WSIC serve as critical enablers for evidence-based healthcare, supporting everything from seasonal vaccine rollouts to predictive modelling for future public health challenges.

Reflecting on this journey, the integration of Imperial and WSIC data into a single Secure Data Environment was not just a technological achievement but a fundamental shift in how data is leveraged for patient and public benefit. The move from what in 2020 began as a single-institution research high-performance compute environment has since become iCARE, a shared, multi-tenant platform. It sits across Imperial College London and the trust, strengthened by the quality of NHS data, and partnerships within Paddington Life Sciences. What began as a response to an immediate crisis has become a lasting framework for innovation, capable of supporting both regional and national projects and ultimately extending its value well beyond north west London.