An investigation has uncovered ‘significant administrative lapses’ in processes used by the NHS Information Centre to oversee the release of patient records to organisations including research bodies, government departments and insurers.

The probe - based on the findings of an audit of 3,057 data set releases between 2005 and 2013 - criticised the “loosely recorded processes” of both the centre and a private company to which it outsourced work.

The investigation was launched by the Health and Social Information Centre, the body which replaced the Information Centre in April last year, following a raft of confidentiality concerns, including claims that records were being erroneously sold to insurers.

The Health and Social Information Centre said that it still had data sharing agreements with three re-insurance companies which allow these firms to continue using the data until the agreements expire in 2015 and 2016.

It said it was confident legal changes under the Care Act 2014, which restricts the flow of potentially identifiable data solely for purposes that benefit health and social care systems, would provide protections to prevent the misuse of data.

The organisation said it had written to the three re-insurance companies to ask them to delete the data ahead of the new legislation coming into force later this year, although the centre was yet to get a formal response.

An audit by PwC - commissioned as part of the centre’s investigation - concluded the failures were “not systemic”, and the body insisted there was no evidence that the law had been broken.  

But the investigation laid bare a governance regime prone to “lapses in the strict arrangements that were supposed to be in place to ensure that people’s personal data would never be used improperly”, according to the report.

The report added that between April 2005 and April 2009 - a period when hospital episodes statistics data releases were managed by private provider Northgate - PwC could “not find evidence that Northgate got permission from the [Information Centre] before making releases, as it was supposed to do”.

It added: “PwC could not find confirmation that the appropriate approval committee authorisation was in place for any of the sample tested.It is difficult to ascertain whether the problem was due to poor record keeping or procedural non-compliance.

“In the absence of evidence it would be unfair to conclude that Northgate was at fault, but it does appear that the [Information Centre] should have managed the contract with Northgate more tightly.

Record keeping improved when the Information Centre took over management of data requests from 1 April 2009, the report said, but auditors said they still “could not find records to confirm full compliance in about 10 per cent of the sample”.

Northgate said it “acted at all times under the express authority of its customer, the NHS Information Centre, in compliance with its contractual obligations”.

Sir Nick Partridge, a non executive director of the Health and Social Information Centre who led the review, said: “The Health and Social Care Information Centre must learn lessons from the loosely recorded processes of its predecessor organisation.

“The public simply will not tolerate vagueness about medical records that may be intensely private to them.

“Although there is a new board and largely new senior executive team, the [Health and Social Information Centre] inherited many of the [Information Centre] procedures and staff.

“This included data agreements with organisations - which have been highlighted by my review and which will subsequently be listed in future versions of the register of all data releases - first published by the [Health and Social Information Centre] in April.

“We can now make sure we conform to recent legislative changes so that data is released when it will benefit the health and social care system.”

The investigation’s publication comes as a group of experts established by the Institute of Global Health Innovation at Imperial College London concluded that people whose data is lost or “irresponsibly used” by the NHS or under the initiative should be able to claim compensation through the NHS Litigation Authority.

A Northgate spokeswoman said: “Northgate acted at all times under the express authority of its customer, the NHS Information Centre, in compliance with its contractual obligations, following a process agreed and approved by the NHSIC.

“Under this service, Northgate provided HES extracts and tabulations to third party requestors. The process agreed with NHSIC was that requests for non-sensitive / non-restricted data would not require case by case approval by the NHSIC. 

“Such requests were handled directly by Northgate in accordance with the agreed process. The process agreed with NHS Information Centre for requests that included sensitive / restricted data (such as requests for patient identifiable data) was that Northgate would request explicit approval by the NHSIC on a case by case basis.

“Requests for sensitive / restricted data were only enacted by Northgate following receipt of such NHS Information Centre approval.”