The friends and family test has been described as an unreliable means of comparing hospital healthcare. The finding comes in a new in-depth study, released weeks before a review into the tests is to be published by NHS England.

The analysis by Picker Institute Europe, seen exclusively by HSJ, revealed data generated for the friends and family test is so skewed by different collection methods it “cannot be reliably used to compare performance”.

The study – due to be published next month in the Journal of Health Services Research and Policy – joins a growing list of reports that have criticised the test’s reliability, undermining prime minister David Cameron’s ambition that it be used as “a single measure [for] quality of care across the country”.

Picker director of research and policy Chris Graham said that while the test could be useful within trusts where data collection methods were kept consistent, it was “not suitable for comparing between trusts”.

“This is further compounded by the demographic differences we found in our statistical analysis, which imply that caution is needed even when looking at trends locally.”

The skew caused by different collection methods has been uncovered through analysis of results from several hospital trusts.

Medway Foundation Trust saw its score fall substantially after switching from paper feedback forms to text messages and automated phone calls, introduced in a bid to increase collection rates.

Bedford Hospital Trust also recorded a sharp decline in its score after making the same change.

The trust’s average score of 73 between April and July last year fell to 39 the following August to December after it switched from paper forms to a text message collection

technique. The test score ranges from -100 to +100.

The critique comes weeks before NHS England is due to publish a review of its controversial patient satisfaction survey.

NHS England is soon expected to publish revised guidance for trusts on how they collect data and use the test, HSJ understands.