The chief secretary to the Treasury has stressed that all public sector employees “should be on the payroll and paying the correct amount of tax”.

Speaking after the government launched a crackdown on off-payroll salaries, having identified more than 2,400 cases of public sector staff being employed indirectly - including members of NHS boards - Danny Alexander said the situation had been going on for years.

Some 40 per cent of cases go back two years or more, with some going back as far as 10 years, he said. In 70 per cent of cases, people had been earning more than £400 per day.

Since January, 350 contracts using off-payroll measures have been ended - with 10 per cent of the people staying with departments under new terms and conditions.

Under the new rules, the most senior staff must be on payroll unless there are exceptional temporary circumstances, departments must seek formal assurance from contractors they are paying full income tax and national insurance, and if assurances cannot be given, contracts will be terminated.

“Tax avoidance and tax evasion make people very angry … everyone should be paying their fair share of tax,” Mr Alexander told ITV1’s Daybreak.

He added: “It’s been going on for years - it was going on under the previous government - and it needs to stop right now.”

Mr Alexander’s review looked at all government departments, their arms-length bodies, and the boards of NHS trusts. Devolved administrations, local authorities and the BBC were not included in the review.

The chief secretary’s review did not seek to identify tax avoidance but only off-payroll contracts - he said these were “opaque” and tax avoidance could have taken place.

Mr Alexander said: “The choice to be on payroll or off-payroll shouldn’t be available to anyone who is doing a normal public sector job.

“People want to know that public officials working for the government are paying their fair share of tax, just like other people are around the country.”

Information from the review is to be shared with the HMRC, with MPs free to pursue any case worthy of consideration. The new rules will be implemented by September this year and will apply to any existing contracts as well as new ones.

Speaking to MPs on Wednesday, Mr Alexander said: “These new tighter rules will be monitored carefully. Any department that does not comply will be fined up to five times the cost of the salary by the Treasury.”

He told them “it is vital everyone pays their fair share” as public finances are so tight.