MP Margaret Hodge told the department that it will need to "get a grip" on the issue after the National Audit Office (NAO) revealed that only £500 million has been saved over the last two years.
This compares to HMRC's own target of £1.4 billion and led to a spokesman for the department saying that progress is being made on the issue.
Head of the NAO Amyas Morse explained: "The tax credit system is complicated, and HMRC will have to overcome significant challenges if it is going to achieve value for money.
"HMRC deserves credit for demonstrating innovation, but it has further to go to achieve sustainable reductions in tax credits error and fraud."
The department hopes the introduction of Real Time Information into the PAYE system in April will help remove more fraud and error in the tax credit system in the future.
Posted by Thomas Fletcher