PARIS (Reuters) - The French government supports removing caps on household electricity taxes as it rushes to plug a massive hole in the country's finances, France's budget minister said on Tuesday.

"These caps have cost billions and billions of euros to French taxpayers," Laurent Saint-Martin told franceinfo. "With inflation falling below 2% and energy prices under control again, should we remove the caps? My answer is yes."

The government is due to present a budget bill later this week reflecting a 60 billion euro ($65.90 billion) belt-tightening drive to hit new fiscal targets in an unprecedented push to rein in France's spiralling deficit.

With cost-of-living worries a hot-button issue that drove many voters to back far-right and far-left parties in recent elections, the government is walking a tightrope as it seeks to identify the best levers to raise revenue.

The tax increase could amount to more than 32.44 euros per megawatt hour (Mwh), the levels in place before the 2022 inflation crisis, according to newspaper Le Parisien, but Saint-Martin said the hike would not lead to higher bills for consumers because of a fall in the base cost of electricity.

"Around 80% of our fellow citizens are on the state-regulated rate, and even without the energy cap, they will still have a reduction in their bill of around 10%", the minister said.

Prime Minister Michel Barnier has promised to reduce the budget deficit - which is set to reach 6.1% of GDP this year - to 5% by the end of 2025, but has said he will need to delay the target date for reaching the euro zone's common 3% deficit goal to 2029 from 2027.

Without an urgent U-turn, the deficit could rise above 7% next year, Saint-Martin said.

($1 = 0.9104 euros)

(Reporting by Tassilo Hummel; Editing by Christina Fincher)