The petition was filed by the M5S whips in both houses of parliament.
“This is a gesture consistent with what we announced in the last few weeks,” the whips said in a statement.
“We will try everything to stop a law that breaches constitutional principles and does not respect the will of the voters.”
The M5S says the law, which encourages parties to form coalitions, is designed to stop it winning the next general election, due by March 2018.
The M5S has explicitly ruled out forming coalitions with other traditional parties.
Founded by comedian Beppe Grillo and led by Lower House Deputy Speaker Luigi Di Maio, the anti-establishment movement has accused the ruling centre-left Democratic Party (PD) and former premier Silvio Berlusconi’s centre-right Forza Italia party of “electoral fraud” in passing the law.
Named after PD Lower House whip Ettore Rosato, the Rosatellum was also backed by the anti-euro, anti-immigration Northern League.
The new law will harmonise the current differing laws for the House and the Senate.
It will introduce a system that is two-thirds proportional representation and one-third first-past-the-post, aimed at favouring the emergence of a winner.
It also does not allow voters to pick their MPs, the majority of who will be chosen by party leaders.
There are fears the next general election could be inconclusive, even with the new law which places pressure on parties to team up.
PD leader Matteo Renzi said that the centre-left group is aiming to reach the 40 per cent threshold at next year’s general election so a grand coalition with Berlusconi will not be necessary to govern.
With ANSA