The decision allows Mr Gandhi, 52, to seek readmission to the Lok Sabha, the lower house of parliament, where he has been a member for nearly two decades.
Mr Gandhi was convicted by a lower court in March of defaming all people with the same surname as Prime Minister Narendra Modi in remarks made during an election speech in 2019.
The court in Gujarat, Mr Modi's home state, sentenced him to two years in jail. Legislators who receive prison terms of two years and above are automatically disqualified from the legislature.
Mr Gandhi appealed to the Supreme Court after the Gujarat high court upheld his conviction last month.
The Supreme Court said there was no reason given by the trial judge for imposing the maximum punishment of two years.
“Except the admonition to the petitioner by this court in a contempt proceeding, no other reason has been granted by the learned trial judge while imposing the maximum sentence of two years,” the three-judge bench of Bhushan Ramkrishna Gavai, Pamidighantam Sri Narasimha and Sanjay Kumar said in its order.
The court, however, said that the leader’s remarks made at an election rally were “not in good taste”.
The Parliament Speaker is expected to revoke Mr Gandhi’s expulsion following the decision, which also leaves the Congress leader eligible to contest the general election due next year.
"Truth always wins. But whatever happens, my path is clear. I have clarity on what I want to do," Mr Gandhi said after the ruling.
"I thank those who helped us and the people who gave us their love and support."
The Congress leader, who represented the Wayanad constituency in Kerala state, had said "how come all the thieves have Modi as the common surname?" during a campaign speech in the southern state of Karnataka in 2019, referring to Mr Modi’s alleged links to the country's corporate leaders.
He also mentioned two fugitive businessmen, Nirav Modi and Lalit Modi, who are not related to the Prime Minister.
The defamation suit against him was filed by Purnesh Modi, a politician from the Prime Minister's Bharatiya Janata Party in Gujarat.
The state legislator and former minister in the Gujarat government said the remarks defamed the entire Modi community. The surname is common in the western state.
Congress party leaders and workers gathered at the party's headquarters to celebrate the Supreme Court ruling.