Live updates: Follow the latest news on Israel-Gaza
During a meeting of the Lancashire Labour Party, Mr Ali was recorded suggesting that Israel had taken the October 7 Hamas assault as a pretext to invade Gaza, comments for which he later apologised.
“Following new information about further comments made by Azhar Ali coming to light today, the Labour Party has withdrawn its support for Azhar Ali as our candidate in the Rochdale by-election," a party representative said.
“Keir Starmer has changed Labour so that it is unrecognisable from the party of 2019.
“We understand that these are highly unusual circumstances, but it is vital that any candidate put forward by Labour fully represents its aims and values.
“Given that nominations have now closed, Azhar Ali cannot be replaced as the candidate.”
Labour had come under serious pressure after the remarks emerged, with the comments condemned by figures inside the party and political opponents.
Leader Mr Starmer has sought to move his party on from the Corbyn era, which was overshadowed by controversies over anti-Semitism.
But the party leadership had been pressed on why Mr Ali had not been immediately suspended after the comments.
Labour recently suspended MP Kate Osamor after she appeared to say the Gaza war should be remembered as genocide on Holocaust Memorial Day.
It is understood Mr Ali has now been suspended from Labour pending a party investigation.
The decision to withdraw backing for the candidate will come as a blow to the party, which had hoped to retain the seat after the death last month of sitting MP Sir Tony Lloyd.
Mr Ali, who is also a Lancashire County Councillor, has apologised to the Jewish community and retracted his remarks, which he described as “deeply offensive, ignorant and false”.
Frontbenchers Lisa Nandy and Anneliese Dodds had been out campaigning for Mr Ali in the constituency at the weekend.
Earlier, shadow minister Nick Thomas-Symonds told BBC Radio 4 that he believed Mr Ali had fallen “for an online conspiracy theory”.
Britain's Labour Party through the years - in pictures
Labour’s decision will add considerable uncertainty to the outcome of the by-election when voters go to the polls at the end of the month.
Also running in Rochdale are former Labour MP Simon Danczuk, now the Reform Party candidate, and George Galloway, of the Workers' Party of Britain, who is campaigning against Labour’s stance on Gaza.
About 20 per cent of the electorate and 30 per cent of the population of the town are Asian, with polls nationally suggesting Labour’s vote could be harmed by Asian people who are unhappy with the party over Palestine and its perceived support for Israel.