The vote was held after the country's Supreme Court on Thursday overturned Mr Khan's bid to dismiss the no-confidence motion and dissolve parliament on April 3.
Opposition parties were able to secure a majority 174 votes in the 342-member house in support of the no-confidence motion, the House Speaker said.
The announcement of the vote's result came shortly before 1am (8pm GMT) after multiple adjournments in the lower house caused by members of Mr Khan's party, who said there was a foreign conspiracy to oust the cricket star-turned-politician.
It was not immediately clear when a new prime minister would be chosen, but Pakistan Muslim League-Nawaz (PML-N) chief Shehbaz Sharif was almost certain to be picked to lead the nuclear-armed nation of 220 million people.
"We will put a balm on the wounds of this nation," Mr Sharif said immediately after the result was announced.
Mr Sharif, the younger brother of three-time prime minister Nawaz Sharif, has a reputation as an effective administrator.
Parliamentary elections are not due until August 2023. However, the opposition has said it wants early elections, but only after it has delivered a political defeat to Mr Khan and passed legislation it says is required to ensure the next polls are free and fair.
Mr Khan was widely expected to lose the vote after defections among members of his ruling coalition and MPs from his Tehreek-e-Insaf Party in recent weeks.
He surged to power in 2018 with the military's support, but recently lost his parliamentary majority when allies quit his coalition government. There were also signs he had lost the support of the military, analysts said.
The military viewed Mr Khan and his conservative agenda favourably when he won the election, but that support waned after a falling-out over the appointment of the country's next spy chief and the economic troubles.
"They [the military] don't want to be seen as supporting him and be blamed for his failures," opposition leader and former prime minister Shahid Khaqan Abbasi said. "They've pulled their support."
Opposition parties say Mr Khan has failed to revive an economy battered by Covid-19 or fulfil promises to make Pakistan a corruption-free, prosperous nation respected on the world stage.
No Pakistani prime minister has completed their full term since independence in 1947, although Mr Khan is the first to be removed through a confidence vote.