He was the shadow man.
Mr Hadi rarely appears in public or speaks to journalists.
When he does, he sweats a lot and falls over his words.
This week, he will become president of one of the Arab world's poorest but strategically important countries.
Few of Yemen's 25 million people know much about the former general from the south. Or what he has done since being dragged into politics.
That may be this reserved 67-year-old general's strength as he prepares for an uncontested presidential poll on Tuesday.
He is taking over a country convulsed by years of ethnic and tribal divisions and which has never been truly united since north and south joined in 1990. That union is still wrought with tension.
But despite being Mr Saleh's deputy, he is relatively untarnished by his association with the autocrat who ruled for decades and who killed hundreds trying to put down a revolt over the past year.
"We support him because this is the best option for Yemen, because this will end the Saleh regime," said Mohammed Qahtan, a leader of the opposition Islamist party, Islah.
Despite its opposition to the now disgraced ruler, Islah itself is struggling for support, partly because it signed on to a deal under which Mr Saleh was given immunity in return for stepping aside.
"People are tired of Saleh's rule, and Hadi gives people a source of hope," Mr Qahtan said.
After backing out three times, Mr Saleh finally signed a deal brokered by Gulf nations, including the UAE, late last year to give power to Mr Hadi and quit politics peacefully.
A unity government has been formed and under the deal Mr Hadi is the only candidate for the presidential election on Tuesday, which is more a confirmation of a power handover than a real poll.
He will head a two-year transitional government.
Not an easy task in a country where militants, often linked to Al Qaeda, are gaining power and which grapples with tribalism, a separatist rebellion among the northern Houthis, poverty, a humanitarian crisis and a wrecked economy.
Critics say Mr Saleh tamed these problems with graft and brute force.
Mr Hadi comes to power, with the support of the opposition, as the Arab Spring reshapes the Middle East, bringing with it the seeds of democracy and accountability.
But Yemen's transition already raises doubts.
"This is just a one-person referendum on Hadi, which is exactly why there were revolutions in the Arab world in the first place," said Hussein Moghram, 28, who protested against Mr Saleh in Sanaa's Change Square for the past year.
Youth activists who - more than the established opposition - led the revolt against Mr Saleh are calling for rallies to oppose Mr Hadi's assumption of power. They see him as just a new face of the old regime.
"This man was the vice president under the person who killed protesters. He will become president because of his ties to Saleh," said Mr Moghram.
Analysts say Mr Hadi is adept at playing both sides.
He is close to a number of opposition figures such as Ali Mohsen Al Ahmar, an army general who turned his units on Mr Saleh last year.
At the same time, he has strong ties to the president's powerful family.
Balancing this will pose him many headaches, said Sarah Phillips, a Yemen expert at the University of Sydney's Centre for International Security Studies.
"Saleh and his family have made it quite clear they intend to retain significant influence, and I think they're hoping he'll be a pliant figure," she said.
Mr Hadi was born in Thukain, a small village in the southern Abyan Governorate. He graduated from Britain's Royal Military Academy Sandhurst in 1966 and then quietly rose through the military ranks of the then independent, Communist south, the People's Democratic Republic of Yemen.
In 1986, a brief civil war in the south convinced Mr Hadi to defect to the north. Mr Saleh, then North Yemen's leader, made him his personal military adviser.
The two briefly fought each as rival military commanders over the city of Taiz in 1972. But Mr Saleh found use of Mr Hadi's military skills and knowledge of the south, appointing him defence minister in 1994. That was when Mr Hadi helped crush the south's attempted secession from the union.
Although that made him few friends in the south, he was rewarded by Mr Saleh with the vice presidency in October 1994.
Since then, observers say he has done little. His adviser, Yahya Al Arasi, described this as an aversion to Yemen's cut-throat politics.
"He is very cautious and doesn't want to anger people," he said.
But there is doubt whether his cautious demeanour will be enough to tame Yemen. For some, his rise is a consequence of Yemen's dysfunctions, which augurs poorly for his ability to bring a semblance of stability, let alone democracy.
"The people are desperate for someone to save the country," said Sabri Muhashin, the former editor in chief of ß, a newspaper in the south.
"It's a like a sinking ship and the people are looking for someone to throw them a life raft, and Hadi just happened to be the man holding the raft."
& Hugh Naylor on