Mr Pistorius, the interior minister of the state of Lower Saxony, was the surprise choice to replace Christine Lambrecht after she bowed to heavy criticism and resigned on Monday.
He could meet his new Nato colleagues as soon as Friday at a key defence summit in Germany, where all eyes will be on whether he approves the export of battle tanks to Ukraine.
Another task is to modernise the German military after years of underfunding and equipment problems, with a €100 billion ($108.2bn) fund available for new hardware.
"I want to make the armed forces strong for the period that lies ahead," he said in his first remarks after his appointment.
Admirers of Mr Pistorius, 62, called him a respected operator who knows the uniformed services from his time overseeing policing in Lower Saxony, a state with many military barracks.
“Pistorius is an extremely experienced politician who is tested in government, has dealt with security policy for years and is exactly the right person thanks to his competence, his ability to deliver and his big heart,” said Chancellor Olaf Scholz.
A lawyer by training, Mr Pistorius did a year of military service when it was the norm in West Germany, but has never held office in national politics.
He last year lobbied Ms Lambrecht successfully to have a new territorial reserve stationed in Lower Saxony.
Mr Pistorius is “a political pro” who “tackles crises in an efficient and purposeful way … he is the right person for this challenging job”, said MP Christos Pantazis.
Like the Chancellor, he is a member of the Social Democratic Party (SPD) and was involved in the coalition talks that put Mr Scholz in power in 2021.
He was once the mayor of Osnabrueck, where Mr Scholz was born, but admitted he was surprised to be approached for the defence role.
Some sceptics pointed out that Mr Pistorius called sanctions on Russia into question in 2018.
Mr Scholz’s government has been accused of moving too slowly to equip Ukraine and criticised for holding back the Leopard tanks that Kyiv has long demanded.
Mr Pistorius will oversee the delivery of Marder fighting vehicles while deciding whether to send the Leopards, after Britain confirmed it was offering Challenger battle tanks to Ukraine.
Pundits will also be watching whether Mr Pistorius can avoid the public relations gaffes that helped to bring down Ms Lambrecht, such as a personal New Year’s Eve message that many saw as misjudged.
The defence minister is commander-in-chief of Germany’s armed forces in peacetime.
Allies of Ukraine will meet at the Ramstein US airbase in Germany on Friday to consider their latest support.
Unless a wider reshuffle ensues, the appointment of Mr Pistorius to replace Ms Lambrecht will break a promise by Mr Scholz to run a gender-balanced cabinet.
“It’s a shame that the chancellor and the SPD have given up their self-declared goal of gender parity. There were several well-qualified women in the frame,” said Nyke Slawik, a Green MP.
Ms Lambrecht’s deputy Siemtje Moeller and the German parliament’s armed forces commissioner Eva Hoegl were among the women tipped for the role.