Mr Grundberg aims to ensure that all sides stick to a two-month nationwide truce that was extended last week and is aimed at ending the conflict.
The reopening of routes to the rebel-blockaded city has proved to be the thorniest problem in implementing a fragile truce.
He is meeting with Houthi rebel leaders to discuss a proposal he has put forward to end the siege on Taez.
Earlier this week Mr Grundberg shared with the parties “a revised proposal on the phased reopening of roads, including an implementation mechanism and guarantees for the safety of civilian travellers, based on the discussions with both sides,” according to a statement from his office.
The UN official's plan is based on the reopening of the main route leading in and out of the south-western city as well as roads in other governorates. This is to ease civilian suffering and enable the delivery of goods to the besieged city.
“I’m pleased to be in Sanaa for the second time after the truce came into effect. I’m also pleased that the truce was extended for an additional two months which is a positive signal of the parties' seriousness to uphold and implement the truce,” Mr Grundberg said in a statement late on Wednesday.
Yemenis have seen the truce’s tangible benefits, he said.
“We have witnessed a significant positive shift and we have a responsibility to safeguard it and deliver on its potential for peace in Yemen,” he added.
The Taez plan is part of a UN-brokered truce that came into effect in early April. It was the first nationwide ceasefire in six years of the Yemen conflict, which is now in its eighth year.
Yemen's warring sides, the rebels and internationally recognised government, agreed last Thursday to extend the truce for two more months after concerted pressure from the UN and international aid groups.
Since the ceasefire came into effect, fighting has decreased, bringing calm to most parts of the country for the first time since 2014 when the Houthi rebels ousted Yemen's government.
This prompted an intervention by a Saudi-led coalition at the government's request in 2015.