Visiting Athens, Mr Scholz said Nato powers should not question each other's sovereignty, amid a long-running dispute over Aegean Islands and waters.
Greek Prime Minister Kyriakos Mitsotakis said he was ready to offer a “hand of friendship” to de-escalate tensions with Turkey.
Turkish President Recep Tayyip Erdogan earlier this month said he had nothing to discuss with Athens. His remark that “we may suddenly come one night” was interpreted as a threat to Greece.
Mr Scholz told Greek newspaper Ta Nea: “It is not acceptable for one Nato partner to challenge the sovereignty of another. This also applies to more or less veiled military threats.”
Speaking later at a press conference, he said his impression was that Greece was ready to pursue diplomacy with Turkey.
“Good neighbourly relations between Greece and Turkey are not only significant for both countries, but for the whole of Europe and the transatlantic alliance,” Mr Scholz said.
“I am sure that all the questions that exist can, and must, be solved via dialogue on the basis of international law.”
The two countries have argued over drilling rights, migration, US military aid and the status of Cyprus, among other issues.
Mr Mitsotakis said he hoped “our neighbours will choose the road of de-escalation, of legality, of peaceful co-existence without rhetorical outbursts but with constructive actions”.
“On my side, they will always find me ready to extend a hand of friendship. We don’t have any room for further needless sources of tension,” he said.
Mr Scholz toured the Acropolis, one of Greece's most famous landmarks, before his talks with the Greek prime minister.
The two leaders touched on the sensitive subject of reparations for Nazi Germany's invasion of Greece in 1941.
Mr Mitsotakis said the question remains open but would not affect the ability of Germany and Greece to deepen economic ties.
Germany, while accepting its historical guilt, considers the question of reparations closed after agreements in 1960 and 1990.