The letter placed high importance on democratic methods of civilian control over the military.
It provided a 16-point breakdown of military "core principles and best practices", which signatories said should be emphasised in “exceptionally challenging times".
It was published just days after US President Joe Biden delivered a prime-time speech condemning Mr Trump and his supporters' threats to democracy.
With uniformed Marines standing in the background, Mr Biden accused Mr Trump's followers of promoting political violence.
“Many of the factors that shape civil-military relations have undergone extreme strain in recent years,” the letter said, pointing to the winding down of Washington's post-9/11 wars, the Covid-19 pandemic and the extreme polarisation in American politics.
Signatories included James Mattis and Mark Esper, both of whom served as defence secretaries under former president Donald Trump.
Mr Mattis and Mr Esper have become outspoken critics of Mr Trump since they left his administration in 2019 and 2020, denouncing the former president as a threat to US democracy.
The former defence officials specifically highlighted the threat of the deadly pro-Trump January 6 insurrection at the US Capitol.
“Politically, military professionals confront an extremely adverse environment characterised by the divisiveness of effective polarisation that culminated in the first election in over a century, when the peaceful transfer of political power was disrupted and in doubt,” the open letter said.
“Looking ahead, all of these factors could well get worse before they get better.”
The former chiefs said “the voters (not the military) decide who will be commander-in-chief. They must prepare for whomever the voters pick, whether a re-elected incumbent or someone new".
After Mr Trump failed to call on his supporters to leave the Capitol on January 6, it was then-vice president Mike Pence who requested that the National Guard be activated, former joint chief of staff Mark Milley testified before the January 6 committee.
The leaders said the military was obliged to assist leaders in their directives “provided that the directives are legal”.
Mr Esper and other members of Mr Trump's Cabinet walked with him to St John's Episcopal Church in Washington for a now-notorious photo opportunity amid racial justice marches in Washington.
Officials had used tear gas on peaceful protesters outside the White House to clear the route for the officials.
In his book, A Sacred Oath, released this year, Mr Esper alleged that Mr Trump considered shooting the protesters.
He was sacked by Mr Trump months after voicing opposition to sending active-duty troops against citizens in the US.