Footage posted on social media showed thick, black smoke rising above the station, a main commuter route close to riverside tourist attractions.
It forced the closure of London Bridge and Southwark stations, rail and Underground services were disrupted and buildings evacuated as “bursts of black smoke” drifted across the sky.
London Fire Brigade sent about 10 engines and 70 firefighters to the blaze.
“The blaze is producing heavy smoke and those living or working in the local area are advised to keep their windows and doors closed,” London Fire Brigade station commander Wayne Johnson said.
“Trains to and from London Bridge are affected and several buildings have been evacuated.”
Student Lauren Chopin, who lives in a flat two minutes' walk from London Bridge station, said she had to wear a mask indoors due to the fumes.
“There is a lot of smoke all through Union Street and neighbouring streets, there are many fire emergency services trying to put out the fire,” she said.
“It sometimes slows down and the smoke becomes whiter, only to be followed by another huge burst of black smoke. We have closed all doors and windows to limit the amount of smoke coming in.”
British Transport Police said Southwark station was closed and officers had cordoned off surrounding streets.
“Due to a fire next to the track between Waterloo East and London Bridge, services across all routes through London Bridge are severely disrupted,” train operator Southeastern wrote on Twitter.
“Trains are being diverted to other London terminals or terminated at alternative locations.”
Hanushe Lala, from Streatham in south London, was evacuated from her workplace. "Everything has been cornered off and there's thick, black smoke pouring out of the archway. All the buildings around us have been evacuated."
Network Rail wrote on Twitter: “London Fire Brigade are on site with multiple pumps and we have closed the railway until we can confirm it’s safe.
“We’re working with London Fire Brigade and we will need to inspect the railway once the fire is out before we can reopen. Thank you for bearing with us.”
Trains on the Underground network's Jubilee Line were also affected.