Former OpenAI executive says safety has taken a backseat as company disbands risk unit

Jan Leike alleges disagreements have led to a breaking point and calls for 'cultural change' at the tech leader

OpenAI's move to dissolve its AI risks division has renewed allegations of rifts over safety within the company. AP

A former executive of OpenAI has issued a warning over the company's direction, saying safety has "taken a backseat" after the industry leader disbanded its unit responsible for mitigating artificial intelligence risks.

Jan Leike, the former co-leader of the ChatGPT maker's "superalignment" division, on Friday provided a glimpse of a dysfunctional episode at the California-based company, centred on apparent disagreements on AI safety.

"Building smarter-than-human machines is an inherently dangerous endeavour. OpenAI is shouldering an enormous responsibility on behalf of all of humanity," Mr Leike said in a series of posts on X, formerly known as Twitter.

"But over the past years, safety culture and processes have taken a backseat to shiny products."

OpenAI confirmed on Friday that it had indeed dismantled the superalignment unit, which has been in existence for only less than a year, although the reasons behind the decision were unclear. Company co-founder Ilya Sutskever also left OpenAI this week.

Mr Leike, however, said that the team had been "sailing against the wind" and that it was "getting harder and harder" to get crucial research done.

"I joined because I thought OpenAI would be the best place in the world to do this research. However, I have been disagreeing with OpenAI leadership about the company's core priorities for quite some time, until we finally reached a breaking point," he said.

"I believe much more of our bandwidth should be spent getting ready for the next generations of models, on security, monitoring, preparedness, safety, adversarial robustness, (super)alignment, confidentiality, societal impact and related topics."

"These problems are quite hard to get right, and I am concerned we aren't on a trajectory to get there."

The accusations come as OpenAI and the wider artificial generative intelligence sector attract more regulatory scrutiny, as the technology becomes more powerful, leading to fears on its risks and dangers.

"We are long overdue in getting incredibly serious about the implications of AGI. We must prioritise preparing for them as best we can. Only then can we ensure AGI benefits all of humanity," Mr Leike said.

"OpenAI must become a safety-first AGI company. We urgently need to figure out how to steer and control AI systems much smarter than us."

OpenAI's decision and the details provided by Mr Leike are the latest in a string of incidents that have beset the company, which is leading the AI revolution.

Chief executive Sam Altman was ousted by OpenAI's board on November 17, accused of not being consistently candid in his communications, only to be reinstated a few days later after a dramatic chain of events that included two interim chief executives.

In-between, Microsoft chief executive Satya Nadella had said that Mr Altman would be joining the company to lead a new AI development unit at the Windows operating system maker.

Then, on February 29, billionaire Elon Musk, who helped to create OpenAI but later withdrew his interest, sued the company and Mr Altman, alleging the "stark betrayal" of its founding principles of being a non-profit organisation that would ensure AI would be for the greater good of the world.

OpenAI immediately fired back, saying Mr Musk, the Tesla chief executive regrets no longer being part of the company.

Mr Musk subsequently stepped up efforts at his own generative AI company, xAI, announcing that it will make its Grok chatbot open source, while calling OpenAI "a lie".

Mr Leike's parting shot to OpenAI employees was to "act with the gravitas appropriate for what you're building" and "'ship' the cultural change that's needed".

OpenAI this week upped the ante in the highly competitive generative AI world by introducing GPT-4o, a "new flagship model that can reason across audio, vision and text in real time" that it hopes will attract more users into its platform and fend off all challengers.

GPT-4o is an updated version of the underlying large language model technology that powers ChatGPT. It is the company's fastest model and free for all users, with paid users enjoying "up to five times the capacity limits" of their free peers.