A report from Anthropic released on Thursday details several incidents in which people used the company's AI models "in ways that could support biological weapons development", among other things.
The company's decision to release the report comes after several high-profile incidents related to AI models led to concern about whether they were becoming too powerful and a risk to public safety.
The report "covers how people tried to misuse Claude – for cyber attacks, influence operations, surveillance, biology, and building weapons – and how we found and stopped them".
Anthropic said it was able to disrupt every operation mentioned, and that it had made sure to keep track of lessons learnt with hopes of bolstering the company's protection.
One of the examples Anthropic highlighted involved a prompt entered into the company's Claude tool that sought assistance with a grant application to research the chikungunya virus.
"Chikungunya virus is a mosquito-borne virus that causes debilitating symptoms (such as severe pain and fever) that can last for weeks or months, and has no licensed therapeutic," the report read.
"One of the reasons we were inclined to think this research was less innocuous was that the institutional affiliation associated with the grant was also a cause of concern."
The report did not elaborate on the institutional affiliation.
The AI company said it has several internal checks for dealing with potentially problematic prompts, and that it actively seeks to improve those workflows.
"We are banning accounts we detect associated with this cluster of activity and are continuing to incorporate our investigative findings into new measures to detect and prevent the actors from misusing Anthropic’s services," the company wrote.
Also in its analysis, Anthropic indicated the difficulty in trying to use nuance when establishing protection.
"We want our models to be useful for scientific research. Indeed, we predict that AI will transform biological science and lead to the rapid development of many new medical treatments," it said.
With AI models escaping test environments and high-level AI researchers resigning with warnings that AI could cause mass casualty events, public perception of the companies at the forefront of the sector has become increasingly negative.
As a result, several legislators in the US Congress have introduced bills that would halt or put limits on AI research, although none of those proposals have become law.
Meanwhile, the companies at the centre of AI development are balancing attempts to project transparency and caution amid accusations of generating panic to display how powerful their models are.
"Anthropic says more stuff that cannot be verified, that the mass media uncritically regurgitates without asking any meaningful questions," one user posted on X in response to Anthropic's report.
On the other side of that debate is US senator Bernie Sanders, who has been outspoken on AI and holding Big Tech firms more accountable.
"AI threatens our economy, privacy, democracy and the well-being of our kids," Mr Sanders posted on social media recently. "Congress must stand up to the Big Tech oligarchs and protect the American people."
