Grok-2's new AI image generator sets off alarm bells after deepfake free-for-all

The service from Elon Musk's xAI is another headache for regulators due to its unrestricted capability to flood the internet with questionable images

Elon Musk's generative artificial intelligence platform xAI has rolled out a new image generator that can proficiently create anything, flooding the internet with deepfakes and other highly questionable imagery.

And just like that, it seems it will trigger another episode in the seemingly never-ending battle for oversight of the technology of the moment.

The service on the latest Grok chatbot on X, formerly Twitter, will let users create images via a prompt and directly post them on their X accounts.

It is part of the latest beta Grok releases, Grok-2 and Grok-2 mini. They offer “a redesigned interface and new features”, though the latter is a scaled-down version of the standard chatbot.

Users of Grok – only available to subscribers of X's Premium+ tier, which starts at $16 per month – had a field day creating and posting images with the new service.

However, the image generator apparently does not have explicit guidelines to deter users from abusing the new service, nor at least a watermark or note to inform viewers that the image has been made with AI.

Images that have been posted include depictions of Donald Trump and a pregnant US Vice President Kamala Harris as a couple, and former US presidents George W Bush and Barack Obama posing with illegal drugs.

Other generated images include those with guns, giving the illusion of violence, and sexualised themes, as well as Mickey Mouse wearing a “Make America Great Again” cap while holding a cigar and pint of beer, and notable women such as the US Vice President and pop star Taylor Swift in lingerie.

Analysts say the new function could present new regulatory challenges.

“As regulators are already dealing with issues like misinformation and harmful content, [the Grok image generator's] unrestricted responses might further complicate efforts to ensure a safe and accurate online environment,” Sid Bhatia, regional vice president at New York-based AI company Dataiku, told The National.

The situation could accelerate regulatory discussions and lead to the faster development of guidelines and laws to address AI-generated content, while balancing innovation and safety, said Andreas Hassellof, chief executive of Switzerland-based technology consultancy Ombori.

"X's approach to content moderation, fact-checking and algorithmic amplification will be far more critical than the initial image generation capabilities," he added.

Tricky situation

OpenAI's Dall·E 3, for instance, states on its website that it works on preventing “harmful generations” to aid in its “risk assessment and mitigation efforts in areas like propaganda and misinformation”.

The lack of safety measures on Grok-2's image generator raises concerns that it may be used as a tool to spread misinformation. xAI did not address this in its release statement.

Grok-2 would only state safety guidelines if you ask it, with responses such as: “I avoid generating images that are pornographic, excessively violent, hateful, or that promote dangerous activities”, The Verge reported.

“I won’t generate images that could be used to deceive or harm others, like deepfakes intended to mislead, or images that could lead to real-world harm,” another response said.

The prompt of OpenAI's ChatGPT, on the other hand, will reject requests to create potentially harmful or offensive content.

Mr Musk has long championed the right to free speech, and his takeover of X has amplified his stance. Under his leadership X has complied with government content moderation requests, though Mr Musk has faced criticism for suspending accounts critical of him.

One of the most controversial decisions X has made was in June, when it said the platform would allow pornographic posts, as long as “it is consensually produced and … properly labelled and not prominently displayed”.

“What makes Grok-2 truly revolutionary is its unrestricted image-generation capabilities, including the ability to produce NSFW [not safe for work] content, a feature that sets it apart from other AI models which often impose strict ethical guidelines,” Anakin.ai, a San Francisco-based generative AI platform, wrote in a blog post.

Whether or not the lack of safeguards in Grok's new image generator is Mr Musk testing user and regulator patience is unclear.

“His legal disputes with various governments and the perceived inconsistency in his moderation practices contribute to a complex view of his commitment to free speech, suggesting that his approach may be influenced by personal and business considerations,” said Paul Turner, executive director at Capex.com for the Middle East.

The timing of Grok's new feature is critical as social media, coupled with the increasing power of AI, has blurred the lines between what is real and what is not – and the ability of certain users to circumvent rules that are in place, finding loopholes to game the system.

“We anticipate that additional safeguards will be implemented,” Alexander Ivanyuk, technology director at Switzerland-based cyber security firm Acronis, told The National. “Other generative AI systems have undergone similar processes, yet they can still be manipulated by clever and creative individuals to produce content that might be considered offensive.”

Still, it cannot be ruled out that Grok will implement restrictions – though there may be fewer than for rival platforms, he added.

“Ultimately, it's challenging to completely prevent individuals from pushing boundaries and attempting to bypass safeguards to generate provocative content,” Mr Ivanyuk said.

Further misinformation

The release of Grok-2's image generator resurrects the long-running battle over online misinformation – and this is particularly noteworthy in 2024, an election year in the US, where social media has been a weapon of choice for candidates and supporters.

Mr Musk has become increasingly political, frequently voicing his views on X, which he bought for $44 billion in 2022. He “fully endorse[d]” Mr Trump on July 14, a day after the assassination attempt on the presumptive Republican presidential nominee seeking a return to the White House. This week, they sat down for an interview on X.

Already carrying inherent risks, the generator has the potential to further spread harmful content, Mr Bhatia said.

“Grok’s capacity to provide detailed responses to sensitive topics highlights the need for effective risk management, especially given the platform’s broad reach and the rapid spread of information.”

AI image generators, especially those used for deepfakes, also raise significant copyright issues: these tools can potentially use images from other social media platforms without the original creator’s consent, leading to copyright violations.

“Since AI-generated images are not eligible for copyright protection due to their lack of human authorship, the legal aspect could remain challenging,” Mr Turner said.

“To address these concerns, clearer licensing agreements, greater transparency in image use, updates to copyright laws and improved public education about AI and copyright may be necessary.”