Elon Musk's social media platform X has caused widespread outrage after individuals employed its AI chatbot Grok to manipulate photos of females by taking off their garments.
Numerous cases have been seen where the system has been employed to virtually strip female subjects to render them in bikinis non-consensually, as well as to insert them in sexualized contexts.
Developer Reaction and Regulatory Stance
XAI, the company behind Grok, did not respond to requests for comment, other than an automatically-generated message stating "legacy media lies".
A government official stated that new laws is being proposed to ban image stripping software. Under a proposed criminal offence, anyone who supplied such technology would "face a jail term and significant penalties".
"X or Grok is capable of halting these forms of misuse if they chose to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Regulator Position and Platform Capabilities
The regulator Ofcom said platforms must "consider the danger" of people seeing illegal content on their networks, but did not specify whether it was currently examining X or Grok in regarding AI-generated pictures.
Grok is a free chatbot—with some premium functions—which reacts to X users' prompts when they reference it in a message.
It is often used to provide commentary or more background to other users' comments, but people on X are also can alter an posted image through its artificial intelligence picture modification function.
History of Controversy and Rules
The chatbot has been condemned for allowing users to produce photos and clips with explicit content and sexually suggestive material, and it was in the past alleged to have been involved in creating sexually explicit content of public figures.
"X has been enabling the production and spread of these images for a long time without taking any action and we have not seen any intervention by regulators," the expert added.
XAI's own usage rules prohibits "depicting likenesses of persons in a graphic manner".
In a statement, Ofcom said it was unlawful to "generate or distribute private pictures or child sexual abuse material" and stated this applied to sexual deepfakes generated with AI.
It said networks such as X were required to take "suitable actions" to "minimize the danger" of users encountering illegal material on their services and to take down it promptly when they learn of it.
