The online network X has sparked major backlash after individuals employed its AI chatbot Grok to change photos of women by taking off their attire.
Multiple cases have been seen where the technology has been employed to digitally undress women to render them in swimwear without consent, as well as to put them in sexualized scenarios.
Developer Response and Regulatory Stance
XAI, the organization responsible for Grok, did not respond to inquiries, except for an auto-reply message stating "legacy media lies".
A government spokesperson stated that new laws is being proposed to ban image stripping technology. Under a proposed law, anyone who distributed such tools would "receive a jail term and significant penalties".
"X or Grok could prevent these forms of misuse if they wanted to," said Clare McGlynn, adding they "seem to have impunity."
Oversight Body Position and Platform Functionality
The regulator Ofcom said tech firms must "consider the potential" of people viewing unlawful material on their networks, but did not confirm whether it was presently examining X or Grok in relation to artificial intelligence pictures.
Grok is a no-cost chatbot—with some additional features—which reacts to X users' queries when they mention it in a message.
It is commonly used to provide commentary or further context to other users' comments, but people on X are also able to alter an posted image through its artificial intelligence picture modification tool.
Background of Criticism and Rules
The tool has been criticised for permitting users to produce pictures and footage with nudity and sexualised content, and it was previously alleged to have been involved in creating inappropriate material of celebrities.
"X has been enabling the production and sharing of these images for months without taking any steps and we have yet to see any challenge by oversight bodies," Professor McGlynn added.
XAI's own terms of service prohibits "showing likenesses of people in a graphic fashion".
In a comment, Ofcom said it was unlawful to "generate or share non-consensual intimate images or illegal content" and clarified this applied to explicit AI images created with AI.
It said platforms such as X were obligated to take "suitable steps" to "reduce the risk" of users coming across unlawful material on their platforms and to remove it promptly when they learn of it.
