The online platform X has caused significant anger after individuals employed its AI chatbot Grok to manipulate pictures of females by removing their garments.
Numerous examples have surfaced where the technology has been employed to virtually strip women to render them in swimwear non-consensually, as well as to put them in sexualized contexts.
Company Reaction and Regulatory Stance
XAI, the company that created Grok, did not respond to requests for comment, except for an auto-reply response stating "legacy media lies".
A Home Office representative stated that legislation is being proposed to prohibit nudification software. Under a new law, anyone who provided such technology would "receive a prison sentence and significant penalties".
"X or Grok is capable of halting these forms of misuse if they wanted to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Oversight Body Position and Tool Capabilities
The communications watchdog Ofcom said platforms must "assess the potential" of people seeing illegal content on their platforms, but stopped short of confirming whether it was currently investigating X or Grok in concerning AI-generated pictures.
Grok is a free chatbot—with some premium functions—which answers X users' queries when they tag it in a message.
It is commonly used to give reactions or further context to other posters' comments, but people on X are also capable of modify an posted picture through its AI image editing tool.
History of Controversy and Rules
The system has been faulted for permitting users to generate images and footage with nudity and sexually suggestive content, and it was previously accused of making graphic content of celebrities.
"The platform has been permitting the production and distribution of these images for a long time without taking any steps and we have not witnessed any action by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "showing images of individuals in a pornographic manner".
In a comment, Ofcom said it was against the law to "make or circulate private pictures or child sexual abuse material" and clarified this covered explicit AI images made with AI.
It said platforms such as X were required to take "necessary actions" to "lessen the risk" of users viewing unlawful content on their services and to take down it promptly when they become aware of it.
