The online network X has provoked major backlash after people used its AI chatbot Grok to manipulate images of females by stripping their clothing.
Multiple instances have emerged where the technology has been employed to digitally undress female subjects to render them in swimwear without consent, as well as to put them in sexualized situations.
Company Response and Legal Position
XAI, the firm responsible for Grok, offered no comment to inquiries, other than an standard message stating "legacy media lies".
A government representative stated that new laws is being introduced to prohibit nudification technology. Under a proposed law, anyone who distributed such technology would "receive a prison sentence and substantial penalties".
"The platform is capable of halting these forms of misuse if they wanted to," said a law professor, adding they "appear to enjoy a lack of consequences."
Regulator Position and Tool Functionality
The communications watchdog Ofcom said technology companies must "consider the risk" of people viewing illegal content on their services, but stopped short of confirming whether it was at this time investigating X or Grok in regarding artificial intelligence images.
Grok is a free AI assistant—with some additional functions—which answers X users' queries when they mention it in a message.
It is frequently used to provide commentary or additional background to other posters' comments, but people on X are also capable of alter an posted image through its artificial intelligence picture modification tool.
History of Controversy and Rules
The system has been condemned for permitting users to generate images and footage with nudity and sexualised material, and it was previously accused of creating inappropriate content of public figures.
"The platform has been allowing the production and distribution of these pictures for a long time without taking any action and we have not seen any challenge by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "portraying likenesses of individuals in a graphic fashion".
In a statement, Ofcom said it was illegal to "create or distribute private pictures or child sexual abuse material" and clarified this covered sexual deepfakes made with AI.
It said services such as X were mandated to take "suitable steps" to "reduce the risk" of users coming across illegal content on their platforms and to remove it quickly when they are notified of it.
