Elon Musk's online network X has provoked significant anger after people used its AI chatbot Grok to manipulate photos of women by stripping their clothing.
Multiple instances have been seen where the tool has been employed to virtually strip female subjects to make them appear in bikinis non-consensually, as well as to insert them in inappropriate scenarios.
Company Response and Legal Position
XAI, the organization behind Grok, remained silent to questions, other than an automatically-generated response stating "legacy media lies".
A Home Office representative stated that legislation is being brought forward to outlaw nudification tools. Under a upcoming law, anyone who provided such tools would "be subject to a prison sentence and significant penalties".
"The platform could prevent these forms of misuse if they desired to," said a law professor, adding they "seem to have a lack of consequences."
Oversight Body Stance and Platform Functionality
The communications watchdog Ofcom said tech firms must "assess the risk" of people accessing illegal content on their platforms, but stopped short of confirming whether it was at this time looking into X or Grok in concerning artificial intelligence images.
Grok is a free artificial intelligence—with some additional features—which responds to X users' requests when they tag it in a post.
It is frequently used to give commentary or further background to other users' remarks, but people on X are also capable of alter an uploaded picture through its AI image editing function.
Background of Controversy and Rules
The system has been condemned for allowing users to produce photos and videos with explicit content and sexualised material, and it was earlier alleged to have been involved in creating sexually explicit content of celebrities.
"The platform has been permitting the production and sharing of these pictures for months without taking any action and we have not witnessed any challenge by oversight bodies," the expert added.
XAI's own terms of service forbids "portraying likenesses of people in a sexually explicit manner".
In a statement, Ofcom said it was unlawful to "make or distribute deepfakes or child sexual abuse material" and confirmed this covered sexual deepfakes created with AI.
It said networks such as X were required to take "suitable actions" to "minimize the danger" of users viewing illegal material on their platforms and to delete it swiftly when they become aware of it.
