The social media platform X has provoked widespread backlash after people used its artificial intelligence assistant Grok to alter images of women by stripping their garments.
Numerous cases have emerged where the system has been used to digitally undress female subjects to render them in bikinis non-consensually, as well as to insert them in sexualized scenarios.
Company Response and Legal Stance
XAI, the company that created Grok, offered no comment to inquiries, other than an standard response stating "legacy media lies".
A government official stated that new laws is being proposed to prohibit nudification technology. Under a upcoming criminal offence, anyone who distributed such tools would "be subject to a jail term and substantial fines".
"The platform has the ability to stop these types of misuse if they chose to," said a law professor, adding they "seem to have impunity."
Oversight Body Position and Platform Capabilities
The regulator Ofcom said tech firms must "consider the risk" of people seeing unlawful material on their networks, but did not confirm whether it was at this time looking into X or Grok in concerning AI-generated pictures.
Grok is a no-cost chatbot—with some paid-for functions—which responds to X users' prompts when they reference it in a tweet.
It is often used to give reactions or additional context to other posters' remarks, but people on X are also capable of edit an posted photo through its artificial intelligence picture modification tool.
Background of Criticism and Policy
The chatbot has been faulted for allowing users to create images and clips with nudity and sexually suggestive material, and it was in the past alleged to have been involved in creating sexually explicit content of public figures.
"The platform has been enabling the creation and sharing of these images for a long time without taking any measures and we have not seen any challenge by oversight bodies," Professor McGlynn added.
XAI's own terms of service prohibits "portraying images of persons in a graphic fashion".
In a statement, Ofcom said it was against the law to "make or share private pictures or child sexual abuse material" and stated this applied to sexual deepfakes generated with AI.
It said services such as X were mandated to take "suitable steps" to "reduce the risk" of users encountering illegal content on their services and to take down it swiftly when they learn of it.
