Elon Musk's social media platform X has caused significant backlash after individuals used its AI chatbot Grok to change photos of females by removing their attire.
Numerous instances have surfaced where the tool has been employed to virtually strip female subjects to render them in swimwear without consent, as well as to put them in inappropriate situations.
Developer Response and Legal Stance
XAI, the firm responsible for Grok, remained silent to questions, other than an auto-reply response stating "legacy media lies".
A Home Office representative stated that new laws is being brought forward to ban nudification tools. Under a proposed law, anyone who distributed such technology would "receive a prison sentence and substantial fines".
"The platform is capable of halting these forms of misuse if they desired to," said a law professor, adding they "seem to have a lack of consequences."
Regulator Position and Platform Functionality
The regulator Ofcom said technology companies must "evaluate the danger" of people viewing unlawful material on their platforms, but did not confirm whether it was currently looking into X or Grok in concerning AI-generated pictures.
Grok is a free chatbot—with some paid-for features—which answers X users' requests when they reference it in a post.
It is frequently used to offer commentary or further background to other users' remarks, but people on X are also capable of modify an uploaded photo through its artificial intelligence picture modification feature.
History of Controversy and Policy
The system has been criticised for allowing users to generate images and videos with nudity and sexualised content, and it was in the past alleged to have been involved in creating sexually explicit content of public figures.
"The platform has been permitting the creation and distribution of these pictures for a long time without taking any measures and we have yet to see any intervention by regulators," Professor McGlynn added.
XAI's own terms of service bans "depicting images of people in a sexually explicit fashion".
In a comment, Ofcom said it was unlawful to "generate or share non-consensual intimate images or illegal content" and stated this included sexual deepfakes created with AI.
It said networks such as X were mandated to take "necessary measures" to "reduce the danger" of users coming across unlawful content on their services and to delete it quickly when they become aware of it.
