Elon Musk's online platform X has caused widespread backlash after individuals used its artificial intelligence assistant Grok to change images of females by stripping their garments.
Multiple cases have been seen where the tool has been used to remove clothing from women to make them appear in bikinis without consent, as well as to insert them in sexualized situations.
Company Reaction and Legal Position
XAI, the company that created Grok, offered no comment to requests for comment, except for an standard response stating "traditional news outlets lies".
A government official stated that new laws is being proposed to ban nudification software. Under a new law, anyone who supplied such tools would "receive a jail term and substantial penalties".
"The platform is capable of halting these types of abuse if they chose to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Regulator Position and Platform Capabilities
The communications watchdog Ofcom said technology companies must "consider the risk" of people accessing illegal content on their platforms, but stopped short of confirming whether it was at this time looking into X or Grok in regarding AI-generated pictures.
Grok is a no-cost chatbot—with some additional functions—which reacts to X users' prompts when they mention it in a message.
It is frequently used to give commentary or more context to other users' remarks, but people on X are also capable of modify an posted image through its artificial intelligence picture modification feature.
History of Criticism and Policy
The chatbot has been criticised for permitting users to generate photos and videos with explicit content and sexualised content, and it was earlier alleged to have been involved in making sexually explicit content of celebrities.
"The platform has been enabling the generation and distribution of these pictures for months without taking any steps and we have yet to see any intervention by oversight bodies," the expert added.
XAI's own usage rules bans "showing images of individuals in a pornographic fashion".
In a comment, Ofcom said it was unlawful to "generate or distribute private pictures or child sexual abuse material" and stated this applied to sexual deepfakes generated with AI.
It said networks such as X were mandated to take "suitable measures" to "lessen the risk" of users coming across illegal material on their services and to remove it swiftly when they are notified of it.
