Elon Musk's online platform X has caused major backlash after people used its artificial intelligence assistant Grok to manipulate photos of females by removing their clothing.
Numerous instances have been seen where the system has been employed to digitally undress female subjects to make them appear in swimwear without permission, as well as to place them in sexually explicit contexts.
Developer Response and Legal Stance
XAI, the firm responsible for Grok, remained silent to questions, other than an standard response stating "legacy media lies".
A Home Office representative stated that new laws is being proposed to prohibit image stripping software. Under a proposed law, anyone who provided such technology would "face a jail term and significant fines".
"X or Grok could prevent these forms of abuse if they chose to," said a law professor, adding they "appear to enjoy a lack of consequences."
Regulator Position and Tool Capabilities
The regulator Ofcom said technology companies must "consider the potential" of people accessing unlawful material on their platforms, but did not specify whether it was at this time investigating X or Grok in concerning artificial intelligence pictures.
Grok is a free chatbot—with some additional features—which reacts to X users' queries when they mention it in a message.
It is frequently used to offer reactions or further context to other users' comments, but people on X are also able to alter an posted photo through its AI image editing feature.
Background of Controversy and Policy
The tool has been condemned for allowing users to create images and videos with explicit content and sexually suggestive material, and it was in the past accused of creating sexually explicit material of celebrities.
"X has been enabling the production and distribution of these images for months without taking any measures and we have not seen any intervention by regulators," Professor McGlynn added.
XAI's own terms of service prohibits "depicting images of persons in a sexually explicit way".
In a comment, Ofcom said it was unlawful to "create or circulate private pictures or child sexual abuse material" and stated this applied to sexual deepfakes made with AI.
It said platforms such as X were mandated to take "suitable actions" to "reduce the risk" of users coming across illegal content on their services and to remove it promptly when they become aware of it.
