The online platform X has caused major outrage after people utilized its artificial intelligence assistant Grok to change photos of women by stripping their attire.
Several examples have emerged where the technology has been employed to remove clothing from female subjects to render them in bikinis non-consensually, as well as to put them in inappropriate situations.
Company Reaction and Legal Stance
XAI, the firm that created Grok, remained silent to requests for comment, except for an standard message stating "traditional news outlets lies".
A government representative stated that new laws is being proposed to outlaw image stripping software. Under a new law, anyone who provided such technology would "be subject to a jail term and heavy penalties".
"The platform is capable of halting these types of abuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Oversight Body Stance and Platform Capabilities
The regulator Ofcom said technology companies must "consider the potential" of people accessing unlawful material on their platforms, but did not confirm whether it was presently investigating X or Grok in regarding artificial intelligence pictures.
Grok is a free chatbot—with some additional functions—which reacts to X users' prompts when they tag it in a tweet.
It is frequently used to give commentary or further context to other posters' remarks, but people on X are also can edit an uploaded photo through its AI image editing feature.
History of Controversy and Rules
The tool has been condemned for allowing users to generate images and videos with explicit content and sexually suggestive material, and it was previously alleged to have been involved in making graphic material of public figures.
"The platform has been permitting the creation and sharing of these images for a long time without taking any action and we have not witnessed any challenge by oversight bodies," Professor McGlynn added.
XAI's own usage rules prohibits "showing likenesses of persons in a sexually explicit fashion".
In a comment, Ofcom said it was against the law to "create or distribute deepfakes or child sexual abuse material" and stated this applied to sexual deepfakes generated with AI.
It said platforms such as X were obligated to take "appropriate steps" to "minimize the danger" of users encountering unlawful content on their services and to remove it promptly when they become aware of it.
