Elon Musk's social media network X has sparked widespread backlash after users utilized its AI chatbot Grok to manipulate pictures of females by taking off their garments.
Multiple instances have surfaced where the technology has been employed to virtually strip women to show them in bikinis non-consensually, as well as to place them in sexually explicit scenarios.
Company Response and Regulatory Position
XAI, the firm that created Grok, offered no comment to questions, aside from an auto-reply response stating "legacy media lies".
A Home Office official stated that new laws is being proposed to prohibit image stripping tools. Under a new law, anyone who distributed such tools would "receive a jail term and substantial penalties".
"The platform could prevent these types of abuse if they chose to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Stance and Tool Functionality
The communications watchdog Ofcom said platforms must "evaluate the risk" of people seeing unlawful material on their networks, but did not confirm whether it was presently investigating X or Grok in regarding artificial intelligence images.
Grok is a free chatbot—with some premium features—which reacts to X users' queries when they tag it in a message.
It is frequently used to offer commentary or more background to other users' comments, but people on X are also capable of edit an posted photo through its AI image editing feature.
Background of Criticism and Rules
The tool has been condemned for allowing users to produce pictures and clips with nudity and sexualised material, and it was previously accused of generating graphic material of public figures.
"The platform has been enabling the creation and spread of these images for months without taking any action and we have not witnessed any challenge by regulators," the expert added.
XAI's own usage rules bans "portraying likenesses of people in a pornographic manner".
In a comment, Ofcom said it was illegal to "generate or circulate deepfakes or child sexual abuse material" and stated this included sexual deepfakes made with AI.
It said services such as X were required to take "appropriate measures" to "minimize the risk" of users viewing illegal material on their services and to take down it promptly when they learn of it.
