California’s Attorney General is investigating Grok’s sexualized deepfakes

• Elon Musk’s xAI is now the subject of an investigation by the California Attorney General.
• Attorney General Rob Bonta is investigating reports that Grok is generating nonconsensual sexualized images of real people.
• Regulators around the world have pressed xAI over the instances.

Elon Musk’s xAI is now facing pressure from US regulators over Grok’s generation of nonconsensual sexualized images of real people.

On Wednesday, California Attorney General Rob Bonta announced that his office was investigating the numerous reports of such images.

“The avalanche of reports detailing the non-consensual, sexually explicit material that xAI has produced and posted online in recent weeks is shocking,” Bonta said in a statement. “This material, which depicts women and children in nude and sexually explicit situations, has been used to harass people across the internet. I urge xAI to take immediate action to ensure this goes no further.”

The news comes after officials in India, the UK, Indonesia, and Malaysia have taken their own varying actions against Grok. Indonesia and Malaysia went so far as to block access to Grok.

Ofcom, the UK’s communications regulator, announced its investigation earlier this week. Prime Minister Keir Starmer previously told lawmakers that X could “lose the right to self-regulate.”

In response to Business Insider’s request for comment about Bonta’s probe, xAI replied, “Legacy Media Lies.” Musk’s AI company frequently responds to media requests with that line.

Amid the backlash, xAI limited access to Grok’s image generation to paying subscribers.

Earlier Wednesday, Musk said he was unaware of “any naked underage images generated by Grok.” Musk had previously said anyone asking Grok to make illegal content “will suffer the same consequences as if they upload illegal content.”

“Obviously, Grok does not spontaneously generate images, it does so only according to user requests,” Musk wrote on X. “When asked to generate images, it will refuse to produce anything illegal, as the operating principle for Grok is to obey the laws of any given country or state.”

Musk’s statement belies many of the instances that regulators were looking into. Namely, the investigations were sparked by concerns about users asking Grok to sexualize images, such as depicting a person in a bikini when the original photo did not show them in such a state of dress.

The US Senate unanimously passed a bill via voice vote on Tuesday that would give victims a federal civil right to sue users who ask AI to create such images. The text of legislation, called “The Defiance Act,” mentions the digital manipulation of a photo to “remove the clothing of the individual so that the person appears to be nude.”

Sen. Richard Durbin, an Illinois Democrat, who wrote the bill, said it was aimed at stopping instances like the Grok images.

“Recent reports showed that X, formerly Twitter, can ask its AI chatbot Grok to undress women and underage girls in photos,” Durbin said on the Senate floor. “Grok will comply to show various states of undress with images I won’t repeat for the record, but they’re horrible.”

It is unclear if the bill will receive a vote in the House.

Last year, President Donald Trump signed bipartisan legislation into law that requires social media platforms to remove non-consensual photos, as well as AI deepfakes, within 48 hours of receiving a request to do so.