Earlier this week, the California Attorney General’s Office announced that it was investigating xAI over reports that its chatbot Grok was being used to create non-consensual sexual images of women and minors. On Friday, the government sent a cease-and-desist letter to the company, demanding that it immediately stop producing CSAM, which contains non-consensual intimate images and child sexual abuse material.
“Today, I sent a cease-and-desist letter to xAI, demanding that the company immediately cease producing and distributing deepfakes, non-consensual intimate images, and child sexual abuse material,” California AG Rob Bonta said in a press release. “The production of this material is illegal. I fully expect xAI to comply immediately. California has zero tolerance for (CSAM).”
The AG’s office further claimed that xAI appears to be facilitating the “large-scale production” of non-consensual nudes, which are “used to harass women and girls on the Internet.” The agency said it expects xAI to demonstrate that it is taking steps to address these issues within the next five days.
At the center of the backlash is Grok’s “spicy” mode feature, which xAI created to generate explicit content. This problem extends beyond California. Japan, Canada, and the United Kingdom have launched investigations into Grok, while Malaysia and Indonesia have temporarily blocked the platform completely. Even though xAI placed some restrictions on its image editing capabilities late Wednesday, the California AG’s office moved forward with the cease and desist letter.
X’s safety account previously condemned this type of user activity, saying, “Those who use Grok or encourage the creation of illegal content will suffer the same consequences as if they had uploaded illegal content.” When TechCrunch reached out to xAI for comment, it received an automated email that read “legacy media lies.” TC also contacted the California AG’s office for further information.
The advent of free generative AI tools has led to an alarming increase in non-consensual sexual content. Many platforms, not just X, are working on this issue. This dastardly activity has caught the attention of state leaders as well as Congress. In fact, on Thursday, lawmakers sent a letter to executives at several companies, including X, Reddit, Snap, TikTok, Alphabet, and Meta, asking them how they plan to stop the spread of sexual deepfakes.
