
As xAI's Grok chatbot faces potential bans and lawsuits in
regions across the globe for generating non-consensual nude images and other harmful content, the company has decided to restrict Grok's image-generation feature to only paying subscribers on X, the
social-media company formerly known as Twitter.
While the official Grok app allows anyone to generate AI images for free, the Grok account issued its own statement on X this week, informing
users that it would no longer generate and edit images on the platform for non-paying users.
Over the past year, the Grok chatbot has helped X, which is owned by Elon Musk, amass more revenue.
However, the images have caught the attention of governments in India, Europe and Malaysia due to the generation
of sexually explicit AI content based on user prompts and searches.
advertisement
advertisement
Earlier this week, the Grok X account apologized to users for generating and sharing “an AI image of two young girls
(estimated ages 12-16) in sexualized attire based on a user's prompt.”
According to a report by Wired, thousands of non-consensual nude images are being generated by
Grok on X every day.
In response, India's Ministry of Electronics and Information Technology has demanded that X take “corrective action” to address Grok’s “obscene,
pornographic, vulgar, indecent, sexually explicit, pedophilic, or otherwise prohibited under law” content, while Malaysia’s Communications and Multimedia Commission has urged X to ensure
that Grok is aligned with the country’s local laws.
Indonesia's Communications Ministry threatened to ban X and Grok in the country if it does not address the “degrading pictures
of women and children” being generated, whereas French authorities are carrying out an official investigation into sexually explicit deepfakes on the platform and the U.K.'s media regulator
Ofcom has requested that X outline what steps it is taking “to comply with their legal duties to protect users in the U.K.”
Prior to this current controversy over Grok, X was
projected to earn $2.9 billion for 2025, based on estimates from a recent Bloomberg report, marking a 10% increase from 2024.
However, the company’s revenue intake is now in jeopardy. In
the U.K., for example, X has witnessed a 60% revenue
decline due to Grok’s harmful image generation.
X’s holding company, xAI, just announced the completion of a funding round giving it $20 billion, which the company plans to use to
advance its newest model, Grok 5, and build a new data center in Memphis, Tennessee.