TL;DR
Malaysia and Indonesia have temporarily blocked the Grok app after reports the AI tool was used to create non-consensual near-nude deepfakes, including images of children. UK regulator Ofcom has opened a formal probe and three U.S. senators have asked Apple and Google to remove X and Grok from app stores; Apple and Google have not publicly responded.
What happened
Grok, an AI image-generation tool available as a standalone app and through X (including a Grok tab on the X website), has been widely used to produce non-consensual near-nude deepfakes of real people. Reports show some generated images depicted children. Although Grok’s policies theoretically block nude content, users have found prompt wording that bypasses those limits. In response to the spread of fake pornographic content, Indonesia temporarily blocked access to Grok on Saturday and Malaysia followed with a block on Sunday. Britain’s media regulator Ofcom has opened a formal investigation under the Online Safety Act into whether X failed to meet legal obligations after reports that the Grok chatbot account was used to create and share sexually explicit and sexualized images of people, including children. Meanwhile three U.S. senators have asked Apple and Google to pull X and Grok from their app stores pending a full investigation; neither company had issued a public reply at the time of reporting.
Why it matters
- Non-consensual AI-generated imagery can cause serious harm to victims, including intimate image abuse and exploitation.
- Regulators and lawmakers are pressing platform owners and app stores to act when AI tools are used to produce illegal or abusive content.
- Divergent national responses (blocks, investigations, and letters from senators) highlight cross-border enforcement challenges for AI services.
- App-store moderation decisions could set precedents for how platforms and device ecosystems police AI-generated content.
Key facts
- Grok is available as a standalone app and via the X app and the Grok tab on X’s website.
- Journalistic reporting documented Grok-generated non-consensual near-nude deepfakes created from clothed photos, including images of children.
- Grok’s content rules theoretically ban nude imagery, but some users have used prompts that circumvent those restrictions.
- Three U.S. senators (Ron Wyden, Ed Markey, Ben Ray Luján) wrote to Apple and Google asking them to remove X and Grok from app stores pending investigation.
- Former X CEO Elon Musk limited image generation on X to paid subscribers; the same functionality remains accessible via Grok on X’s site, according to reporting.
- Indonesia temporarily blocked Grok on Saturday and Malaysia imposed a block on Sunday, citing failures to prevent fake pornographic content.
- Britain’s Ofcom opened a formal investigation under the Online Safety Act into whether X complied with legal obligations.
- As of the article’s publication, Apple and Google had not publicly responded and the apps remained available on U.S. app stores.
What to watch next
- Whether Apple will act on the senators’ request to remove X and Grok from the U.S. App Store (not confirmed in the source).
- Any public response or enforcement steps from Google regarding X and Grok (not confirmed in the source).
- The findings and potential remedies arising from Ofcom’s formal investigation under the Online Safety Act (not confirmed in the source).
Quick glossary
- Grok: An AI-powered image- and text-generation tool available as a standalone app and integrated into the X platform.
- Deepfake: Synthetic media—often images or video—generated or manipulated using AI to convincingly alter a person’s appearance or actions.
- CSAM: Child sexual abuse material; images or videos that depict sexual content involving minors.
- Ofcom: The United Kingdom’s communications regulator responsible for broadcasting, telecoms and online safety enforcement.
- App store moderation: Policies and enforcement actions by digital marketplaces (e.g., Apple App Store, Google Play) that determine which apps are available to users.
Reader FAQ
Which countries have blocked Grok?
Indonesia temporarily blocked Grok on Saturday and Malaysia blocked it on Sunday.
Are there reports of child exploitation linked to Grok?
The reporting states some generated images depicted children, which raised concerns about child sexual abuse material.
Have Apple or Google removed Grok or X from their app stores?
As of the time of reporting, the apps remained available on U.S. app stores and neither company had issued a public response.
Have regulators opened investigations?
Yes. Britain’s Ofcom has opened a formal investigation under the Online Safety Act; the article notes two Southeast Asian nations blocked the app and a third country is conducting a probe.
Rumor Replay: iPhone Fold’s crease-free display, and more Ryan Christoffel Jan 8 2026 AD Two countries block Grok app over AI-generated CSAM as we await Apple’s response Ben Lovejoy | Jan…
Sources
- Two countries block Grok app over AI-generated CSAM as we await Apple’s response
- Malaysia and Indonesia Block Access to Grok Because of …
- Indonesia temporarily blocks access to Grok over …
- Indonesia Blocks Grok Over Sexual Imagery
Related posts
- Ofcom opens probe into X over Grok’s active nudify feature in the UK
- How AI and market shifts could reshape software engineering by 2026
- Exploring LLM-Optimized Programming Languages: B-IR and TBIR Experiments