Technology Secretary Liz Kendall said Elon Musk’s X must urgently deal with its artificial intelligence Grok being used to create sexualised deepfake images.
She backed regulator Ofcom, which is looking into X and xAI, the firm founded by Mr Musk which created Grok, to take “any enforcement action” deemed necessary.
Users of social media platform X appear to have prompted Grok to generate images of children “in minimal clothing”.
A post on the Grok X account said that there have been “isolated cases where users prompted for and received AI images depicting minors in minimal clothing”, and added: “xAI has safeguards, but improvements are ongoing to block such requests entirely.”
Ms Kendall said: “What we have been seeing online in recent days has been absolutely appalling, and unacceptable in decent society.
“No one should have to go through the ordeal of seeing intimate deepfakes of themselves online.
“We cannot and will not allow the proliferation of these demeaning and degrading images, which are disproportionately aimed at women and girls.
“X needs to deal with this urgently.
“It is absolutely right that Ofcom is looking into this as a matter of urgency and it has my full backing to take any enforcement action it deems necessary.”
Ms Kendall said efforts to curb the spread of fake nude images were not about restricting freedom of expression.
Donald Trump’s US administration has hit out at European regulators and online safety campaigners for attempts to regulate what appears online on American platforms.
But Ms Kendall said: “Services and operators have a clear obligation to act appropriately. This is not about restricting freedom of speech but upholding the law.
“We have made intimate image abuse and cyberflashing priority offences under the Online Safety Act – including where images are AI-generated. This means platforms must prevent such content from appearing online and act swiftly to remove it if it does.
“Violence against women and girls stains our society – and that is why we have also legislated to ban the creation of explicit deepfakes without consent, which are both degrading and harmful.
“Make no mistake – the UK will not tolerate the endless proliferation of disgusting and abusive material online. We must all come together to stamp it out.”
In response to a Press Association request for a comment on Ms Kendall’s statement, an automated response from xAI said: “Legacy media lies.”
Tech tycoon Mr Musk has previously insisted that “anyone using Grok to make illegal content will suffer the same consequences as if they uploaded illegal content”.
X has said it takes action against illegal content, including child sexual abuse material “by removing it, permanently suspending accounts, and working with local governments and law enforcement as necessary”.
