Child abusers are creating AI-generated “deepfakes” of their targets in order to blackmail them into filming their own abuse, beginning a cycle of sextortion that can last for years.

Creating simulated child abuse imagery is illegal in the UK, and Labour and the Conservatives have aligned on the desire to ban all explicit AI-generated images of real people.

But there is little global agreement on how the technology should be policed. Worse, no matter how strongly governments take action, the creation of more images will always be a press of a button away – explicit imagery is built into the foundations of AI image generation.

In December, researchers at Stanford University made a disturbing discovery: buried among the billions of images making up one of the largest training sets for AI image generators was hundreds, maybe thousands, of instances of child sexual abuse material (CSAM).

There may be many more. Laion (Large-scale AI Open Network), the dataset in question, contains about 5bn images. With half a second a picture, you could perhaps look at them all in a lifetime – if you’re young, fit and healthy and manage to do away with sleep. So the researchers had to scan the database automatically, matching questionable images with records kept by law enforcement, and teaching a system to look for similar photos before handing them straight to the authorities for review.

In response, Laion’s creators pulled the dataset from download. They had never actually distributed the images in question, they noted, since the dataset was technically just a long list of URLs to pictures hosted elsewhere on the internet. Indeed, by the time the Stanford researchers ran their study, almost a third of the links were dead; how many of them in turn once contained CSAM is hard to tell.

But the damage has already been done. Systems trained on Laion-5B, the specific dataset in question, are in regular use around the world, with the illicit training data indelibly burned into their neural networks. AI image generators can create explicit content, of adults and children, because they have seen it.

Source: Can AI image generators be policed to prevent explicit deepfakes of children? – Stigmatis