AI-Generated Child Sexual Abuse Images Are Rampant, Could Flood the Internet, UK Watchdog Warns

UK watchdog Internet Watch Foundation (IWF) warned on Tuesday that AI-generated child sexual abuse materials (CSAM) are rampant and could flood the internet.

"What we're starting to see is this explosion of content," said Dan Sexton, the watchdog group's chief technology officer.

In a written report, the IWF warned that the alarming proliferation of child sexual abuse materials on the internet could worsen if nothing is done to put controls on AI tools that generate deepfake photos.

The group then urged governments and technology providers to act quickly before these images overwhelm law enforcement investigators and significantly increase the number of potential victims, the Associated Press reported.

IWF analysts saw abusers exchanging advice and gushing about how simple it was to convert their personal computers into production centers for producing sexually graphic images of kids of all ages.

The analysts examined 20,254 AI-generated images uploaded on a dark web CSAM forum in a one-month period. Of these images, 11,108 were determined most likely to be criminal, while the remaining 9,146 did not contain children or contained kids but were non-criminal in nature.

The IWF further noted that 2,562 images were assessed as criminal pseudo-photographs, and 416 were assessed as illegal prohibited images. The report also indicated that most AI-generated CSAM found are realistic enough to be treated as "real" CSAM.

Even for skilled IWF analysts, the most convincing AI CSAM is visually indistinguishable from actual CSAM. The IWF said its analysts and law enforcement personnel will face further hurdles as text-to-image technology improves.

IWF also found that AI-generated CSAM has increased the potential for the re-victimization of known child sexual abuse victims, as well as the victimization of celebrity kids and children whom the offenders know. The IWF has discovered many examples of AI-generated images featuring known victims and famous children.

"This is happening right now and it needs to be addressed right now," Sexton said.

Urgent Need to Stop AI-Generated Child Sexual Abuse Materials

IWF recommended several actions to the government, law enforcement, regulators, and tech companies. The UK-based watchdog looks forward to the government's forthcoming AI summit, making it an avenue to explore and converse for a global agreement on how this will be handled in various legal systems and secure commitment to continuous collaboration from international governments and stakeholders.

IWF also called on law enforcement agencies to update the College of Policing training course to include the proper way to process this type of image.

The group also asked regulators to "ensure there is proper regulatory oversight of AI models" before it is made open to the market or have in-built protections for closed AI models.

IWF also recommended that tech companies using generative AI and large language models (LLMs) should prohibit the use of such technologies to generate child sexual abuse materials. Any AI models "fine-tuned" to generate child sexual abuse material must also have "de-indexed" links.

Global Cases of AI-Generated Child Sexual Abuse Materials

AI-generated CSAM has already seen legal action. In a first-of-its-kind case, a South Korean man was sentenced last month to 2 1/2 years in prison for using AI to create 360 virtual child abuse images, the country's Busan District Court said.

Another alarming case is how children have been known to use these tools against one another. Police are looking into allegations that teenagers at a school in southwest Spain used a phone software to make their fully clothed classmates appear nude in pictures.