A critical safety setting on TikTok, designed to shield children from inappropriate content, is proving ineffective, allowing graphic videos of rape and torture to be viewed by underage users. These disturbing clips are often AI-generated and exploit gaps in the platform's rules, which ban realistic violence but permit animated depictions.
One startup, Director AI, has developed video creation tools that have enabled users to produce over 15,000 such videos, many of which are algorithmically designed to be addictive. Some of these videos have garnered millions of views. For instance, one clip depicts a cartoon doctor performing a graphic facial surgery, while another illustrates a violent revenge scenario involving a burglar and a wealthy woman.
Despite TikTok's community guidelines prohibiting content that promotes or glorifies violence and sexual abuse, these videos have bypassed safeguards. An investigation using a TikTok account set to 'restricted mode' and a 14-year-old age setting revealed a significant volume of content falling into restricted categories, including violent or threatening imagery and sexually suggestive material.
Ben Chiang, the former Snapchat director behind Director AI, has referred to the output as "AI slop" and boasted of billions of views. The company's companion app, Reel AI, which offers some content for free before requiring payment, has an age rating on Google Play suitable for children as young as seven. One purchased episode reportedly shows a mother electrocuting her infant in a bathtub.
Online safety expert Dr. Catherine Knibbs criticized the creators, stating, "This is about creating wealth at the expense of children." Following the Mail on Sunday's inquiry, Reel AI's age rating was updated, and TikTok removed the specific videos cited in the report. However, the accounts that produced them remain active and continue to feature similar inappropriate material.
In response to the findings, Shadow Education Secretary Laura Trott stated that social media companies should not have access to children until their platforms can be proven safe, urging for stronger action from the Labour party to remove children from social media. A spokesperson for the Department for Culture, Media and Sport affirmed the government's support for broadcast regulator Ofcom to ensure tech companies comply with the Online Safety Act, emphasizing their legal duty to remove illegal content and protect children.