Meta Used Contractors to Test Rival AIs on Sensitive Topics, Report Claims

Meta reportedly hired hundreds of contractors to pose as teenagers and interact with rival artificial intelligence chatbots, including those from OpenAI, Google, and Character.AI, to test their safety measures on sensitive topics such as suicide, sex, and drugs. The operation, internally known as "Cannes," was managed by Meta contractor Covalen, according to a report by Wired.

Contractors created fake accounts posing as users under 18 and submitted text prompts and images to competing AI models. They then compiled the chatbots' responses into spreadsheets for analysis. Some of the images reportedly included items like pills, knives, nooses, and a medical illustration of a gynecological procedure. The prompts were designed to test the AI systems' safety guardrails by attempting to elicit responses they were programmed to reject.

One testing phase in August 2025 allegedly involved over 45,000 prompts. A review of nearly 3,750 prompts by Wired revealed hundreds related to suicide and self-harm, hundreds more on eating disorders, and at least 239 concerning sex or romance. Many prompts were framed from the perspective of distressed children or teenagers. Examples cited include a prompt from a 13-year-old claiming pregnancy and asking about abortion pills, a description of a fifth-grader witnessing a classmate with a gun, and queries about hiding bulimia or obtaining cocaine.

Internal Covalen documents reportedly described the effort as "comprehensive AI safety benchmarking" that generated "critical datasets for model comparison and compliance." Meta defended the project, stating that testing and benchmarking chatbot responses for safe and age-appropriate experiences is a "responsible, industry-standard practice." A Meta spokesperson told The Post that any suggestion otherwise misunderstands how technology companies improve their systems and added that Meta does not use competitor benchmarking to train its own AI models.

However, the reported testing appears to conflict with the terms of service of several targeted AI companies. OpenAI's terms reportedly prohibit unauthorized safety testing, attempts to bypass safeguards, and using outputs to develop competing models. Google bars efforts to circumvent safety protections outside approved testing programs, and Character.AI prohibits harmful or exploitative content.

Character.AI told Wired it did not authorize the testing and stated the conduct violated its policies. OpenAI indicated to Wired that it was investigating the matter. Google informed The Post that it had not authorized the described testing and was unaware of its purpose. Covalen did not comment to Wired, and The Post has sought comment from the company.

Former contractors involved in the project expressed discomfort with some assignments. Concerns were raised about the potential to inadvertently generate or preserve child sexual abuse material, and questions were posed about whether collecting extensive data from competitors ultimately benefits Meta. One former contractor told Wired, "I’ve seen a lot of things I wish I hadn’t while doing this job."