OpenAI's Teen ChatGPT Safety Features Fall Short, Research Finds
A new study indicates that crucial safeguards designed to protect young users from harmful content and emotional dependence on ChatGPT are not consistently effective.
New research suggests that OpenAI's safety features for its "ChatGPT for Teens" program, implemented to promote healthier AI use among minors, frequently fail to meet their intended purpose. The study, released by the non-profit Common Sense Media, concluded that the platform poses an "unacceptable risk" to young people and recommended banning users under 18 until the safeguards are proven reliable.
OpenAI introduced several teen-focused safety measures in August, including curbs on emotional dependence and alerts for parents regarding problematic conversations. The company stated on Wednesday that its own observations indicated limited teen engagement with ChatGPT, primarily for learning activities. According to OpenAI, the average teen user spends less than 15 minutes a day on the platform, and automatic break reminders are often effective, with nearly half of teen users stopping sessions after such prompts. Furthermore, OpenAI reported that in over 80% of extended usage sessions (three consecutive hours or more), teen prompts were related to learning.
However, Common Sense Media's investigation, which utilized multiple accounts registered as belonging to minors, found significant shortcomings. While ChatGPT was effective in preventing "sexual roleplay" with young users, other guardrails, including those for discussions about suicide, failed. The research indicated that even after an hour of conversation on topics like suicidal ideation, self-harm, or disordered eating, newly created parent-linked accounts sent "zero" alerts to parents. Alerts were only triggered on older accounts with a history of sensitive conversations. In critical situations, the report also found that ChatGPT "failed to reliably recommend that teens in crisis connect to a hotline or professional."
The research also noted that ChatGPT for Teens continued to assist with schoolwork and engage in anthropomorphized language or conversational tones, despite OpenAI's stated aim to reduce such behaviors. Tom Siegel, head of Common Sense's Youth AI Safety Institute, stated that "ChatGPT for Teens could give parents false confidence in guardrails and safety alerts that frequently don't work."
The interaction of young people with online platforms has become a significant concern amid rising rates of self-harm and suicide among children. Several tech companies, including Meta, TikTok, and others, have faced lawsuits alleging harm to children's mental health.
OpenAI has also faced scrutiny and legal pressure related to teen usage of ChatGPT. In February, an 18-year-old who carried out a mass shooting in Canada had reportedly discussed gun violence with ChatGPT for months prior. OpenAI has since apologized for not flagging the individual's account to law enforcement and is facing multiple lawsuits concerning the incident.