Meta is introducing a new AI safety feature that alerts parents if their teenagers discuss self-harm or suicide with Meta's AI chatbot. The notifications will be sent through Instagram's supervision tools, aiming to provide an early warning for guardians.
If a teenager mentions suicide or self-harm in a conversation with Meta AI, the system is designed to direct them to a crisis helpline and encourage them to contact a trusted adult. Simultaneously, Meta will send an alert to parents if the chat suggests the child is at risk.
This development comes after collaboration with over 75 clinicians specializing in teen mental health. These experts helped Meta identify signals that warrant an alert, distinguishing between clear and subtle references to self-harm. A dedicated AI system was developed to detect these sensitive conversations. Before an alert is sent, flagged chats undergo a manual review. Meta stated that in cases of ambiguous intent, the company will err on the side of caution and notify the parent, prioritizing a potential false alarm over missing a genuine risk.
"We understand how distressing these alerts may be for a parent to receive," Meta said in a statement. "That’s why, as we continue to improve our detection, all chats flagged by our AI will be manually reviewed before an alert is sent."
This AI-driven alert system is now live for parents using Instagram parental supervision in the United States, the United Kingdom, Australia, and Canada, with a global rollout planned by the end of the year. Instagram already has a feature that alerts supervising parents if a teen searches for suicide or self-harm content.
Parents enrolled in Meta's supervision program will receive alerts across multiple apps if their child's chat is flagged. The company is also developing a plan to contact emergency services if a conversation indicates a user is at imminent risk of suicide. Meta currently monitors Facebook and Instagram for posts suggesting a credible risk of suicide and makes referrals to emergency services, having already made over 19,000 global referrals.
Feedback from the AI Wellbeing Expert Council, the Suicide and Self-Harm Advisory Group, and Youth Advisors was incorporated to ensure Meta AI acknowledges the teen's feelings without abruptly ending conversations. Parents will also receive expert resources to help them discuss sensitive topics like suicide and self-harm with their teenagers.
Meta is also implementing conversation tools for Teen Accounts, which are set to a 13+ content setting by default. These tools aim to provide age-appropriate responses and avoid engaging with sexual or romantic conversations or providing information related to alcohol. Additionally, Instagram's Limited Content feature, released in October, will now apply to AI experiences, prompting Meta AI to decline responding to certain topics to minimize inappropriate conversations.
These measures follow the UK government's announcement of a ban on social media use for individuals under 16, scheduled to take effect in spring 2027. The ban will encompass platforms such as Snapchat, TikTok, YouTube, Instagram, Facebook, and X. The UK is also introducing an opt-out social media curfew to disable features between midnight and 6 a.m.