Microsoft has committed to implementing new privacy standards and guardrails for its artificial intelligence tools used by students, following negotiations with the American Federation of Teachers (AFT). The agreement aims to address growing concerns among educators, parents, and students about how their data is collected and used by AI chatbots and other educational technologies.
Millions of students utilize AI for various aspects of their schoolwork, from research and writing assistance to seeking emotional support and personal advice. However, many are unaware of how their conversations and personal information are handled by these platforms. This lack of transparency has fueled worries about data privacy and the role of schools in safeguarding student information as AI becomes increasingly integrated into the daily educational landscape.
In response to these concerns, some of the nation's largest school districts, including New York City and Los Angeles, have placed a one-year moratorium on student AI use as they evaluate how to best incorporate the technology. Microsoft's agreement with the AFT, the country's second-largest teachers union, is seen as a significant step in establishing industry-wide benchmarks for AI in education.
Under the legally binding agreement, Microsoft has pledged not to use student or educator data for training AI systems, with a narrow exception for data used to protect students. The company also states that collected data will not be sold, used for advertising, or for product development. The agreement further prohibits AI features designed to foster emotional attachment or dependency, and mandates transparency measures requiring Microsoft to explain its tools to families in clear language.
"This standard sets a high bar for child privacy and AI safety, and we’ll extend this agreement to every school district across the country," said Brad Smith, Microsoft’s vice chair and president. The new standards are set to apply to all schools with Microsoft contracts beginning November 1.
Randi Weingarten, president of the AFT, described the agreement as the "first of its kind" and a crucial move toward setting an industry standard in the absence of federal and state legislation governing AI in schools. She emphasized the union's goal to ensure parents and educators retain control over children's education, rather than technology companies.
Two other major AI companies, OpenAI and Anthropic, have indicated they are in discussions with the AFT regarding similar safety and privacy pacts. However, Google, a dominant provider of educational technology in the U.S., has not yet announced whether it plans to offer comparable protections.
Josh Golin, executive director of the online safety nonprofit Fairplay, noted that the effectiveness of these standards depends on broader adoption by all AI companies, particularly tech giants. He also raised concerns that the framework might lead schools to believe that AI tools are inherently safe and appropriate for classrooms simply because they meet certain privacy criteria.
Google's recent move to enable its Gemini chatbot for many K-12 students using Google Classroom, its learning management system, has drawn criticism. Google Classroom is used by an estimated 150 million teachers and students globally. Critics view this as an acceleration of the competition among tech companies to gain a foothold in the education sector.
Experts in data privacy, such as Elizabeth Laird, director at the Center for Democracy and Technology, suggest that the new standards could influence how schools negotiate with AI providers, regardless of whether they are Microsoft customers. Laird highlighted that the agreement goes beyond legal requirements by committing to not use sensitive data like gender, race, or age for training AI models.
Laird also pointed out that without broader legislative action, the burden of protecting student privacy continues to fall heavily on tech companies, rather than parents, students, or schools. Microsoft's Smith acknowledged that the tech industry has been slow to fully accept this responsibility.