Microsoft agrees to stronger AI privacy rules for students
As AI becomes increasingly common in classrooms, educators, parents and students are raising concerns about data privacy and the role schools should play in protecting children’s information.
Highlights:
-
Millions of students use AI chatbots for schoolwork and advice.
-
Schools are increasingly concerned about student data privacy.
-
Microsoft agreed to stronger privacy and safety standards.
-
Student data cannot be used for AI training.
-
AI companions designed to create dependency are banned.
-
Experts urge technology companies to bear privacy responsibilities.
Keep updated, follow The Business Standard's Google news channel
Millions of students use AI chatbots every day for schoolwork, emotional support and personal advice, often without knowing how their conversations and other personal information may be stored or used.
As AI becomes increasingly common in classrooms, educators, parents and students are raising concerns about data privacy and the role schools should play in protecting children's information.
The issue has gained attention as the two largest US school districts, New York City and Los Angeles, announced plans to limit student use of AI for a year while they examine how the technology should be used in education.
Against this backdrop, Microsoft last week agreed to adopt stronger privacy and safety standards for AI used in schools following negotiations with the American Federation of Teachers (AFT), the country's second-largest teachers union.
The agreement has important safety measures, but they will have a wider impact only if other major AI companies adopt similar standards, said Josh Golin, executive director of online safety nonprofit Fairplay.
"I worry that a high-profile framework like this will be misunderstood and schools will think, 'If they're doing a good job on data privacy and safety, then that means we should be using the tool,'" Golin said.
The AFT said the legally binding agreement, which includes independent audits, could become an industry standard. OpenAI and Anthropic said they are also discussing similar agreements with the union. Google, however, has not said whether it will adopt the same protections.
Limits on use of student data
Under the agreement, Microsoft will not use student or educator data to train its AI systems, except for a limited purpose related to protecting students.
The company also says collected data cannot be sold, used for advertising or used to develop products.
The agreement bans AI companions and other features designed to create emotional attachment or dependency or keep students engaged beyond what is needed for learning.
It also requires audits and greater transparency, including explaining to families in clear language how Microsoft's AI tools work.
"This standard sets a high bar for child privacy and AI safety, and we'll extend this agreement to every school district across the country," Microsoft Vice Chair and President Brad Smith said.
The standards will apply from Nov. 1 to all schools that have contracts with Microsoft, he said.
AFT President Randi Weingarten described the agreement as the first of its kind and an important step toward creating industry-wide standards in the absence of federal or state laws governing AI use in schools.
"We want parents to have control of their kids' education. We want educators to have control of kids' education, not ed tech," Weingarten said.
She made the comments at a joint news conference with Smith at the headquarters of the United Federation of Teachers in New York City. The building houses the AFT's National Academy for AI Instruction for teachers, which launched a year ago with $23 million in support from Microsoft, OpenAI and Anthropic.
New York City and Los Angeles are also planning extensive reviews of their education technology contracts and AI tools, focusing on privacy, transparency and accountability.
New York City's one-year AI restriction applies to elementary and middle school students, while Los Angeles' policy covers students through high school.
Union seeks similar commitments from Google and others
Weingarten said she contacted Google before the Microsoft agreement was announced and invited the company to become a signatory.
Google has not responded to an AP request for comment on whether it plans to adopt similar protections.
"Is Google going to embrace this framework? That is a huge question," Golin said, noting that schools also use thousands of AI-powered educational technology products made by smaller companies.
Google faced criticism in August after announcing that it had enabled its Gemini chatbot for many K-12 students using Google Classroom, its learning management platform.
Google says Classroom is used by about 150 million teachers and students worldwide.
Critics said the move intensified competition among technology companies to expand their presence in schools and attract the next generation of AI users.
The Gemini feature is available only to schools that allow students to use the Gemini app or Gemini Notebook. Until August, Google had allowed Gemini access only to students aged 18 or older.
Weingarten said the union is also negotiating with OpenAI and Anthropic, both of which have welcomed the Microsoft agreement.
OpenAI, the creator of ChatGPT, called the deal an "important milestone for AI safety and privacy in schools" and said it hoped to finalize its own agreement with the AFT.
Anthropic, which makes Claude, said it welcomed efforts to promote responsible AI use in education and was working with the AFT toward what it called a "gold standard for safety and privacy."
Experts say companies should carry the privacy burden
Data privacy experts said the new standards could influence how schools negotiate with AI companies, even when they do not use Microsoft products.
Elizabeth Laird, director at the Center for Democracy and Technology, said the agreement could reduce the pressure on individual school districts to negotiate privacy protections with large technology companies.
Previously, schools' ability to secure stronger protections often depended on their financial resources and bargaining power.
Laird said the agreement could help reduce such differences between school districts. She also praised Microsoft for going beyond existing legal requirements for companies providing technology to schools.
For example, Microsoft has agreed not to use sensitive information such as gender, race, age and ZIP codes, which can sometimes be used to identify individuals even after their names have been removed.
However, Laird said the agreement also shows how difficult it is to establish consistent AI standards without legislation.
She said responsibility for protecting students' privacy should rest with technology companies rather than parents, students or individual schools.
Microsoft's Smith acknowledged that technology companies have been slow to accept that responsibility.
"There are important discussions and debates all across the country about when AI should be used in schools, how it should be used, for what purpose," he said. "It is our role, even our responsibility, to ensure that AI is provided in a form that teachers and kids and parents can trust."
