SAN FRANCISCO: The parents of a 13-year-old girl have sued Google, claiming she planned a potential suicide and funeral using its AI technology on a school-issued computer.
Filed on Monday in a California court, the complaint is the first known case accusing Google of negligence related to AI integrations on Chromebooks, the tech giant’s low-cost laptops used by millions of students in the United States and around the world.
The teenager, identified by the name Emma, was unable to access Google’s Gemini app or OpenAI’s ChatGPT because they were blocked by her school.
She was nonetheless able to use Google’s search engine, which contains AI-powered tools.
Those features were notably not among the tools that Google announced this week would be disabled by default for students.
“Schools should turn search off and keep it off until Google removes AI mode from search or ensures that every single student is always safe when they’re interacting with AI,” Andrew Liddell, one of the family’s attorneys, told AFP on Thursday.
The complaint alleges that Emma had been a “happy 13-year-old girl” in North Carolina who received good grades, played music and participated in Girl Scouts.
Despite having “no history” of mental illness, she began a “quiet spiral into a severe psychological crisis” within months of using Google’s AI-powered search on her school laptop, according to the complaint.
On March 17, the school’s monitoring software flagged a document titled “Funeral”, and the school notified her parents.
Her mother then saw Emma writing a farewell letter in real time after logging into her daughter’s account, and her father intervened.
“We are reviewing the claims in the lawsuit and we take this very seriously,” Google spokesperson Jose Castaneda told AFP.
Google argues that AI mode includes safeguards against self-harm and attempts to bypass them.
But Emma “was able to keep going back over and over and over again,” Liddell said. “Whatever safeguards they think they put in place were obviously insufficient.”
Developers of fast-evolving AI technology are facing a litany of similar lawsuits in the United States.
OpenAI faces several wrongful death suits, including one filed by the parents of 16-year-old Adam Raine, a teenager who died by suicide after allegedly using methods advised by ChatGPT.
Google and the startup Character.AI agreed in January to settle several lawsuits filed by families of teenagers out of court.
On Wednesday, Google, TikTok and Meta each separately sued California to block a law prohibiting social media companies from providing addictive features to users under 16.
Governor Gavin Newsom signed the law in September, and it is due to take effect on Jan 1, 2027.
