A lawsuit against Google has raised fresh concerns about the safety of artificial intelligence tools accessible to children, following allegations that a 13-year-old girl used the company’s AI-powered search features while experiencing a serious psychological crisis.
The case was brought by the teenager’s parents in a California court on Monday. They are accusing the technology company of negligence, arguing that its AI features failed to adequately protect their daughter during interactions involving suicide-related content.
Channels Television reported that the girl, identified as Emma, used a school-issued Chromebook to access Google’s AI-enhanced search functions. Although her school had restricted access to standalone AI applications, including Gemini and ChatGPT, the search engine’s AI capabilities remained available to her.
The extent of the situation became apparent on March 17, when the school’s monitoring system detected a file bearing the title “Funeral”. The discovery prompted school authorities to contact her parents, who subsequently found her preparing a farewell letter. Her father intervened.
The family described Emma as a previously happy and academically successful child who participated in music activities and the Girl Scouts. Their complaint alleges that she had no history of mental illness before the events surrounding her use of Google’s AI technology.
Google has disputed the implication that its safeguards were adequate grounds for the allegations. Its spokesperson, Jose Castaneda, said the company was reviewing the case and treating the claims seriously. Google also maintains that its AI Mode has measures intended to prevent harmful self-harm interactions.
Separately, Google, Meta and TikTok have challenged a California law seeking to restrict addictive social media features for users below 16. The legislation, signed by Governor Gavin Newsom in September, is due to take effect on January 1, 2027.
