'I only told the AI' — chatbot conversations are showing up as courtroom evidence

Private conversations with AI are increasingly finding their way into courtrooms. As more people turn to chatbots for help with work, health concerns and emotional support, the companies behind them have accumulated vast stores of intimate data — and courts are taking notice.

An analysis by The Washington Post of public court records and local news reports found that chatbot conversation logs were cited as evidence in at least 12 lawsuits over the past two years. Because police and civil litigants are not required to submit all evidence they obtain, the actual number of cases is likely higher.

A teenage boy in the United States had turned to ChatGPT whenever he struggled to understand his father. Those conversations were entered into the court record by the defense in a lawsuit he filed in 2023 against Meta, Snapchat, TikTok and YouTube. His attorney, Mike Morgan, said the boy had no idea the exchanges could end up in an opposing party's report. "A 15-year-old will tell a chatbot things he wouldn't tell a therapist or his parents," Morgan said, "but he has no idea those conversations can be cited in the other side's filings."

AI has also caught warning signs early enough to prevent crimes. In May, OpenAI alerted the FBI after a user repeatedly told ChatGPT about plans to kill his ex-girlfriend. The man was arrested on stalking and threat charges and sentenced to eight years of probation. In Brazil, a separate case in which a man sought to hire a hitman to kill himself and his son was passed by OpenAI to local authorities, also resulting in an arrest.

OpenAI has not always acted so swiftly, however. In February, the company suspended the account of the suspect in a mass shooting in Tumbler Ridge, Canada, for policy violations — but did not notify Canadian authorities. OpenAI CEO Sam Altman later apologized. The suspect in a shooting at Florida State University that killed two people had also exchanged messages with ChatGPT about his plans before the attack, yet was similarly not reported. Facing mounting criticism, OpenAI said it would more proactively alert authorities to concerning conversations.

Legal experts say chatbot logs contain far richer information than search histories. Michael Price, litigation director at the National Association of Criminal Defense Lawyers, said the full context of what a user typed and how a conversation unfolded makes it possible to see clearly what a person was thinking. He described such logs as a window into the most intimate parts of a person's life.

According to data released by OpenAI, the company provided information on more than 80 accounts to government and law enforcement agencies in the second half of last year — more than four times the figure from the year before.

Altman has argued that chatbot conversations should receive the same legal protections as consultations with doctors or lawyers. But Laura Abelson, a professor at Southern Methodist University, said courts are very unlikely to recognize such a privilege. "This is an unregulated space," she said, "and courts are very unlikely to extend confidentiality protections to these conversations."

A deeper question looms: should dangerous statements that never lead to actual crimes be grounds for reporting and punishment? The stalking and murder-for-hire cases were backed by concrete circumstances that justified arrests. But if every flash of anger or extreme expression becomes reportable, people could face prosecution for thoughts they never acted on.

The dilemma is set to grow as AI "agents" — systems that act on a user's behalf and demand access to ever more data — become more widespread. Society has yet to reach a consensus on where prevention ends and invasion of privacy begins, caught between AI's potential to stop crimes before they happen and the risk of subjecting unacted-upon thoughts to surveillance and punishment.