Judges Are Noticing Error-Ridden, AI-Generated Legal Briefs

Judges around the world are dealing with a growing problem: legal briefs that were generated with the help of artificial intelligence and submitted with errors such as citations to cases that don't exist, according to attorneys and court documents. The trend serves as a cautionary tale for people who are learning to use AI tools at work, the AP reports. Many employers want to hire workers who can use the technology to help with tasks such as conducting research and drafting reports. As teachers, accountants, and marketing professionals begin engaging with AI chatbots and assistants to generate ideas and improve productivity, they're also discovering the programs can make mistakes.

A French data scientist and lawyer, Damien Charlotin, has catalogued at least 490 court filings in the past six months that contained "hallucinations," which are AI responses that contain false or misleading information. The pace is accelerating as more people use AI, he said. "Even the more sophisticated player can have an issue with this," Charlotin said. "AI can be a boon. It's wonderful, but also there are these pitfalls." Charlotin, a senior research fellow at HEC Paris, a business school located just outside France's capital city, created a database to track cases in which a judge ruled that generative AI produced hallucinated content such as fabricated case law and false quotes. The majority of rulings are from US cases in which plaintiffs represented themselves without an attorney, he said.

But even high-profile companies have submitted problematic legal documents. A federal judge in Colorado ruled that a lawyer for MyPillow Inc., filed a brief containing nearly 30 defective citations as part of a defamation case against the company and founder Michael Lindell. The legal profession isn't the only one wrestling with AI's foibles. The AI overviews that appear at the top of web search result pages frequently contain errors. And AI tools also raise privacy concerns. Workers in all industries need to be cautious about the details they upload or put into prompts to ensure they're safeguarding the confidential information of employers and clients. (For more, the AP has tips for how to use AI at work more safely.)