ChatGPT and its AI cousins have become the go-to tools for anyone looking to cut corners. Need to whip up a report you haven’t read? There’s a bot for that. Want to sound smarter in an email than you actually are? AI’s got your back. But when this habit creeps into professions where accuracy isn’t just important-it’s everything-things get messy fast.
Take the legal profession, for example. Those lawyers who are paragons of thorough research and proper argumentation have seemingly found that AI can enable them to blow through the drudgery of writing briefs. The problem? These chatbots have a nasty habit of making stuff up, and those lawyers relying on them without double-checking are finding themselves in hot water.
The Growing Crisis of AI-Generated Nonsense in American Courts
Recently, The New York Times did an eye-opening exposé into this growing problem, where courts across the country are increasingly handing out punishments to attorneys caught submitting AI-generated nonsense. We are talking about fabricated legal cases, made-up precedents, and citations that simply do not exist presented to judges as legitimate legal arguments.
The thing is, this isn’t actually a Rules violation. The American Bar Association has said that such tools can be used by lawyers so long as the lawyer verifies that what the AI produced is accurate. Sounds like a pretty reasonable requirement.
Yet somehow, attorneys continue skipping that required verification and submitting documents full of hallucinated case law.
The consequences are beginning to mount. Judges are responding with fines and discipline, according to court filings and legal experts. These punishments might sound minor—a small fine here, a stern warning there, but they signal a growing crisis of credibility for the legal profession.
The Times observes that AI errors in legal venues can be traced to two sources. First, there are the regular people representing themselves in court, who use chatbots because they can’t afford lawyers. Jesse Schaefer, a North Carolina attorney, indicates that this kind of tool can allow self-represented litigants “to speak in a language that judges will understand.”
That’s actually somewhat defensible, people doing their best with limited resources.
But the second source is considerably more disturbing: real lawyers, those who attended law school and passed the bar examination, submit work created by AI without even taking the time to fact-check.
Isolated incidents no more, the problem has grown widespread enough that some lawyers have dedicated themselves to tracking down these cases and then making the public aware of them.
500+ Cases of Fabricated Information Submitted by Lawyers
Enter Damien Charlotin, a French attorney who became so concerned about AI misuse in legal proceedings that he designed an online database devoted to documenting these failures. His website is a sobering scroll through professional incompetence. It runs 11 pages now, and it includes 509 documented cases of lawyers submitting AI-generated material that contained fabricated information.
Charlotin isn’t a solo act. According to the Times, a growing network of lawyers has formed around the mission of unmasking AI abuses committed by their colleagues. They’re posting those cases online, creating a sort of public record of shame they hope will prevent others from committing the same mistakes.
Stephen Gillers, an ethics professor at New York University School of Law, does not mince words about the situation. “These cases are damaging the reputation of the bar,” he told the newspaper. “Lawyers everywhere should be ashamed of what members of their profession are doing.”
The real question is whether public shaming and minor penalties are enough to fix the problem. So far, the evidence suggests they’re not. Cases continue to mount in Charlotin’s database, meaning lawyers are still making the same mistakes despite knowing that others have been caught and punished.
The whole incident speaks volumes about the deeper problem of how we’re integrating AI into professional work. The technology can be truly useful for drafting, brainstorming, and research-but it requires human oversight.
The Legal Profession’s Urgent AI Verification Crisis
When professionals in high-stakes fields start treating AI output as reliable without verification, they’re not just being lazy. They’re betraying the trust placed in them by clients and the legal system itself.
For attorneys, the stakes don’t get much higher. The outcomes of legal cases affect people’s lives, their freedom, their finances, and their futures. Submitting fabricated legal precedents isn’t just embarrassing-it’s potentially devastating to the clients depending on competent representation.
Perhaps the real solution is not shame and small fines. It may be time for bar associations to take steps toward requiring training about AI tools, the limitations of those tools, and how to properly verify what’s come out of them. Right now, the legal profession has an AI problem, and hoping that lawyers will suddenly develop better judgment does not appear to be working.
