AI hallucinations in court

Arizona Court of Appeals decision urges caution when using generative AI in legal briefing.

Artificial intelligence has changed the way many people handle everyday tasks, and the law is no exception. If you are facing a legal problem in Arizona — a will dispute, a landlord issue, a small business question — you may be wondering whether AI tools can help. The short answer is yes, they can. But a recent Arizona case illustrates how AI is a tool, not an attorney, and should be used with caution.

A Cautionary Tale from an Arizona Courtroom.

The case is called Matter of Estate of Acciavatti. It grew out of a family dispute among the adult children after their 93-year-old mother passed away. One of the siblings disagreed with a ruling of the trial court and appealed. He did not hire an attorney. He represented himself. To prepare his written argument to the court— called a brief—he turned to generative AI. That is the type of AI that writes text for you when you type in a request. He used it to research the law and build his case. On its face, this looked efficient: type in your problem, get a polished-looking legal document in return.

But there was a serious problem hiding inside that polished document.

When the judges reviewed the brief, they found that the legal citations—the references to court cases that are supposed to support an argument—were riddled with errors. Of the eight citations in his brief, six were defective. Some pointed to the wrong pages. Some described cases inaccurately. And most alarming of all, two of the cases simply did not exist.

This is a known flaw in AI tools. When an AI does not have a real answer, it sometimes invents one that looks convincing. Lawyers and judges now call these made-up cases “hallucinations.” The Arizona court described unchecked AI bluntly: left unchecked, it is an unreliable narrator and researcher that produces citations to authorities that do not exist.

Here is the part that should stay with anyone thinking about relying on AI in court. The court explained that you can type a simple request into an AI tool and instantly receive a brief filled with plausible-looking legal arguments and citations. If you do not check it carefully, the result can look like competent, convincing legal work, but very often it is not.

In other words, even though the output looks professional, it may not be. The party who filed the problematic brief in Acciavatti admitted to the court that he did nothing to independently confirm the citations were real. He simply trusted AI.

The court did not accept that “the AI did it” as an excuse. It made clear that anyone who files documents with a court is personally responsible for making sure they are truthful and accurate. You cannot hand that responsibility off to a piece of technology. As the court put it, a person may not offload their duty to ensure the truth and accuracy of their filings to any technological tool, including AI.

You may be wondering what happened to the party who filed the brief. Well, he lost. Worse, the court imposed sanctions—a financial penalty—and ordered him to pay part of his sister’s attorneys’ fees and costs. What began as an attempt to save money on legal help ended up costing him, both in his case and in his wallet.

It is worth noting that the court held the self-represented party to the same standard as a trained lawyer. In Arizona, people who represent themselves must follow the same rules as attorneys. They do not get a pass simply because they are not lawyers, and they do not get a pass for mistakes made by an AI tool.

It is also worth noting that Acciavatti is not a unique case. The court cited cases from across the country where parties cited incorrect or hallucinated cases in support of their positions. This case adds to the many cautionary tales from other states. It also shows why it is important to be aware of the possible pitfalls of using AI in the courtroom.

Does AI have any use in the legal field?

The takeaway from Acciavatti is not that one should never use AI. The court itself said AI can be a genuine aid. The takeaway is about how you use it.

Think of AI the way you would think of a calculator or an online map. A calculator is fast and reliable for math, but it cannot tell you whether you are solving the right problem. A map can show you a route, but you still have to notice the road is closed. AI can gather information and draft text quickly, but it cannot exercise judgment. It does not truly understand your situation, it cannot weigh strategy, and it cannot always tell the difference between a real court case and one it invented.

AI can be a helpful tool. It is not a substitute for qualified legal judgment. A skilled attorney does more than find cases. They confirm the law is real and current, apply it to your specific facts, spot risks you would never think to look for, and stand behind their work professionally.

If you are considering using AI for a legal matter, here is a sensible way to think about it:

• Use AI to learn and prepare. Let it help you understand basic terms, organize your questions, and get oriented before a meeting with a lawyer.
• Never treat AI output as final legal advice. Anything it tells you about the law needs to be verified by you or a qualified professional.
• When in doubt, bring a professional into the loop. If you are filing something with a court, signing a contract, or making a decision with real consequences, ask a licensed Arizona attorney to review it.

Used wisely, AI can make legal help more accessible and less intimidating. Used carelessly, it can lead you into exactly the kind of trouble Acciavatti warns of. The smartest approach combines the speed and convenience of new technology with the judgment and accountability of a real professional.

If you have a legal question and are not sure where AI fits in, reach out to a qualified attorney. The right guidance early on can save you far more than it costs.

IMPORTANT NOTE: This post is for general educational purposes and is not legal advice. The image for this article was generated by OpenAI.