Sloplaw and disorder: AI-generated citations nearly fool judge in court ruling

The legal profession faces another AI citation scandal as a judge narrowly avoids incorporating hallucinated case law into an official ruling. This incident highlights the growing problem of unverified AI-generated content in legal proceedings and demonstrates how even experienced legal professionals can be deceived by convincingly fabricated citations, raising serious questions about professional responsibility in the AI era.

The incident: A retired US magistrate judge serving as special master has sanctioned two law firms and ordered them to pay $31,100 for submitting fake AI-generated citations in legal briefs.

• Judge Michael Wilner admitted he initially thought the citations were legitimate and “almost” included them in a judicial order before discovering they didn’t exist.
• The judge described the situation as “scary,” noting that “strong deterrence is needed” to prevent attorneys from using AI as an “easy shortcut” in legal writing.

The underlying case: The sanctioned attorneys represent former Los Angeles County District Attorney Jackie Lacey in a lawsuit against State Farm Insurance.

• Lacey’s lawsuit alleges the insurance company refused to provide legal defense to her late husband, who faced a civil suit after pointing a gun at activists on their porch.
• The legal team included attorneys from the major firm K&L Gates and the smaller Ellis George LLP in what the judge called “a collective debacle.”

How it happened: The attorneys admitted using AI to generate an outline for a supplemental brief without proper verification.

• Attorney Trent Copeland of Ellis George used AI tools to draft the document, but no one at either firm reviewed or cite-checked the research before filing.
• Judge Wilner rejected the attorneys’ defense that the AI hallucinations “weren’t too far off the mark” in their recitations of substantive law, calling it a “pretty weak no-harm, no-foul defense.”

The consequences: The judge imposed financial penalties on the law firms rather than sanctioning individual attorneys.

• Ellis George and K&L Gates must pay $26,100 to reimburse the defense for fees paid to an arbitration and mediation firm.
• The firms must pay an additional $5,000 to cover some of the defense’s other costs.

The bigger picture: This case represents part of a growing trend of lawyers being caught submitting AI-generated fake citations in court.

• In some cases, opposing attorneys discover the fabricated citations and alert the judge.
• The incident demonstrates how AI hallucinations can infiltrate the legal system, potentially affecting judicial decisions if not caught and corrected.