MANDEL: AI 'hallucinations' hit second GTA court case in a month
Article content
They’re called AI “hallucinations” and the made-up citations generated by artificial intelligence chatbots have just landed a second Ontario lawyer this month in hot water.
The latest case involves Ontario Court Justice Joseph Kenkel rejecting written final arguments by defence lawyer Arvin Ross in an aggravated assault trial because he relied on a case that apparently doesn’t actually exist.
tap here to see other videos from our team.
MANDEL: AI 'hallucinations' hit second GTA court case in a month Back to video
tap here to see other videos from our team.
“Unfortunately, there are serious problems with the defence submissions,” the Newmarket judge wrote in a ruling on Monday. “One of the cases cited appears to be fictitious. The court was unable to find any case at that citation. There was no case by that name with that content at any other citation.
Kenkel instructed him to redraft his submission without using generative AI or commercial legal software that uses GenAI.
“Mr. Ross has done a good job presenting the defence in this case. I’m confident that he will be able to prepare proper submissions within these guidelines,” he concluded.
Reached by email, the lawyer declined comment since the matter is before the court. “I am focused on complying with the court’s directions and appreciate your understanding,” Ross wrote.
The first court case to deal with the new problem of AI-generated fictions, known as “hallucinations,” occurred earlier this month in Toronto.
Lawyer Jisuh Lee was before Superior Court Justice Fred Myers to argue that her client’s divorce should be set aside and a trustee removed when the judge noticed her written materials rested on past decisions where the hyperlinks took him to non-existent cases or to ones which dealt with a completely different area of law.
Recommended video
tap here to see other videos from our team.
He suspected a wonky AI program was at fault but chastised the lawyer for not reviewing her cases before arguing them in court. Myers ordered Lee to appear before him on May 16 to show cause why she shouldn’t be cited for contempt.
“A court decision that is based on fake laws would be an outrageous miscarriage of justice to the parties and would reflect very poorly on the court and the civil justice system,” he said.
In response, the shocked 30-year veteran lawyer has “thrown herself on the mercy of the court,” Myers wrote.
Lee delivered a letter to the court on May 9 explaining that she’s learned that her staff had used ChatGPT to prepare her factum and the non-existent cases were AI hallucinations. She apologized for not verifying their authenticity and asked not to be found in contempt.
“This was a serious lapse in judgment. It does not reflect the standard of diligence and care I have always tried to uphold in my practice, nor does it reflect my respect for this court,” she said in a later statement to the court.
Lee also pledged to take six hours of continuing professional education in the proper use and risks of AI in a law practise.
Following her apology and noting the widespread notoriety of the case — the judge said she was bombarded by calls from reporters and colleagues — Myers withdrew the contempt motion.
“In my view, the publicity surrounding this case has served both to publicly denounce inappropriate conduct and as general deterrence to the bar and others who might rely on AI for legal submissions,” he wrote.
In the U.S., lawyers who have filed briefs with fake cases generated by AI have been fined $5,000 — but Myers declined to follow suit.
“There had to be someone who was going to be the first lawyer to file AI hallucinations here. It was likely to be someone so junior as to overestimate the infallibility of AI, or someone so senior as to not really yet understand its fallibility. Ms. Lee has suffered a public shaming near the end of an unblemished career.
“The denunciation and deterrent effect produced her immediate and forthright response in a manner far beyond any reasonably expected impact of a small fine.”
Postmedia is committed to maintaining a lively but civil forum for discussion. Please keep comments relevant and respectful. Comments may take up to an hour to appear on the site. You will receive an email if there is a reply to your comment, an update to a thread you follow or if a user you follow comments. Visit our Community Guidelines for more information.