Civil Lit Tweet’s Gordon Exall highlighted a cringeworthy judgment today that really demonstrates the dangers of AI in a legal context. More importantly it demonstrates the particular dangers for people who are not egally trained, and don’t know what they are looking for or how to verify results or spot hallucinations and bad points.
In this case a litigant in person who was representing herself at a tax tribunal appeal quoted nine AI-generated Tribunal decisions on ‘reasonable excuse’. They looked plausible, but when the lawyers smelled a rat, it was discovered that none of them actually existed and they didn’t actually reflect the law. Luckily for the litigant in person the tribunal accepted that the litigant in person was unaware of the issue – she hadn’t known to check BAILII or The National Archives for the source material. There is a slightly more detailed summary of the case on Gordon’s blog (link above), and for those who want to read the full judgment (which has possibly the longest and most unwieldy title I’ve ever seen), you can read it here:
Harber v Commissioners for His Majesty’s Revenue and Customs (INCOME TAX – penalties for failure to notify liability to CGT – appellant relied on case law which could not be found on any legal website – whether cases generated by artificial intelligence such as ChatGPT)  UKFTT 1007.
Anyway, the moral of the story is: Do not rely on AI to do your legal research for you. It is a VERY. BAD. IDEA.
If you MUST use AI to assist you with your legal research, do so with extreme caution: at the very least you must verify all references, source all the judgments in full from an authoritative source, and read all the cases (assuming they exist) to ensure the AI has got it right. This is not an easy task if you aren’t a lawyer. Even once you have done all that, you run the risk, I suspect, of missing important decisions or pieces of law which undermine your arguments, which the AI might have selected out as inconvenient, but it would be better to understand before you put your foot in it.
It’s also worth pondering: if you’ve done all that have you actually saved yourself any time? I’d say probably not.
If you can pay for legal advice from someone who knows their onions, do it. If you can’t, see if you can get some free advice from a law centre, clinic or other free provision or go to authoritative sources: there are plenty of books and leaflets and other publications by lawyers or legal charities that are designed specifically to accurately explain law and procedure to people who aren’t lawyers, so that they are steered in the right direction rather than into a hole or hot water.
If you are a lawyer and are relying on Chat GPT without properly verifying EVERY DARNED WORD, then shame on you.