The news and social media are so saturated with stories about artificial intelligence (AI) that another piece on the topic may seem one straw too many for the camel to bear. We are inundated on a daily basis with accounts that AI will improve efficiency, enhance productivity, replace human workers and/or bring about the end of humanity. School districts, however, must be cautioned about the unique risks that AI poses when relied upon for legal advice. This article will highlight two serious problems with using publicly-available AI to perform research on legal issues or generate documents in litigation: so-called “hallucinations” and lack of privilege.
The existence of AI “hallucinations” has been recognized for several years. Put in simple terms, AI functions are constructed to supply the requestor with what they sense the requestor wants – sometimes to a fault. Thus, for example, if asked whether the law permits a particular action, an AI agent will provide such legal support as it can find in support of that action. It often overlooks or disregards legal authority that is contrary to the action, even if that contrary legal authority may present the legally correct response. It is also known to mistakenly interpret proposed legislation for actual law. Even worse, if sufficient legal support for the proposition does not exist, the AI will simply make something up! This is not a rare occurrence – an increasing number of lawyers have been sanctioned for submitting materials in litigation that turned out to be entirely AI-generated and which cited nonexistent cases.
In one well-publicized case from downstate New York, Mata v. Avianca, Inc., the lawyers for a plaintiff suing an airline submitted an AI-generated brief to the court, only to have the airline’s lawyers point out that the six primary cases relied upon in the brief were all imaginary. They also noted that other cases cited by the brief, though real, did not stand for the propositions cited – that is, the AI had taken real cases and assigned to them the meaning it desired to answer the inquiry, without regard to what they actually said. The lawyers in the Mata case were fined $5,000 for their actions.
In another case, a lawyer from Texas who submitted briefs with AI hallucinations in a federal lawsuit in Indiana was initially fined $15,000, although the fine was later reduced to $6,000. These examples involving lawyers who were persuaded by the seemingly plausible legal analysis presented by AI platforms should serve as a warning for non-attorneys in particular as they lack legal training and experience, which presumably would make them even more vulnerable to mistaken reliance on faulty AI legal analysis.
While AI can be a powerful tool for streamlining the initial hunt for relevant legal authority, it is up to the user to verify the information it provides – and that is something that relies heavily on the special expertise of a lawyer. Responsible attorneys will use authoritative sources with which they are uniquely familiar to verify not only the existence, but the substance, of the caselaw and arguments provided by AI functions.
Conversations with public AI platforms like ChatGPT can be expected to be “discoverable” – that is, subject to being obtained by an opposing party in litigation, or potentially by parties in litigation where the school district is merely a bystander. (There are AI platforms which, unlike ChatGPT, do not share information outside the organization using them, and they are generally viewed as safe for use within law firms.) The information that can be obtained by reviewing AI conversations used to generate legal advice is potentially of high value to an adversary. As a result, it is not a good idea to use public AI platforms to investigate legal issues, whether litigation is underway or not, as doing so may expose your school to liability.
While AI may be a useful tool in other contexts, its limitations and lack of legal protection from disclosure render it a poor source for legal advice on school matters. Educational institutions are advised to rely on their school attorneys for their legal needs instead.
Members of the New York State Association of School Attorneys represent school boards and school districts. This article was written by Charles “Chris” Spagnoli Chris Spagnoli of Ferrara Fiorenza PC.