Artificial intelligence chatbots like ChatGPT are reshaping how we communicate and conduct business. It is also affecting our judicial system, but not always in a helpful way.
Courts across the country have been sanctioning trained attorneys for filing briefs containing citations imagined by AI chatbots. If licensed lawyers are getting caught by this, the risk to someone researching their own claim is worse. You don’t have a lawyer checking the citations, and you might not learn that answer you depended on was wrong until a deadline has already passed.
ChatGPT is a remarkable tool, but it has limits. Here’s what those limits mean for you, where a chatbot’s limits start costing real money, and what a San Pedro personal injury lawyer can tell you that a chatbot can’t.
The Risks of Relying on AI for Legal Advice
The expanding use of AI in legal proceedings has not gone unnoticed by the American Bar Association (ABA). This is a growing concern not only for lawyers but also for any potential client seeking legal advice. As the ABA notes, “AI platforms also lack the human judgment and empathy that lawyers bring to the table.”
Here are four risks that attorneys and clients can face when relying on AI for legal advice:
- AI Hallucinations: ChatGPT has access to a vast amount of legal knowledge. However, in its effort to be “helpful,” it frequently invents fake court cases, nonexistent statutes, and unverified citations. These are referred to as hallucinations. Lawyers and pro se users have faced severe court sanctions and penalties for submitting these fabricated precedents.
- Loss of Privilege and Confidentiality: One of the biggest concerns with using a chatbot is surrendering your privacy. Buried deep in the “terms of service” for most chatbots is the permission for the bot to use whatever conversation you generate for “training purposes.”
- Lack of Nuance and Strategy: AI chatbots lack genuine legal reasoning, professional judgment, and local jurisdictional awareness. They tend to give overconfident, generalized answers. That doesn’t always account for the complex risks or practical consequences of a case’s particulars.
- No Accountability: Unlike attorneys who can be sanctioned or worse, there is no regulatory oversight for a chatbot, at least not yet. That exposes users to no recourse when errors occur.
Attorneys across the country have learned hard lessons about relying on ChatGPT and other AI tools for legal research and guidance.
A California attorney was recently fined $10,000 for filing a state court of appeal brief in which 21 out of 23 quotations had been invented by ChatGPT.
In Mississippi, a federal judge punished both sides involved in a sanctions order for using AI-created case quotations. The judge removed four lawyers from the case and barred two of them from appearing in district court for two years.
These are just two recent examples of how using ChatGPT has gotten attorneys in hot water and put their careers at risk.
Why ChatGPT Cannot Replace a Qualified Personal Injury Lawyer
Only a flesh-and-blood, bar-certified attorney can bring your case forward. Only a real lawyer can help determine how much your case could be worth.
A chatbot might spit out a false number backed by an imaginary legal citation. Acting on that information can be quite expensive. The chatbot’s figure will not likely be based on anything about your specific case. Not your medical records, nor the defendant’s policy limits, your share of fault, or what juries in your county have historically awarded. You don’t want a false figure in your head as you negotiate with an adjuster.
There’s also the work a chatbot simply cannot do.
- It can draft a demand letter but can’t make the follow-up call when the letter is ignored.
- It can describe a deposition, but it can’t take one.
- It can’t cross-examine the driver who hit you, argue a motion, or read a room during a mediation.
- It won’t notice that the adjuster’s second offer arrived suspiciously fast, or that the police report has the lanes reversed.
None of that means the technology is useless. Used carefully, it can help you frame a question before a consultation or make sense of a term in a letter you don’t understand. That’s a reasonable use for the tool. Treating a chatbot’s answer as legal advice you can act on is not helpful.
If a chatbot gives you a case citation, a deadline, or a dollar figure, treat all three as a starting point for a question, never as an answer.
Talk to Someone Who Is Accountable for the Answer
The trouble with a confident wrong answer is that it doesn’t feel any different from a right one. That’s what caught the attorneys in California and Mississippi, and they had training, research databases, and every reason to double-check. Someone recovering from an injury and trying to understand their own claim has none of those advantages and far more riding on it.
Frost Law Firm, PC handles personal injury, product liability, and catastrophic injury claims out of San Pedro, and our attorneys can tell you what your case is actually worth after looking at your records, the coverage available, and how these claims resolve in California. If you have questions about an injury claim, speak with an attorney about your options. An answer you can rely on comes with someone’s name attached to it.
