Professors at the University of Minnesota Law School published a study on April 8 on SSRN that tested whether upper-level law students who relied on AI during the early stages of a project experienced declining comprehension and impaired legal reasoning skills in later stages, when the AI was no longer an option. The results painted a more complex picture of AI's impact on legal reasoning.
As researchers anticipated, participants who used AI early in a project achieved higher quality work more quickly than those without access to the technology. But it also turned out that those same participants with AI access in the early stages outperformed the other students in later stages, even when neither group had AI access.
The researchers concluded that AI exposure in the initial stages did not diminish the downstream comprehension of underlying legal principles.
"Using AI for lawyering tasks well can not only help you produce better work faster, but can also make you a better lawyer," Daniel Schwarcz, a professor at the University of Minnesota Law School and a co-author of the study, told Law360 Pulse.
Schwarcz said that the improvement stemmed from AI helping participants develop a stronger understanding of the governing legal framework and authorities.
The results showed that recent AI tools produced better quality work than earlier models and that AI usage improved the overall performance by nearly 60%.
Researchers conducted a randomized controlled trial with about 100 second- and third-year law students at the University of Minnesota Law School. The four tasks in the trial included writing a memo synthesizing the law from a package of legal materials, answering multiple-choice questions on the materials, writing a memo applying the material to a fact pattern and revising the second memo.
Participants were randomly assigned to either a control group that didn't get AI access until the final revision task or a group that had AI for both the initial and final tasks but not for the intervening comprehension and application tasks.
When all participants used AI to revise memos, those who started with weaker memos improved while participants with stronger memos regressed, according to the results.
Researchers concluded that AI neither hurts independent legal reasoning nor helps promote it, with its impact depending on when and how students get access to AI. Schwarcz said that cognitive fatigue and time pressure may have also impacted the results during the revision stage.
While questions remain about the long-term impact of AI on junior attorneys, the researchers believe the results show that short-term AI does not automatically subvert independent reasoning.
Schwarcz said there are a few takeaways for legal professionals. First, AI should only be used when the AI's reasoning can be assessed and explained. Second, professionals should limit AI to narrow and well-defined parts of a project. Finally, AI use should be limited while under tight deadlines and during cognitive fatigue.
--Editing by Robert Rudinger.
Social Science Research Network, or SSRN, is owned by Elsevier, part of the RELX Group, which also owns Law360.
For a reprint of this article, please contact reprints@law360.com.
