Courting AI opportunities (and hallucinations)

A view of the US Supreme Court, in Washington, D.C., on Monday, Jan. 8, 2024.

A view of the US Supreme Court, in Washington, D.C., on Monday, Jan. 8, 2024.

Graeme Sloan/Sipa USA via Reuters

The AI boom, he said, brings both opportunities and concerns, noting that legal research may soon be “unimaginable” without the assistance of AI. “AI obviously has great potential to dramatically increase access to key information for lawyers and non-lawyers alike,” he wrote.

But he also cautioned “humility,” noting how “one of AI’s prominent applications made headlines this year for a shortcoming known as ‘hallucination,’ which caused the lawyers using the application to submit briefs with citations to nonexistent cases. (Always a bad idea.)” Indeed, AI chatbots tend to make stuff up — or hallucinate, which has been a repeated problem since the debut of ChatGPT.

So far, US federal courts have taken a decentralized approach, with 14 of 196 publishing their own gruidance on how AI tools can and cannot be used in litigation.

Meanwhile, across the pond, the United Kingdom recently took the first step toward allowing AI as an assistive tool in legal opinion writing. “Judges do not need to shun the careful use of AI,” high-ranking judge Geoffrey Vos wrote. “But they must ensure that they protect confidence and take full personal responsibility for everything they produce.”

So British courts will begin allowing AI to be used in legal writing, but not research — because of the aforementioned tendency to hallucinate.

Will AI judges take over? Roberts made an eloquent case against this and an impassioned defense of the humanity central to being an effective judge.

“Machines cannot fully replace key actors in court,” he wrote. “Judges, for example, measure the sincerity of a defendant’s allocution at sentencing. Nuance matters: Much can turn on a shaking hand, a quivering voice, a change of inflection, a bead of sweat, a moment’s hesitation, a fleeting break in eye contact. And most people still trust humans more than machines to perceive and draw the right inferences from these clues.”

More from GZERO Media

Listen: In seven short weeks, the Trump administration has completely reshaped US foreign policy and upended trade alliances. Will China benefit from US retrenchment and increasing global uncertainty, or will its struggling economy hold it back? On the GZERO World Podcast, Bill Bishop, a China analyst and author of the Sinocism newsletter, joins Ian Bremmer for a wide-ranging conversation about China—its domestic priorities, global administration, and whether America’s retreat from global commitments is opening new doors for Beijing.

German Chancellor-in-waiting Friedrich Merz speaks to the media after he reached an agreement with the Greens on a massive increase in state borrowing just days ahead of a parliamentary vote next week, in Berlin, Germany, on March 14, 2025.
REUTERS/Axel Schmidt

Germany’s election-winning center-right Christian Democratic Union/Christian Social Union, led by Friedrich Merz, and the Social Democrats have reached a preliminary agreement with the Green Party on a deal to exclude defense spending from the country’s constitutional debt break and establish a dedicated $545 billion fund for infrastructure investments.

A Russian army soldier walks along a ruined street of Malaya Loknya settlement, which was recently retaken by Russia's armed forces in the course of Russia-Ukraine conflict in the Kursk region, on March 13, 2025.

Russian Defence Ministry/Handout via REUTERS

The Russian leader has conditions of his own for any ceasefire with Ukraine, and he also wants a meeting with Donald Trump.

Mahmoud Khalil speaks to members of the media about the Revolt for Rafah encampment at Columbia University on June 1, 2024.

REUTERS/Jeenah Moon

The court battle over whether the US can deport Mahmoud Khalil, the 30-year-old Palestinian-Algerian activist detained in New York last Saturday, began this week in Manhattan. Khalil, an outspoken activist for Palestinian rights at Columbia University, was arrested Saturday at his apartment in a university-owned building at Columbia University by Immigration and Customs Enforcement officers, and he is now being held in an ICE detention center in Louisiana.

The Israeli Air Force launched an airstrike on Thursday, targeting a building in the Mashrou Dummar area of Damascus.
(Photo by Rami Alsayed/NurPhoto)

An Israeli airstrike destroyed a residential building on the outskirts of Damascus on Thursday in the latest Israeli incursion into post-Assad Syria.

Lars Klingbeil (l), Chairman of the SPD parliamentary group, and Friedrich Merz, CDU Chairman and Chairman of the CDU/CSU parliamentary group, talk at the end of the 213th plenary session of the 20th legislative period in the German Bundestag.

Germany’s government is in a state of uncertainty as the outgoing government races to push through a huge, and highly controversial, new spending package before its term ends early this spring.

EPA Administrator Lee Zeldin, a Republican, speaks as the U.S. vice president visits East Palestine, Ohio, U.S., February 3, 2025.
Rebecca Droke/Pool via REUTERS/File Photo

On Wednesday, Environmental Protection Agency chief Lee Zeldin redefined the agency’s mission, stating that its focus is to “lower the cost of buying a car, heating a home, and running a business.”