AI

Beyond the Citation Check

A paired evaluation of two commercial legal AI research tools on six questions of Australian law, and why most of their errors would pass a citation check.

76%
of failures identified could not be caught by confirming a citation exists, is quoted faithfully and supports what it is cited for
2
defects in quoted source text found, one invented and one truncated and presented as complete, both in one system’s standard mode, across 24 outputs and two regenerations
19.5 vs 0
median case citations per deep-research answer, one system versus the other. The zero-citation system wrote the longer answers, not the shorter ones

Enter the password to read the paper

Ask Foundry Labs for access if you don’t have it. This isn’t a public post: it names two commercial systems directly, so we’re sharing it deliberately rather than publishing it broadly.

AI governance for your practice ↗

More ai research

Name-Swapping and Redaction for Offshore Legal AI

3 Nov 2026

Australia, We Need to Talk About AI

14 Oct 2026

Parliament’s AI Inquiry: What You Missed

13 Oct 2026

Can a Fast Decision Model Check Legal Citations?

September 2026