AI
Beyond the Citation Check
A paired evaluation of two commercial legal AI research tools on six questions of Australian law, and why most of their errors would pass a citation check.
- 76%
- of failures identified could not be caught by confirming a citation exists, is quoted faithfully and supports what it is cited for
- 2
- defects in quoted source text found, one invented and one truncated and presented as complete, both in one system’s standard mode, across 24 outputs and two regenerations
- 19.5 vs 0
- median case citations per deep-research answer, one system versus the other. The zero-citation system wrote the longer answers, not the shorter ones