Images In Pdf Test

Yochai Bankler

University Of Miami School Of Law

Public since June 17, 2026

Authors

Or Cohen-Sasson

Abstract

The legal system prizes consistency—like cases should be treated alike—yet human decision-makers routinely fall short. Enthusiasts, therefore, tout artificial intelligence (AI) tools as "impeccably consistent" arbiters that could purge inconsistency from courts and agencies. This Article challenges that promise. Through 12,000 controlled runs on four top-tier large language models (LLMs) across legal knowledge, research, and analysis tasks, the Article shows that the same AI model, given the very same prompt, still produces divergent answers. Top models achieve only 57% consistency when handling complex legal tasks. Even in low-complexity conditions, models produce divergent answers, on average, one in every fifteen responses. These results expose a striking reality: Far from purging legal inconsistency, AI proves remarkably inconsistent, shattering the promise that machines would deliver what humans cannot. Empirical findings are only the starting point. Beyond measurement lies a deeper question—whether human and AI inconsistencies are a single phenomenon wearing different masks or two distinct phenomena that demand different treatment. The Article develops a normative framework that compares human and AI inconsistencies along three core rule-of-law principles—equality, predictability, and legitimacy. The analysis demonstrates that human and AI inconsistencies differ significantly in their nature and consequently in their impact on these values. The comparison reveals surprising tradeoffs: AI inconsistency poses predictability challenges but proves less discriminatory in its variations; human inconsistency exhibits a mirror image—it better preserves predictability but tends to embed discriminatory patterns. These findings establish an upper bound on responsible legal automation, yet they also reveal opportunity: by treating inconsistency as a design parameter, legal architects can deploy hybrid systems calibrated to institutional priorities.

Full Paper

Download PDF