About

About KahneBench

A comprehensive cognitive bias benchmark for evaluating Large Language Models, grounded in Nobel Prize-winning research.

Our Mission

As Large Language Models transition from experimental tools to integrated components of high-stakes workflows—from healthcare to legal analysis—the imperative to understand and mitigate their cognitive vulnerabilities has never been more critical.

KahneBench provides the tools to not only diagnose cognitive biases in AI systems but to quantify their magnitude, consistency, and potential for mitigation. Our goal is to build more trustworthy and rational AI agents.

Key Contributions

Bias Taxonomy for AI

The first systematic classification of which human cognitive biases transfer to LLMs and which do not, creating a foundational taxonomy for AI psychology.

Mechanistic Understanding

Insights into the underlying mechanisms that give rise to cognitive patterns by correlating bias presence with model architecture, scale, and training data.

Debiasing Strategies

An empirical testbed for validating the effectiveness of various mitigation techniques, from simple prompting strategies to complex interventions.

Model-Specific Profiles

Unique "cognitive fingerprints" for different LLMs, highlighting their specific strengths and weaknesses to guide responsible deployment.

Theoretical Foundation

KahneBench is uniquely grounded in the foundational "two-system" view of cognition articulated by Nobel laureate Daniel Kahneman and his collaborator Amos Tversky. This dual-process theory distinguishes between:

System 1: Fast, automatic, and intuitive operations
System 2: Slow, serial, and deliberately controlled operations

Human judgment often relies on the heuristics of System 1, which, while efficient, can lead to predictable errors or "cognitive biases." KahneBench measures LLM susceptibility to these same cognitive illusions and their capacity for deliberate, corrective thought.

Academic References

Thinking, Fast and Slow

Kahneman, D. (2011). Thinking, Fast and Slow. Farrar, Straus and Giroux.

Judgment Under Uncertainty

Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185(4157), 1124-1131.

Prospect Theory

Kahneman, D., & Tversky, A. (1979). Prospect theory: An analysis of decision under risk. Econometrica, 47(2), 263-291.

Resources

GitHub Repository Documentation

Get Started

Ready to evaluate your models for cognitive biases? Check out our documentation to get started with KahneBench.

View Getting Started Guide