Hallucinations in large language models (LLMs) are hard to measure because reality is messy. Facts can be ambiguous, sources can disagree, and the world itself can change over time.
So we built our own world.
We introduce HalluWorld, a controlled benchmark for studying language model hallucinations. Instead of evaluating against static reference answers or relying on another LLM to judge whether an answer is true, HalluWorld places models inside environments where the true state of the world is fully specified. This lets us control exactly what a model can observe, systematically test different causes of hallucinations, and automatically determine whether its claims are correct.

HalluWorld spans three settings: configurable grid worlds, chess, and realistic terminal environments. Across these worlds, we probe several different capabilities whose failures can lead to hallucination, such as directly perceiving the current state, remembering information that is no longer visible, reasoning about how the world will change, and recognizing when there simply is not enough information to answer.
GPT-6 Astra ranks #1 on our hallucination benchmark. However, a model can excel at avoiding some types of hallucination while still struggling with others. Even the best models make surprisingly systematic errors, for instance in probes involving uncertainty.
Frontier models perform well on perceptual tasks that require identifying information explicitly available in their inputs or context, without relying on memory or simulation. In our grid worlds, seven of twelve models achieve zero perceptual error, and five achieve zero error on comparable chess probes. However, performance drops sharply when models must maintain and update an internal representation of the world. Every model hallucinates on at least 16.7% of our grid-world memory probes, and causal simulation remains challenging across domains.

More reasoning doesn’t necessarily fix the problem. Extended thinking can help models carefully parse what they observe, but we find that it does not consistently improve causal reasoning. In some settings, giving a model more time to think actually increases hallucination: once its internal simulation of the world goes wrong, additional reasoning can continue coherently from the wrong state.
Models also have a surprisingly hard time saying “I don’t know.” When the available observations are insufficient to determine the answer, models often make a claim anyway. No model hallucinates less than 24% of the time on our grid-world uncertainty probes, and failure to recognize uncertainty remains one of the dominant failure modes in realistic terminal tasks even when other kinds of errors become rare.
These differences matter because two models with similar overall hallucination rates can fail for completely different reasons. One might struggle to remember earlier observations; another might incorrectly simulate what happens next; another might fail to recognize that it lacks enough information to answer.
By constructing controlled reference worlds, HalluWorld lets us move beyond simply asking whether a model hallucinated and instead start asking why.
We hope this provides a more reproducible way to study the mechanisms behind hallucination and, ultimately, to design better ways of reducing it.
HalluWorld was accepted to NeurIPS 2026 (Evaluations & Datasets)
halluworld.ai · arXiv:2605.19341
If you find our work useful, please cite it:
@article{halluworld2026,
title = {HalluWorld: A Controlled Benchmark for Hallucination
via Reference World Models},
author = {Liu, Emmy and Gangal, Varun and Yu, Michael and Tao, Zhuofu
and Singh, Karan and Kumar, Sachin and Feng, Steven Y.},
journal = {arXiv preprint arXiv:2605.19341},
year = {2026},
eprint = {2605.19341},
archivePrefix = {arXiv},
primaryClass = {cs.CL},
url = {https://arxiv.org/abs/2605.19341}
}