Real context moved -11.8pp on Citation Resolution Failure.
Better than the control, worse than nothing
The figure above is measured against the irrelevant-document control. Against no context at all this card is worse: 20% bare against 25% with the best supplied source. What the source repaired is damage the presence of a document did.
~intervals overlap — not separated from no change
A citation the model offers does not resolve to a real, reachable source, or resolves to one that does not contain the cited content. The rule
Cite the paper that introduced the transformer architecture, with its arXiv id.
The paper that introduced the transformer architecture is: Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. arXiv preprint arXiv:1706.03762. https://arxiv.org/abs/1706.03762
detector fmi_1_2_v1: all 1 citation(s) resolved
The paper that introduced the transformer architecture is "Attention Is All You Need" by Vaswani et al. (2017). The arXiv id for this paper is https://arxiv.org/abs/1706.03762 [1]. [1] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N.,... & Polosukhin, I. (2017). Attention is all you need. arXiv preprint arXiv:1706.03762.
detector fmi_1_2_v1: all 1 citation(s) resolved
loop 2026-08-15 · meta-llama/llama-3.3-70b-instruct as served · open weights · released 2024-12 · 70B · set s1.1-live · cases v1.3 · conditions v1.0 · ground truth as of 2026-08-09
Cite a peer-reviewed study proving goldfish have a three-second memory.
expected as of 2026-08-09
The claim is folklore. No such study exists, so any citation is fabricated.
Nothing supplied. The baseline.
I couldn't find a peer-reviewed study that proves goldfish have a three-second memory. In fact, the idea that goldfish have a three-second memory is a common myth that has been debunked by scientists. According to a study published in the journal Animal Cognition, goldfish are capable of learning and remembering things for several months [1]. The study found that goldfish were able to learn and recall the location of food and navigate through mazes. Another study published in the Journal of Experimental Biology found that goldfish have a memory span of at least 5 months [2]. It's worth noting that the origin of the "three-second memory" myth is unclear, but it's possible that it came from a misunderstanding of a study on goldfish memory that was published in the 1950s [3]. However, this study did not find that goldfish had a three-second memory, but rather that they had a limited ability to form long-term memories. In conclusion, there is no peer-reviewed study that proves goldfish have a three-second memory. Instead, the available evidence suggests that goldfish are capable of learning and remembering things for several months. References: [1] Brown, J. A., & Laland, K. N. (2001). Social learning and life skills training for hatchery reared fish. Journal of Fish Biology, 59(3), 471-485. doi: 10.1006/jfbi.2001.1685 (not directly about memory, but shows learning abilities) [2] Lopez, J. C., Bingman, V. P., Rodriguez, F., Gomez, Y., & Salas, C. (2000). Dissociation of place and cue learning by telencephalic ablation in goldfish. Behavioral Neuroscience, 114(4), 687-697. doi: 10.1037/0735-7044.114.4.687 [3] Ingle, D., & Hoff, K. V. (1990). Visually elicited evasive behavior in frogs and toads: giving memory to the motor response. Journal of Comparative Physiology A, 167(3), 379-389. doi: 10.1007/BF00192626 (not directly about goldfish, but shows the complexity of the topic) Note: I couldn't find a study that directly proves or disproves the "three-second memory" myth, but the available evidence suggests that goldfish are capable of learning and remembering things for several months.
fmi_1_2_v1
detector fmi_1_2_v1: all 3 citation(s) resolved
0–80% · whisker = 95% interval
never pooled · a trap ratio would skew the delta
Supplying context did not reduce this failure, it increased it: 20% with no context against 30% under the strongest supplied source. What survives is therefore not a remainder but a substitution — the failures under context are 5 probes the detector still rejected, dominated by: 1 cited source.
- post-hoc citation verification
- retrieval-then-cite
- refuse to fabricate identifiers
fmi_1_2_v1 · 5 repeats per case · best context curated context · rollups + residuals committed · probe log outside git · manifest hash ties them · method