The Hamsa Book · contentsThe proof · Chapter 9
The numbers, and what went wrong
How Hamsa was tested, what came out, and the failures, named first.
The most important number is quick-check precision: of the cases put one click from publishing, how many had the right verdict. A wrong one there is the costliest mistake.
8 of 8quick checks right on the labelled set
7 of 7quick checks right on 88 real posts sent to Hamsa
9 of 9"this footage is from elsewhere" calls right
| Posts | Quick checks right | Fake or not, right | Exact label | Sent to a person |
|---|---|---|---|---|
| Labelled set (40) | 8 / 8 | 31 / 40 | 21 / 40 | 31 / 40 |
| Real posts (88) | 7 / 7 | 57 / 88 | 45 / 88 | 78 / 88 |
Fresh run on 24 October 2026; real posts scored on 25 October 2026.
Reading them honestly
- Most posts go to a person, on purpose. Quick checks are kept for signals that cannot be faked. Everything else arrives drafted, with its evidence.
- Exact labels are lower partly by design. Fact-checkers themselves disagree: Nepal Factcheck usually rates foreign footage passed off as Nepal "misleading", AFP rates it "false". And Hamsa prefers "unverifiable" to a guess.
- Time saved has not been measured with a professional fact-checker. What is measured is that most cases reach the desk in under a minute, with the original footage, sources and a bilingual draft.
What went wrong
- A genuine news clip of a rescuer describing a rescue was judged "not a checkable claim" and never reached a checker. No false post was cleared, but a true one should have reached a person.
- Three early real posts were wrongly merged or dismissed because a TikTok fetching bug made different posts look like the same file. Fixed on 22 October 2026.
- The held-out test set is small (4 posts), which is why the 88 real posts were sent through Hamsa exactly as a reader would.