# Hive Mind — LoCoMo SOTA results index (single source of truth)

> Tracked, in-repo, reproducible. **Headline = 86.49%** (7-lane W4, N=1540, same-judge vs Memori).
> Verify: `node recount.mjs`. Supersedes the non-reproducible 87.66% (see correction note below).

**Headline:** On LoCoMo under Memori's own protocol (gpt-4.1-mini answerer + judge, N=1540),
Hive Mind = **86.49% overall, +4.54pp over Memori 81.95 (z=4.64, p<10⁻⁵)**, leading/tying every
category. Two same-judge head-to-heads anchor it (Memori + Mem0). Fully local substrate.

## Correction note (2026-07-01)
The prior headline **87.66%** (2026-06-11 report) does **not reproduce**. Fresh 7-lane W4 + fresh
judge = **85.19%** on its own archived 2026-06-11 substrate, **86.49%** on the current substrate.
Cause: stale-verdict-replay bug (harness note 2026-06-15) inflated the original judge pass; that
1350-correct judgment set is lost. Adopted 86.49% as canonical (founder decision 2026-07-01).

## The numbers (gpt-4.1-mini answerer+judge, N=1540)

### 7-lane W4 vs Memori — same-judge
| Category | Ours | Memori (pub) | Δ |
|---|--:|--:|--:|
| single-hop | 92.27 | 87.87 | +4.40 |
| multi-hop | 80.50 | 72.70 | +7.80 |
| temporal | 81.62 | 80.37 | +1.25 |
| open-ended | 69.79 | 63.54 | +6.25 |
| **overall** | **86.49** | **81.95** | **+4.54 (z=4.64, p<10⁻⁵)** |

### Mem0 — same-judge, our ruler
Ours 86.49 vs Mem0 73.96 = **+12.53pp** overall; temporal 81.62 vs 50.78 = **+30.84pp**
(write-time dating vs ingestion-time). Mem0 judgments 1139/1540 committed.

### Token efficiency
Prior Config-D knee (−28% tokens for ~−1pp) was measured on an earlier answer set — **re-measure
on the current substrate before re-citing**. Do not carry the old 87.66-anchored Pareto numbers.

## Where everything lives

### Committed + reproducible in THIS monorepo
- `benchmarks/results/locomo-sota-2026-06/` — **this dir**: report + INDEX + `recount.mjs` +
  `data/{answers,judgments}/…N1540.jsonl` (the pinned raw evidence, recount = 1332/1540).
- Substrate code: `packages/hive-mind-core/src/mind/{inprocess-reranker,search,resolve-relative-date,parse-date-window,raw-detail-lane,recall-context}.ts`.

### Reproduction harness (`hive-mind-test/scripts/locomo`)
`40-cell-retrieval-gpt41mini.mjs` (7-lane: PROFILES/DATEWIN/EPISODIC/RAWDETAIL), `41-judge-memori-gpt41mini.mjs`,
`42-report-memori.mjs`. **Always fresh `OUT_TAG`** to avoid stale-verdict replay.

### OSS public (`marolinik/hive-mind`)
Substrate code @ `bc4eba1` (PR #14). Benchmark results being updated 73.1% → 86.49% (this arc).

## Gotchas (cost real time)
- **Stale-verdict replay:** `41-judge` resumes by row count on the OUT_FILE → reusing a judgments
  file replays old verdicts. This inflated 87.66. Use a fresh `OUT_TAG` every run.
- `42-report` tokens/query line is a display bug; real tokens = `context_tokens` in answer rows.
- Mem0 shares `~/.mem0/migrations_qdrant` lock → run per-conv.
- Zep + LangMem stay literature-sourced (founder 2026-06-16).