waggle-os/docs/decisions/2026-04-24-pm-ratify-v6-kappa.md

LOCKED — PM-RATIFY-V6-KAPPA PASS ACCEPT, Phase 2 Authorization

Date: 2026-04-24 Ratified by: Marko Marković (2026-04-24 evening, pending Phase 2 GO) PM: claude-opus-4-7 (Cowork) Predecessor: Phase 1 (v6 emission + config amendment + κ re-calibration) complete; PASS verdict

Odluka

κ PASS ACCEPT + Phase 2 N=400 authorization. Conservative trio κ=0.7878 exceeds 0.70 substantial threshold with 11.1% margin. Operational metrics exemplary (MiniMax 100/100 parse, p50 11.9s, 0 routing errors, $0.075 spent vs $30 cap). Agentic cell κ(GPT, MiniMax)=0.6875 flagged as non-blocking dijagnostika.

Empirijski rezultati

κ matrica (100-instance re-calibration)

Pair	κ	Agreement
Opus × GPT	0.8480	93/100
Opus × MiniMax	0.8549	93/100
GPT × MiniMax	0.7878	90/100
Conservative trio	0.7878	min

Per-cell κ breakdown

Cell	Trio κ	Status
no-context	1.0000	perfect
retrieval	0.8936	excellent
oracle-context	0.7059	substantial
full-context	0.7000	substantial
agentic	0.6875	moderate (PM-flag)

MiniMax operational

• Parse: 100/100 (no failures)
• Latency p50: 11.9s, p95: 31.4s
• Routing errors: 0/100
• Retries: 0
• Tokens: 53,855 prompt + 48,920 completion

Ključni nalazi

Finding 1 — Swap je upgrade, ne kompromis. κ(Opus, MiniMax) = 0.8549 > κ(Opus, GPT) = 0.8480. MiniMax se slaže sa Opus-om više nego GPT se slaže sa Opus-om. Nova ensemble je methodologically jačih veza ka anchor judgment-u nego originalni v5 trio. Ovo otklanja prethodnu brigu u PM correctness re-analysis memo-u da je MiniMax "correctness-first compromise over diversity-first" — empirijski, MiniMax je oba (correctness-aligned I drži κ well above threshold).

Finding 2 — Historical consistency preserved. Opus-GPT pairwise κ=0.8480 konzistentan sa očekivanom matematikom iz v5 three-way Fleiss κ=0.7458 (pairwise typically higher than multiway). Nema drift-a u postojećem judge paru. Baseline validity preserved.

Finding 3 — Agentic cell edge. Jedan sub-0.70 pair u celom matriksu (GPT × MiniMax na agentic cell-u = 0.6875). Ostali četiri cells (no-context, retrieval, oracle-context, full-context) svi ≥ 0.70. Primary hypothesis H1 (retrieval > no-context) pokriven najvišim κ vrednostima — potpuno powered.

Obrazloženje ne-blockiranja Phase 2 zbog agentic cell-a

Tri razloga:

1. Aggregate trio κ je autoritativni gating criterion per v6 §6 methodology. Per-cell κ je dijagnostika, ne pre-registration gate. Aggregate 0.7878 PASS je definitivno.

2. Primary hypothesis nije ugrožen. H1 test koristi retrieval (κ=0.8936) i no-context (κ=1.000) cells. Agentic i drugi sekundarni cells su descriptive u output-u, ne hypothesis-testing.

3. Agentic historical difficulty. Ovaj cell je najteži za sve judges u LoCoMo-alike dataset-ovima (spektralno fragmentisan scoring pattern). 0.6875 je "moderate agreement" po Landis-Koch — informative, ne random. Post-analysis će documentovati agentic findings sa explicit caveat.

Autorizacija Phase 2

GATE-D-REKICK-GO authorized. Phase 2 brief emitovan: briefs/2026-04-24-cc1-manifest-v6-phase2-n400-execution-brief.md.

Scope:

• Pre-flight: MiniMax + Kimi cold checks (3+3 calls, $0.10, 5 min)
• N=400 execution sa novim trio (Opus+GPT+MiniMax primary, Kimi per-instance backup)
• Fisher one-sided primary hypothesis test (H1: retrieval > no-context, p < 0.10 threshold)
• Deliverables: results JSONL, analysis MD, operational report, memo
• Halt on PM-RATIFY-V6-N400-COMPLETE

Budget:

• Phase 2 cap: $30 (pre-flight $0.10 + N=400 ~$25-28)
• Phase 1 spent: $0.075 — Phase 2 + Phase 1 total ~$25-28 << originalni v5 envelope $30

Wall-clock projection:

• Pre-flight: 5 min
• N=400 execution: 90-150 min
• Analysis + commits: 20 min
• Total: 2-3h

Parent commit chain

fc16925  v5 anchor (superseded)
ad324cc  Step 2 rpm:20
3a146ef  §1.3c probe v2
e5696f4  Fold-in 3.5a
d0ab680  Fold-in 3.5b
1d3851d  §1.3e RPD memo
8ad0567  §1.3f Vertex Batch INFEASIBLE
8a2f0e6  §1.3g 4-candidate MULTI_PASS
ae0d312  §1.3h stratified re-probe
005a19a  §1.3h-C DeepSeek mt=2048
60d061e  v6 manifest emission (Phase 1 Commit 1)
38a830e  litellm-config amendment under v6 (Phase 1 Commit 2)
01f7ead  κ re-cal artefacts (Phase 1 Commit 3)

13 commits od v4 dedd698. HEAD = 01f7ead. Phase 2 execution se grana od 01f7ead.

Task #29 trace

• §2.0 v6 emission ✓ (Commit 1: 60d061e)
• §2.1 κ re-calibration ✓ (Commit 3: 01f7ead, κ=0.7878 PASS)
• §2.2 N=400 execution: AUTHORIZED, brief emitted
• PM-RATIFY-V6-N400-COMPLETE: pending Phase 2 completion
• Gate D exit: pending PM-RATIFY-V6-N400-COMPLETE

Odbacivanja (considered but rejected)

• Phase 2 block zbog agentic cell-a: aggregate κ je authoritative gating criterion; per-cell flag je diagnostic ne gate.
• Rekalibracija κ na split-oversampled sample: bilo bi skuplje i odlaže Phase 2 bez material benefit-a za H1 test validnost.
• Kimi promoted to primary: MiniMax empirijski superior na κ (0.8549 vs Kimi untested at this scale); backup role adekvatna.

How to apply

Post-SOTA claim audit trail: v6 Phase 1 results demonstrate that:

1. Chinese GA-status reasoning flagship (MiniMax M2.7) can match or exceed US provider (Google preview) on κ agreement with anchor judge
2. Correctness-oriented judge selection outperforms diversity-first selection on this task class
3. κ re-calibration on supersession is standard audit-trail practice, not optional