Files
waggle-os/docs/decisions/2026-04-22-bench-spec-locked.md
Oleg Maslov 0c3e2ead3b
Some checks failed
Installer Smoke / installer-smoke (push) Has been cancelled
moving
2026-09-02 10:10:29 +02:00

280 lines
24 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Bench-Spec LOCK — Stage 2 Mini + Full (H-42a/b)
**Datum:** 2026-04-22
**Sprint:** 11 · Track A · Task A3
**Authority:** PM (Marko Marković) — 7/7 A3 interview zatvoren 2026-04-22 PM via Cowork ratification
**Sources:**
- Sprint 11 backlog: A3 "Benchmark spec LOCK" exit criterion
- A1 ratification `PM-Waggle-OS/decisions/2026-04-22-h-audit-1-design-ratified.md` (scope inheritance §Q1Q5)
- B1 LOCK `PM-Waggle-OS/decisions/2026-04-22-stage-2-primary-config-locked.md` (Stage 2 primary config referenced by §4)
- B2 LOCK `PM-Waggle-OS/decisions/2026-04-22-tie-break-policy-locked.md` (quadri-vendor + PM-escalation referenced by §4)
- B3 LOCK `PM-Waggle-OS/decisions/2026-04-22-model-route-naming-locked.md` (Surface A/B naming referenced throughout)
- Sprint 10 κ=0.7458 baseline (referenced by §4)
**Status:** LOCKED — binding for Stage 2 mini (C3) and Stage 2 full (H-42a/b). Revision allowed only via explicit PM ratification that supersedes this doc.
**Scope:** Benchmark protocol for Stage 2 evaluation of `qwen3.6-35b-a3b-stage2` on LoCoMo, including four-cell mini (C3) and full H-42a/b run with Opus 4.6 probe control arm.
---
## 1. Decision summary
Seven bench-spec axes LOCKED:
1. **Threshold tiering with dual CI.** Point estimate threshold ≥ 91.6% (Mem0 LoCoMo SOTA reper); Wilson score 95% CI (primary) + conversation-level cluster-bootstrap 95% CI (secondary, 10 000 iterations, seed 42). Four verdict tiers: STRONG-PUBLISHABLE / PUBLISHABLE / WEAK / FAIL, defined in §2.
2. **Instance counts and budget envelope.** Stage 2 mini: N=100 per cell × 4 cells = 400 evaluations. Stage 2 full: Qwen N=1540 × 3 runs + Opus probe N=500 × 3 runs = 6120 evaluations (H-42a) + 4620 primary evaluations on Qwen (H-42b dedup frame). Budget ceiling $2600 hard; expected $13002300, defined in §3.
3. **Multiple comparisons via tiered framing.** Mini (C3) is declared exploratory — all metrics reported with descriptive CIs, no family-wise correction, no pass/fail gating of downstream work on a single mini metric. Full (H-42a/b) declares one single primary confirmatory hypothesis H1: Qwen3.6-35B-A3B-Thinking ≥ 91.6% point estimate on LoCoMo with Wilson lower bound ≥ 89.0%. All other reported metrics on full run are secondary descriptive. No Bonferroni / BH correction needed because only H1 is confirmatory. Defined in §5.
4. **Judge ensemble composition and quality monitoring.** Status quo 3+1: Opus 4.7 + GPT-5.4 + Gemini 3.1 primary + xai/grok-4.20 tie-break reserve. Consistency constraint: same physical judge models on mini and full. κ monitoring rules: Fleiss' κ computed per Stage 2 run; <0.65 flag PM review; drop >0.10 from Sprint 10 baseline (κ=0.7458) HALT run. Defined in §4.
5. **Failure mode taxonomy — hybrid F1F6 + F-other.** Six LOCKED categorical failure modes: F1 contradicts-ground-truth, F2 partial-answer, F3 off-topic, F4 refusal, F5 tool-use-error, F6 format-violation, plus null (correct) and F-other (requires mandatory `rationale` field with ≥10-word free text). Judge rubric updated per §6.
6. **Reproducibility manifest — hybrid markdown + YAML with 16 required fields.** Canonical manifest lives as markdown (this doc's §7 embedded template + per-run copy at `PM-Waggle-OS/decisions/<DATE>-stage2-<mini|full>-manifest.md`) with parallel machine-readable YAML twin at same basename `.manifest.yaml`. CI sync guard in §8 freezes drift between the two surfaces. Storage path and versioning protocol defined in §7.
7. **Reasoning_content retention for Stage 2.** Tier 2 of A2 §Q5 applies verbatim to Stage 2 mini and full runs: full JSONL with `reasoning_content` preserved (unpruned) goes to gzipped archive. Bundle layout + tiered access policy + retention horizon defined in §9.
## 2. Threshold tiering and confidence intervals
Primary reference: Mem0 LoCoMo 91.6% single-run SOTA. All Stage 2 numbers compare against this reference.
**Verdict tiers (evaluated on full H-42a/b aggregate across 3 runs):**
- **STRONG-PUBLISHABLE** — Point estimate ≥ 91.6% **and** Wilson 95% lower bound ≥ 91.6% **and** cluster-bootstrap 95% lower bound ≥ 91.6%. Launch-ready claim.
- **PUBLISHABLE** — Point estimate ≥ 91.6% **and** Wilson 95% lower bound ≥ 89.0%. Claim supportable with appropriate CI disclosure.
- **WEAK** — Point estimate in [89.0%, 91.5%]. Do NOT claim SOTA. May publish as "approaching SOTA" with explicit caveats. Triggers PM-review-gate before any external-facing use.
- **FAIL** — Point estimate < 89.0%. Do NOT publish. Triggers post-mortem.
**Why Wilson primary + cluster-bootstrap secondary.** Wilson score interval is the correct frequentist CI for Bernoulli proportion on instance-level binary verdicts; it is tighter than Wald at boundaries and does not require normal approximation. Cluster-bootstrap is the correct non-parametric approach for LoCoMo's structural dependence: each conversation (~300 per cell) contributes multiple instances, violating Wilson's independence assumption. Reporting both hedges against the case where intra-cluster correlation is higher than expected and Wilson would underestimate uncertainty. If Wilson and bootstrap disagree on which tier the run lands in, the more conservative tier prevails.
**Cluster-bootstrap parameters LOCKED:** 10 000 iterations, seed 42 (matches A1 seed convention), cluster unit = conversation_id, resample with replacement at cluster level, compute percentile 2.5/97.5 for CI.
**Per-cell mini (C3) reporting:** Same Wilson + cluster-bootstrap CIs, but no tier assignment. Mini is exploratory; numbers are informational inputs to A3 v2 pre-registration refinement (§5).
## 3. Instance counts and budget envelope
**Stage 2 mini (C3) — exploratory four-cell.** N=100 per cell × 4 cells = 400 evaluations. Cells per B1 LOCK: raw / filtered / compressed / full-context. Cost model: Qwen3.6-35B-A3B-Thinking via OpenRouter bridge ~$0.15/1K output tokens × ~2K output × 400 = ~$120 target; judge triple ~$0.05/instance × 400 = ~$20; tie-break sparse; aggregate target $120200, hard cap $250 per C3 exit-criterion budget.
**Stage 2 full — H-42a/b.** Primary run: Qwen3.6-35B-A3B-Thinking, N=1540 (LoCoMo full), 3 independent seeded runs for stability. Opus 4.6 probe control arm: N=500 (LoCoMo stratified subsample matching H-42b hypothesis), 3 runs. Total target: 4620 Qwen evaluations + 1500 Opus evaluations = 6120 primary evaluations. Judge triple runs on every primary evaluation; tie-break fires on ~25% of evaluations per ensemble-tiebreak module expected split rate.
**Budget envelope LOCKED:**
| Component | Expected | Ceiling |
|---|---|---|
| Qwen3.6-35B-A3B primary (4620 evals) | $6001100 | $1400 |
| Opus 4.6 probe (1500 evals) | $200350 | $450 |
| Judge triple (6120 × 3 judges) | $450750 | $900 |
| Tie-break grok-4.20 (~200 fires) | $515 | $40 |
| Buffer / retries | $4585 | $110 |
| **Total** | **$13002300** | **$2600** |
Hard abort if cumulative spend crosses $2600 at any mid-run checkpoint.
**N=1540 justification.** LoCoMo official eval set has 1540 instances per dataset card. Running full rather than stratified subsample eliminates stratification-bias concerns that would otherwise need to be addressed in publication methods section. Running 3 seeded repeats provides a 4620-evaluation aggregate that gives Wilson 95% half-width of ~0.85pp at p̂=91.6%, which is tight enough to resolve PUBLISHABLE vs WEAK boundary with statistical confidence.
**Opus 4.6 probe (N=500) justification.** Not a full confirmatory run; designed as a control to answer "is H-42b's claim directionally correct — does Opus 4.6 approach or exceed Mem0's 91.6% on LoCoMo under our harness?" N=500 at p̂≈0.90 yields Wilson 95% half-width of ~2.6pp, sufficient to distinguish "meaningfully above Mem0" from "meaningfully below" but not to make a STRONG-PUBLISHABLE claim for Opus. If Opus probe lands in PUBLISHABLE tier, that is a secondary dual-axis narrative input (Marko's multiplier thesis dual-axis framing) but not a launch-blocker.
## 4. Judge ensemble and consistency constraints
**Primary ensemble LOCKED (status quo from Sprint 10 Task 2.2):**
- `anthropic/claude-opus-4-7` (Surface A floating alias per B3 LOCK §1; dated snapshot per Stage 2 run resolved and pinned in that run's manifest per §7).
- `openai/gpt-5.4`
- `google/gemini-3.1`
**Tie-break reserve LOCKED (B2 §1):** `xai/grok-4.20`. Fires on 1-1-1 three-way split per `resolveTieBreak(votes, {path: 'quadri-vendor'})`. 2-2 defensive tie yields `pm-escalation` path (never silent coin-flip).
**Consistency constraint.** The same three physical primary judges + same tie-break must run across mini and full. If any judge model has a provider rotation (floating alias resolves to a new dated snapshot mid-campaign), the rotation is flagged in the manifest (§7 field `judge_dated_snapshots`) and the mini is rerun before accepting the full. This prevents the mini from calibrating to one set of snapshots and the full to another.
**κ monitoring.** Fleiss' κ computed across primary triple on every Stage 2 run (mini and full) over the pre-tie-break vote matrix (i.e., before `resolveTieBreak` fires). Baseline: κ=0.7458 from Sprint 10 Task 2.2 LIVE calibration.
- κ ≥ 0.65 — pass, no flag.
- 0.60 ≤ κ < 0.65 — PASS-WITH-FLAG, run completes, but exit ping must note the drop and PM reviews before advancing to next stage.
- κ < 0.60 OR κ drops >0.10 from Sprint 10 baseline (i.e., κ < 0.6458) — HALT mid-run. Do NOT clean up partial JSONL. Write `sessions/<DATE>-stage2-halted-kappa-drop.md` with captured state and notify PM.
κ drop of this magnitude signals judge-prompt drift or provider-schema drift affecting judge reliability. Investigation precedes any reuse of the harness.
## 5. Multiple comparisons — tiered framing
The multiple-comparisons problem would arise if we treated every reported metric as a separate hypothesis requiring significance. We avoid it by explicitly declaring which metrics are confirmatory vs exploratory/descriptive.
**Mini (C3) declaration.** All mini metrics are **exploratory / descriptive**. No pass/fail gating. No family-wise correction. Output serves two purposes: (a) harness readiness check, (b) input to the A3 v2 pre-registration refinement for the full run. If mini discovers an unexpected failure pattern (e.g., F3 off-topic rate >20% in one cell), that finding informs v2 — it does not constitute a publishable claim.
**Full (H-42a/b) declaration.** Exactly one confirmatory hypothesis:
> **H1 (primary confirmatory):** Qwen3.6-35B-A3B-Thinking achieves ≥ 91.6% point estimate on LoCoMo with Wilson 95% lower bound ≥ 89.0% (PUBLISHABLE tier per §2).
All other full-run numbers (per-cell rates, failure mode distributions, latency quantiles, reasoning-shape distribution, Opus 4.6 probe result, etc.) are **secondary descriptive**. They are reported with appropriate CIs but are not subject to significance testing and do not require multiple-comparisons correction.
**Pre-registration protocol LOCKED — tiered v1/v2/vN+1:**
- **v1** is frozen at A3 LOCK (this doc). v1 manifest = §7 template + parallel YAML. Any run executed against v1 is bound to v1 parameters.
- **v2** may be issued at mini (C3) exit if mini surfaces a material harness or methodology refinement. v2 must explicitly cite what changed vs v1 and why. v2 requires PM ratification in a new decision doc.
- **vN+1** protocol: any subsequent change to manifest parameters after full run kicks off requires HALT of in-flight run, new decision doc, and new manifest hash. No mid-run amendments without HALT-and-restart.
**H-AUDIT-2 integration.** Per A1 ratification §Q3, the harness logger emits `bench.preregistration.manifest_hash` event on run start carrying the SHA-256 of the frozen YAML manifest. This event is the audit anchor: any subsequent claim that a run conformed to v1 must demonstrate that event's hash matches v1 YAML hash at the run's commit.
## 6. Failure mode taxonomy — hybrid F1F6 + F-other
Categorical failure modes LOCKED for all Stage 2 runs:
- **F1 — contradicts-ground-truth.** Model output asserts a fact that directly contradicts the LoCoMo reference answer. Most severe failure class.
- **F2 — partial-answer.** Model output contains correct information but is incomplete against the reference's required components.
- **F3 — off-topic.** Model output is tangentially related or addresses a different question than asked.
- **F4 — refusal.** Model declines to answer (safety response, capability disclaimer, "I don't know").
- **F5 — tool-use-error.** Model attempted a tool call but the harness returned an error, a malformed response, or an infinite loop; applies only in cells where tool use is permitted.
- **F6 — format-violation.** Model output is correct in content but violates the required output format (JSON schema mismatch, wrong key names, escape errors).
Plus:
- **null (correct)** — judge triple majority verdict is "correct" per rubric. No F-code assigned.
- **F-other** — judge identifies a failure that does not fit F1F6. Mandatory `rationale` field with ≥ 10-word free-text explanation. F-other rate on any run > 10% triggers taxonomy review and potential v2 amendment (§5 protocol).
**Judge rubric update.** Rubric prompt includes the F1F6 taxonomy verbatim, with a single-line instruction "If no category fits, select F-other and provide ≥10-word rationale explaining the failure." Rubric path cited in manifest (§7 field `judge_rubric_path`).
**Per-instance output schema (JSONL row extension):**
```json
{
"verdict": "correct" | "incorrect",
"failure_code": null | "F1" | "F2" | "F3" | "F4" | "F5" | "F6" | "F_other",
"failure_rationale": string | null
}
```
`failure_rationale` is non-null iff `failure_code == "F_other"`.
## 7. Reproducibility manifest — hybrid format, 16 required fields
**Canonical surface:** markdown decision doc (human-readable primary). Parallel surface: YAML twin (machine-readable, CI-checkable).
**Per-run manifest path convention:**
- Mini (C3): `PM-Waggle-OS/decisions/2026-XX-XX-stage2-mini-manifest.md` + `.manifest.yaml`
- Full (H-42a/b): `PM-Waggle-OS/decisions/2026-XX-XX-stage2-full-manifest.md` + `.manifest.yaml`
The manifest is emitted once per run kickoff and frozen. Any mid-run change requires HALT per §5 vN+1 protocol.
**16 required fields LOCKED:**
1. `manifest_version` — semver-like, e.g. `v1.0.0` for A3 LOCK v1.
2. `manifest_hash` — SHA-256 of the YAML file content, computed pre-freeze. Emitted as `bench.preregistration.manifest_hash` event on run start.
3. `run_id` — ULID or UUID assigned by harness at kickoff.
4. `run_stage``mini` | `full`.
5. `target_model` — Surface B dated snapshot per B3 LOCK (e.g., `qwen3.6-35b-a3b-stage2-20260422`).
6. `target_model_thinking_mode``on` | `off`. Stage 2 LOCKED `on` per B1.
7. `judge_primary` — array of 3 Surface B dated snapshots for Opus + GPT + Gemini, resolved at run kickoff.
8. `judge_tiebreak` — Surface B dated snapshot for grok-4.20.
9. `judge_rubric_path` — path to judge prompt file (expected under `benchmarks/harness/prompts/`).
10. `dataset``locomo` fixed; `dataset_version` field carries the LoCoMo release hash.
11. `instance_count` — per-cell and aggregate counts.
12. `cells` — array of cell configs (raw / filtered / compressed / full-context) with per-cell parameters.
13. `ci_method` — fixed `wilson_95 + cluster_bootstrap_95` with bootstrap seed 42 and iterations 10 000.
14. `failure_taxonomy_version` — fixed `F1-F6+other v1` (this doc §6).
15. `budget_cap` — hard USD ceiling (§3 values).
16. `retention_policy` — fixed `A2-Q5-tier-2-full-preserved` with pointer to §9.
**Versioning protocol.**
- v1 (frozen at A3 LOCK): this doc + accompanying `2026-04-22-bench-spec-locked.manifest.yaml`.
- v2 issued at mini (C3) exit iff mini surfaces material change. PM ratification required via new decision doc.
- Manifest hash changes on any YAML byte change. The hash is the audit anchor.
## 8. CI sync guard — freeze markdown/YAML drift
**Problem:** Hybrid format risks drift between the human-readable markdown and the machine-readable YAML. Drift silently undermines the audit trail.
**Guard mechanism LOCKED:**
A new CI script `scripts/check-manifest-sync.mjs` (to be added to waggle-os repo; authorization granted below) performs the following on every PR touching `PM-Waggle-OS/decisions/**-manifest.md` or `**.manifest.yaml`:
1. Enumerate all `*-manifest.md` files in `PM-Waggle-OS/decisions/`.
2. For each, require a sibling `*.manifest.yaml`.
3. Parse the markdown, extract the 16 required fields from the structured "Fields" section (convention: a fenced YAML block in the markdown mirrors the YAML file content).
4. Parse the YAML file.
5. Assert byte-level equality of the parsed field set.
6. On any mismatch: fail CI with a diff output showing which fields diverged.
**Authorization for CC-1 to implement:** PM authorizes the script addition as a waggle-os repo change, scoped as a low-priority ticket for inclusion in the Sprint 11 close commit or first Day-3 commit (alongside B3 LOW cleanup items). Budget: $0 (read-only CI check). Validation gates: the script must produce a known-good pass on this doc's v1 manifest pair, and a known-fail on an intentionally divergent test fixture.
**Pre-CI adoption (interim):** Until the script lands, the manifest author (PM for v1) manually verifies synchronization at the time of commit. Commit message should state "Manifest sync verified manually — CI guard pending script landing."
## 9. Reasoning_content retention for Stage 2
**LOCK.** A2 §Q5 Tier 2 retention applies verbatim to Stage 2 mini and Stage 2 full H-42a/b runs.
**What goes to archive.** Full JSONL with `reasoning_content` preserved (unpruned). No runtime-style `includeReasoning: false` filter applied to the archived copy. Storage cost is negligible against audit-trail value; unpruned is LOCKED.
**Bundle layout LOCKED:**
```
waggle-os/benchmarks/archive/2026-XX-XX-stage2-<mini|full>.tar.gz
├── runs/
│ ├── qwen-run-1.jsonl (full reasoning_content preserved)
│ ├── qwen-run-2.jsonl
│ ├── qwen-run-3.jsonl
│ ├── opus-probe-run-1.jsonl (full only; mini omits this)
│ ├── opus-probe-run-2.jsonl
│ └── opus-probe-run-3.jsonl
├── aggregates/
│ ├── qwen-aggregate.json (Wilson + bootstrap CI, F-distributions, κ)
│ └── opus-aggregate.json (full only)
├── manifest.yaml (frozen vN, matches §7 §8)
├── manifest.md (markdown twin)
├── exit-ping.md (from PM-Waggle-OS/sessions/<DATE>-stage2-<mini|full>-exit.md)
├── git-state.txt (commit hash + dirty flag + branch at kickoff)
├── docker-state.txt (`docker images --digests` + `docker ps` snapshots)
└── README.md (1-page index for archive auditor)
```
**Access policy — tiered.**
- **Internal (Egzakta team / repo write access):** open. Direct download and unpack. Low friction for engineering / research iteration.
- **External (regulator / partner audit / due diligence):** PM signoff required. Access log entry recorded in `PM-Waggle-OS/audit-log/<DATE>-<requester>-<purpose>.md`. Chain-of-custody preserved for EU AI Act audit triggers and equivalent regulator requests.
- **Hybrid cases** (e.g., external consultant operating under Egzakta MSA): default to external tier; PM may grant ad-hoc internal-equivalent access with written rationale in the same audit log.
**Retention horizon LOCKED.**
- Minimum 12 months from run completion (inherits A2 §Q5 floor).
- Indefinite while the run supports an active launch claim (H-42a/b backs the SOTA narrative on Waggle/KVARK landing and any external collateral — retention lasts while that claim is live).
- Plus 24 months post-decommissioning of the launch claim. Decommissioning event = explicit PM decision doc superseding the SOTA claim OR product line retirement.
**Cost envelope.** Gzipped full run ~30100MB × 6 runs × ~$0.02/GB/mo S3 standard ≈ ~$0.30/year for the entire set. Negligible against audit-trail value.
## 10. Validation gates
Before C3 (Stage 2 mini) kickoff is authorized to run against this LOCK:
1. Parallel YAML manifest `2026-04-22-bench-spec-locked.manifest.yaml` committed to `PM-Waggle-OS/decisions/` alongside this doc. SHA-256 hash recorded in commit message.
2. `scripts/check-manifest-sync.mjs` spec documented (§8). Implementation may follow; interim manual sync verification is acceptable.
3. H-AUDIT-2 logger emits `bench.preregistration.manifest_hash` event with the v1 hash on first Stage 2 invocation. Spot-verified in mini exit ping.
4. Mini (C3) kickoff brief cites this LOCK by path.
5. Mini exit ping reports: κ per run, Wilson + bootstrap CI per cell, failure-code distribution including F-other rationale sample, manifest hash match, budget actual vs expected.
Before H-42a/b (Stage 2 full) kickoff:
1. Mini (C3) exit completed and PASS-with-or-without-flag reviewed by PM.
2. v2 manifest issued if mini surfaced material change; otherwise v1 carries forward with explicit "v1 carried forward" note in full kickoff brief.
3. Dated snapshots for all 3 primary judges + 1 tie-break + target model re-resolved and pinned at run kickoff time (prevents mini-vs-full snapshot drift per §4 consistency constraint).
4. Budget guard in harness configured to hard-abort at $2600 cumulative.
## 11. Out of scope
- **Provider rotation policy** for floating alias → dated snapshot resolution timing (owned by B3 LOCK §6 + harness maintainer).
- **Judge rubric evolution** beyond the F-taxonomy update in §6. Any substantive rubric change requires separate ratification.
- **Tertiary metrics beyond F1F6** (e.g., fine-grained reasoning-chain analysis) — not blocked by this LOCK but not in scope for H-42a/b primary confirmatory claim.
- **Third-party replication by external researchers** — out of scope; LOCK governs our internal run. If external replication is pursued later, a separate replication protocol doc handles it.
- **Landing copy / marketing narrative derivation** from H-42a/b results — separate PMM decision per multiplier thesis dual-axis framing.
## 12. Related
- `PM-Waggle-OS/decisions/2026-04-22-bench-spec-locked.manifest.yaml` — v1 YAML twin, machine-readable surface per §7.
- `PM-Waggle-OS/decisions/2026-04-22-h-audit-1-design-ratified.md` — A1 ratification (reasoning_content, retention Tier 1/2, turnId plumbing inheritance).
- `PM-Waggle-OS/decisions/2026-04-22-stage-2-primary-config-locked.md` — B1 Stage 2 primary config (thinking mode, cells).
- `PM-Waggle-OS/decisions/2026-04-22-tie-break-policy-locked.md` — B2 quadri-vendor tie-break + PM-escalation defensive path.
- `PM-Waggle-OS/decisions/2026-04-22-model-route-naming-locked.md` — B3 Surface A/B naming convention.
- `PM-Waggle-OS/sessions/2026-04-22-sprint-11-h-audit-1-exit.md` — A2 implementation exit ping.
- `PM-Waggle-OS/sessions/2026-04-22-sprint-11-b1-stage2-config-exit.md` — B1 exit ping.
- `PM-Waggle-OS/sessions/2026-04-22-sprint-11-b2-tiebreak-exit.md` — B2 exit ping.
- `PM-Waggle-OS/sessions/2026-04-22-sprint-11-b3-opus46-audit-exit.md` — B3 audit exit ping.
- `PM-Waggle-OS/briefs/2026-04-22-cc-c2-stage1-mikroeval-kickoff.md` — C2 Stage 1 mikro-eval brief (upstream of C3).
- Sprint 11 master status: `PM-Waggle-OS/sessions/2026-04-23-sprint-11-day-2-am-status.md`.
---
**LOCKED. A3 CLOSED 7/10 → 7/10 Sprint 11 exit criteria. C3 unblocked. H-42a/b cleared for kickoff pending C3 PASS and v1/v2 manifest carry-forward. CC-1 authorized to implement §8 CI sync guard in Sprint 11 close commit or Day-3. Any manifest parameter change post-LOCK requires explicit PM ratification via new decision doc.**