oleg.m/waggle-os

Fork 0

Files

Oleg Maslov 0c3e2ead3b

Installer Smoke / installer-smoke (push) Has been cancelled

Details

moving

2026-09-02 10:10:29 +02:00

10 KiB

Raw Permalink Blame History

Sprint 11 Scope — LOCKED (Pre-flight Readiness Sprint)

Datum: 2026-04-22 Ratified by: Marko (2026-04-22, post Sprint 10 full-close + PR #2 ready) Author: PM Supersedes: strategy/2026-04-22-sprint-11-kickoff-memo.md (DRAFT, judge-methodology axis framing) Status: LOCKED — CC-1 kickoff brief issued under this scope

1. Scope statement

Sprint 11 je Pre-flight Readiness Sprint. Jedini exit kriterijum je green-light autorizacija za H-42a/b LoCoMo benchmark execution ($1500-2600 envelope, documented u project_preflight_gate.md). Sprint 11 ne izvršava H-42a/b. Izvršenje benchmarka je Sprint 12 scope, condicionovano Sprint 11 full-close verdiktom.

Readiness se definiše kao zbir tri track-a — svaki mora CLOSE da bi green-light bio legitiman. Delimični close = no-go, bez izuzetaka.

2. Track struktura

Track A — Audit-grade traceability (H-AUDIT-1 + H-AUDIT-2)

A1 — H-AUDIT-1 design doc (gate ispred implementation-a). CC-1 isporučuje 1-page design doc za turnId propagaciju (UUID v4 generisan na orchestrator entry, threaded kroz cognify.ts, tools.ts, combined-retrieval.ts, prompt-assembler.ts, agent-loop), wired u existing trace store u chat.ts. Design doc mora pokrivati: generation point, propagacioni surface (grep target ≥6 hits za turnId), persistence format, test scenario (reconstruct full turn graph from single turnId). PM ratifikaciona gate-a pre implementation start-a.

A2 — H-AUDIT-1 implementation. Posle A1 ratifikacije. Acceptance: grep -n "turnId" packages/**/*.ts vraća ≥6 hits u 5+ različitih fajlova; unit test reconstruct-uje full turn graph iz single turnId-a; zero regresija na postojećim suite-ovima; tsc clean.

A3 — H-AUDIT-2 bench-spec resolution. 30-min call PM + Marko oko odluke "wall-clock vs recall correctness only" (per project_audit_findings.md Must-Fix pre Track 2). Ishod dictatje Cognify O(E²) timing metodologiju za Stage 2. Deliverable: decisions/2026-04-XX-bench-spec-wall-clock-resolution.md.

Track B — Methodology locks (Stage 2 config + bolt-on A + bolt-on C + bolt-on D)

B1 — Stage 2 Qwen config ratifikacija. PM dostavlja strategy/2026-04-22-stage-2-qwen-config-ratification-memo.md sa 5 safe configs iz Task 1.1 live run-a side-by-side, tradeoff tabelom, i PM preporukom (preliminary: thinking=off, max_tokens=16000). Marko ratifikuje; decision dokument se LOCK-uje kao decisions/2026-04-XX-stage-2-primary-config-locked.md.

B2 — Tie-break policy lock (bolt-on A iz deprecated memo-a). Preneseno iz superseded kickoff memo-a. PM preporuka ostaje Opcija 3 (Sonnet 4.6 kao fourth vendor) sa Opcija 2 fallback (PM escalation na 1-1-2 quadri-vendor split). CC-1 implementira u packages/server/src/benchmarks/judge/ensemble-tiebreak.ts + 4 unit testa (1-1-1, 1-1-2, 2-1-1, 3-0). Decision dokument: decisions/2026-04-XX-tie-break-policy-locked.md.

B3 — Opus 4.6 route audit (bolt-on C iz deprecated memo-a). Preneseno. CC-1 grep-uje packages/server/** i packages/cli/** za Opus model string reference, klasifikuje dated snapshot vs floating alias, migrira (b) i (c) reference na dated ako treba, LOCK-uje naming konvenciju u docs/BENCHMARK-INFRASTRUCTURE.md. Deliverable: docs/reports/opus-4-6-route-audit-2026-04-XX.md + decisions/2026-04-XX-model-route-naming-locked.md.

B4 — Stage 2 kickoff memo framing (bolt-on D iz deprecated memo-a). Preneseno. PM draft Stage 2 kickoff memo sa 4-nedeljnim planom (Week 1 Qwen3.6 × LoCoMo, Week 2 Gemma probe, Week 3 three-model comparison, Week 4 failure mode analysis + publishable results draft). Deliverable: strategy/2026-04-XX-stage-2-kickoff-memo.md. Ne locks Stage 2 execution autonomy — postavlja scope + exit criteria, ne izmenjuje pre-registered LoCoMo thresholds.

Track C — Pre-flight gate execution (Stage 0 → Stage 1 → Stage 2 4-cell mini)

C1 — Stage 0 verifikacija. Stage 0 je već CLOSED per memory; CC-1 verifikuje state, link artifacts, potvrđuje da nije regredirao. Ako regredirao → re-run ~$5.

C2 — Stage 1 mikro-eval. Per project_preflight_gate.md, Stage 1 $5-10, kratka eval baseline. Izvršava CC-1 posle B1 (config ratifikovan) + B2 (tie-break LOCKED) + A2 (turnId landed).

C3 — Stage 2 4-cell mini. 50 pitanja, fiksni seed=42, four-cell ablation (raw / memory-only / evolve-only / full-stack), Sonnet judge. Pass kriterijum: full-stack ≥85% recall + ordinal consistency (full ≥ memory, full ≥ evolve, full > raw sa delta ≥10pp, bar jedan layer > raw). Budget $67-134. Re-run policy: max 3 attempts, sample curation samo za legitimate scope gap ≤5 pitanja.

3. Explicit deferral: bolt-on B (judge prompt refinement trigger matrix)

Bolt-on B iz superseded memo-a (Instance 10 temporal-qualifier trigger matrix) premešten u Sprint 12. Razlog: Instance 10 je single case od 14; trigger matrix zahteva ≥3 instances da bi prompt refinement bio triggered. Ova osa ne blokuje H-42a/b benchmark — interpretivni disagreement logging postoji kroz postojeću ensemble observability. Sprint 12 pokreće ovaj rad ako Stage 2 main run donese dodatne interpretivni divergence signale.

4. Sprint 11 exit criteria (hard gates)

Sprint 11 CLOSES kad svih 10 od sledećih:

A1 CLOSED — H-AUDIT-1 design doc ratifikovan
A2 CLOSED — turnId landed, grep ≥6 hits, tests green
A3 CLOSED — bench-spec decision LOCKED
B1 CLOSED — Stage 2 config LOCKED
B2 CLOSED — tie-break policy LOCKED + implementovana + tests green
B3 CLOSED — Opus 4.6 route audit report + naming decision LOCKED
B4 CLOSED — Stage 2 kickoff memo draft ratifikovan
C1 CLOSED — Stage 0 verifikacija
C2 CLOSED — Stage 1 mikro-eval PASS
C3 CLOSED — Stage 2 4-cell mini PASS sa ordinal consistency

Sprint 11 close-out report + origin/main push.

Green-light verdikt za H-42a/b izdaje PM kad i samo kad svih 10 CLOSED.

5. Sprint 11 budget envelope

Track	Line	Budget
A	A1 design doc	$0 (markdown)
A	A2 implementation	≤$0.10 (unit tests)
A	A3 bench-spec call	$0 (call + markdown)
B	B1 config memo	$0 (PM markdown)
B	B2 tie-break impl + tests	≤$0.20
B	B3 route audit	≤$0.10
B	B4 kickoff memo draft	$0
C	C1 Stage 0 verify	$0 (verify only) ili $5 (re-run ako regredirao)
C	C2 Stage 1 mikro-eval	$5-10
C	C3 Stage 2 4-cell mini	$67-134

Sprint 11 ceiling: ~$150 (absorbs Stage 0 re-run ako triggered). Sprint 10+11 kumulativno projected: $2.28 + $150 = ~$152, unutar ukupnog benchmark prep budget envelope-a pre Stage 2 full-run.

6. Sprint 11 duration estimate

Track	Estimate
A1 + A2 + A3	1-1.5 dana (design doc + implementation + call + markdown)
B1 + B2 + B3 + B4	1-1.5 dana CC-1 + ~4h PM iteration
C1 + C2 + C3	1-1.5 dana (C3 je najteže wall-clock stavka)

Ukupno: 4-5 radnih dana realno. Parallelizovano gde tracks ne dele resurse. Track C3 je rep-ograničenje jer čeka A2 (turnId), B1 (config), B2 (tie-break).

7. Sekvenciranje i blocking graf

A1 design doc → PM ratify → A2 implementation ─┐
                                                │
B1 config memo → Marko ratify ─────────────────┼→ C2 Stage 1 mikro-eval
                                                │
B2 tie-break impl ─────────────────────────────┘
                                                         │
                                                         ↓
                                                   C3 Stage 2 4-cell mini
                                                         │
                                                         ↓
                                              Sprint 11 full-close
                                                         │
                                                         ↓
                                               Green-light H-42a/b

A3 bench-spec ──┐
                ├→ Sprint 11 exit gate (informational, ne blokira C3)
B3 route audit ─┤
B4 kickoff memo ┘

A3, B3, B4 mogu da se izvrše paralelno sa bilo kojim tackom, ne blokiraju C3.

C1 se izvrši kao prvi krak Track C, bez ikakvih blocker-a.

8. Anti-pattern lock

Ispod stoje ograničenja koja Sprint 11 poštuje kroz sve track-ove:

No post-hoc ground truth reformulation (Workflow Reality Check anti-pattern #4) — Instance 9 Option C već LOCKED, Instance 10 interpretivni case ne trigger-uje GT izmenu.
No retry-and-hope policy — C3 max 3 attempts, sample curation samo na legitimate scope gap ≤5 pitanja, dokumentovano.
No scope creep u Sprint 11 — bilo koja nova stavka zahteva explicit PM ratifikaciju novim decision dokumentom.
No H-42a/b execution u Sprint 11 — benchmark izvršenje je Sprint 12 scope. Sprint 11 stane na green-light verdikt.

9. Rollback klauzule

Ako A2 ne CLOSE u 1.5 dana → PM eskalira, razmatra scope cut (H-AUDIT-2 defer ka Sprint 12), ali A1+A2 MORA landati pre green-light-a.
Ako C3 faila 3 attempt-a → HARD STOP, Sprint 11 ne zatvara se. PM review utvrđuje da li je gap systemic (engine problem) ili sample-specific (legitimate scope gap ≤5 pitanja) pre bilo kakvog remediation plana.
Ako B1 Stage 2 config predlog ne prođe Marko ratifikaciju → PM iteracija memo-a sa novim preporukom; C2/C3 čekaju.

strategy/2026-04-22-sprint-11-kickoff-memo.md — SUPERSEDED by ovom odlukom; gets header notice, ostaje u repu kao audit trail
strategy/2026-04-22-stage-2-qwen-config-ratification-memo.md — B1 deliverable (paired with ovom odlukom)
briefs/2026-04-22-cc-sprint-11-kickoff.md — CC-1 execution brief (paired)
sessions/2026-04-22-sprint-10-task-1-1-exit.md — Task 1.1 PASS verdict i safe config pool
.auto-memory/project_preflight_gate.md — 3-stage pre-flight gate struktura
.auto-memory/project_audit_findings.md — H-AUDIT-1 + H-AUDIT-2 Must-Fix
.auto-memory/project_h_audit_1_not_implemented.md — turnId je implementation, ne verification
.auto-memory/project_sota_benchmark_governance.md — governance framework za green-light kriterijume
.auto-memory/project_locked_2026_04_20_benchmark_gemma_cc.md — 7 obligations framework

Sprint 11 scope LOCKED. CC-1 kickoff brief ratified under ovim scope-om. Sprint 11 kick-off autorizovan čim Sprint 10 full-close push landuje.

10 KiB Raw Permalink Blame History Unescape Escape