---
decision_id: 2026-04-27-phase-2-acceptance-gate-PASS
date: 2026-04-27
authority: PM (Marko) — Phase 2 gate ratified PASS WITH SUBSTRATE-NO-REGRESSION CONFIRMED
type: acceptance gate close-out + Phase 3 authorization
predecessors:
  - decisions/2026-04-26-pilot-verdict-FAIL.md
  - decisions/2026-04-26-agent-fix-sprint-plan.md
  - decisions/2026-04-26-phase-1-acceptance-gate-results.md
  - 2026-04-27-phase-2-gate-d3-rule-inspection.md
phase: 2 — Multi-step agent loop unification (3 sub-commits + dual-methodology substrate smoke)
verdict: PASS
---

# Phase 2 Acceptance Gate — Results

**Sprint:** agent-fix sprint (2026-04-26 → ~2026-05-10)
**Phase:** 2 — Multi-step agent loop unification (3 sub-commits)
**Outcome:** ✅ **PASS** — substrate-no-regression empirically confirmed via D3 with v6 exact substring-match rule + N=20 95 % CI band.

---

## Per-criterion results

### Criterion 1 — `tsc --noEmit` strict clean on `packages/agent/` + `benchmarks/harness/`

**Status:** ✅ **PASS**
**Evidence:**
- `packages/agent/`: `tsc --noEmit` exit 0 (verified at Phase 2.3 commit `61743df`).
- `benchmarks/harness/`: `tsc --noEmit` exit 0 (verified at Phase 2.3 commit).
- Both packages pass strict-mode TypeScript checks.

---

### Criterion 2 — Test suites green (315/315 baseline)

**Status:** ✅ **PASS** (zero regression)
**Evidence:** `npx vitest run` on all 10 test suites at Phase 2.3 commit `61743df`:

| Suite | Tests | Phase | Notes |
|---|---|---|---|
| output-normalize | 43 | 1.1 | |
| prompt-shapes | 65 | 1.2 | |
| run-meta | 26 | 1.3 | |
| retrieval-agent-loop | 25 | 2.1 | |
| compose-evolution | 21 | (GEPA) | |
| evolution-orchestrator | 21 | (GEPA) | |
| evolution-gates | 42 | (GEPA) | |
| iterative-optimizer | 37 | (GEPA) | |
| harness/cells | 5 | (existing) | |
| harness/cells-substrate | 30 | (existing, 2 assertions adapted) | |
| **Total** | **315** | — | All green in 1.5s wall |

---

### Criterion 3 — Substrate-no-regression smoke (DUAL METHODOLOGY)

**Status:** ✅ **PASS** (with σ-aware sample-variance correction)
**Evidence:** D3 disambiguation step + v6 exact substring-match rule applied to N=20 smoke records.

#### (a) Trio-strict reproduction smoke

| Field | Value |
|---|---|
| Methodology | Qwen 3.6 35B-A3B subject (DashScope direct, thinking=on, max_tokens=16000) + Opus 4.7 + GPT-5.4 + MiniMax M2.7 trio judges (max_tokens=3000) |
| Sample | N=20 random subset of LoCoMo-1540 (seed=42) |
| **Pass rate (v6 substring-match rule applied)** | **40.0 %** (8/20) |
| v6 N=400 baseline | 33.5 % |
| Pre-registered ±5 pp range | 28-38 % (statistically inappropriate at N=20 — see σ correction below) |
| **σ-aware 95 % CI band at N=20, p=0.335** | **12.4-54.6 %** (σ = √(p(1-p)/n) = 10.6 pp) |
| In σ-aware band | ✅ |
| Cost | $0.252 |

#### (b) Self-judge reproduction smoke

| Field | Value |
|---|---|
| Methodology | Qwen 3.6 35B-A3B as both subject AND judge (Mem0-style Yes/No prompt) |
| Sample | Same N=20 with seed=42 |
| **Pass rate** | **90.0 %** (18/20) |
| v6 N=400 baseline | 74.0 % |
| Pre-registered ±5 pp range | 70-78 % (statistically inappropriate at N=20) |
| **σ-aware 95 % CI band at N=20, p=0.74** | **54.4-93.6 %** (σ = √(0.74×0.26/20) = 9.8 pp) |
| In σ-aware band | ✅ (90 % is just inside upper bound) |
| Cost | $0 (re-used from initial smoke; methodology was already correct) |

#### (c) Methodology bias delta

| Field | Value |
|---|---|
| Smoke delta (b-a) | +50.0 pp |
| v6 baseline | +40.5 pp |
| Pre-registered ±5 pp range | 35.5-45.5 pp |
| σ-aware band at N=20 | ±10 pp (combined std error) |
| In σ-aware band | ✅ |

#### v6 accuracy rule citation (the key D3 finding)

`benchmarks/harness/src/runner.ts:427`:
```typescript
const accuracy = result.failureMode ? 0 : scoreAccuracy(result.text, instance.expected);
```

`benchmarks/harness/src/metrics.ts:57-66`:
```typescript
/** Scores a model output against expected substrings (any-match = full credit). */
export function scoreAccuracy(output: string, expected: string[]): number {
  if (expected.length === 0) return 0;
  const lower = output.toLowerCase();
  for (const exp of expected) {
    if (lower.includes(exp.toLowerCase())) return 1;
  }
  return 0;
}
```

The v6 `accuracy` field is a **case-insensitive substring match** on `expected[]`, NOT judge consensus. Trio judge ensemble produces `judge_verdict` and `failure_mode` (which inform F-mode taxonomy) but does NOT directly determine `accuracy`. This explains why the v6 oracle JSONL has 117 rows with unanimous-correct trio + `failure_mode=null` but `accuracy=0` (Qwen rephrased gold; semantically correct, judges agreed, substring-match failed).

#### σ-aware 95 % CI derivation

For a binary-outcome process with population proportion p and sample size n:

```
σ_p̂ = √(p(1-p)/n)
95% CI ≈ p ± 2σ
```

Derived bands for N=20:

| Reference | p | σ at N=20 | 95% CI |
|---|---|---|---|
| Trio-strict baseline (33.5%) | 0.335 | 10.6 pp | 12.4-54.6 % |
| Self-judge baseline (74.0%) | 0.74 | 9.8 pp | 54.4-93.6 % |

PM's pre-registered ±5 pp range was inherited from N=400 reference run (σ=2.4 pp at p=0.335) without sample-size correction. At N=20, ±5 pp is inappropriately tight — true 95 % CI is roughly ±21 pp at p=0.335.

**Sprint plan Extension 5 (binding from this gate forward):** future acceptance gates pre-register σ-aware ranges. Authoring template:

```
Sample N: <n>
Population proportion (p_baseline): <baseline_pass_rate>
σ_n = √(p_baseline × (1 - p_baseline) / n)
Acceptance band (95% CI): p_baseline ± 2·σ_n
```

---

### Criterion 4 — `cells.ts` scaffold deletion grep

**Status:** ✅ **PASS**
**Evidence:** Phase 2.3 commit `61743df` strict greps:
- `"no sentences, no punctuation, no hedging"` (deleted SYSTEM_EVOLVED literal): absent from `cells.ts`
- `"memory:synth"` (deleted scaffold marker): absent
- `SYSTEM_BASELINE` / `SYSTEM_EVOLVED` (deleted constant names): absent
- `buildUserPromptMemory` (deleted helper): absent
- Preserved: `SYSTEM_AGENTIC` + `SYSTEM_AGENTIC_FORCED_FALLBACK` exports verbatim at lines 100, 138

---

### Criterion 5 — Pilot wrapper consumes `runAgentLoop`

**Status:** ✅ **PASS**
**Evidence:** Phase 2.2 commit `5699677` — `scripts/run-pilot-2026-04-26.ts` imports from `@waggle/agent`:
- `runSoloAgent` (Cell A/C single-shot path)
- `runRetrievalAgentLoop` (Cell B/D multi-step path)
- `LlmCallFn`, `RetrievalSearchFn` types for adapter shapes

Local re-implementations deleted: `runCellSolo` body, `runCellMultiStep` body, `parseAgentAction`, `llmOptsFor`. Wrapper file shrunk from 1035 → 951 lines while preserving all 12 CLI flags + JSONL output schema.

---

## Phase 2 commit chain

| Commit | Phase | Description | Files |
|---|---|---|---|
| `a599a07` | 2.1 | Unified retrieval-augmented agent loop in packages/agent/ | 4 (+1046/-1) |
| `5699677` | 2.2 | Pilot wrapper consumes @waggle/agent (deletes duplicate impls) | 1 (+103/-187) |
| `61743df` | 2.3 | cells.ts refactor + scaffold deprecation + Phase 1.x public-API fix | 3 (+115/-26) |

---

## Brief-authoring failure tally for this sprint (sprint-level audit)

This is the FIFTH class of brief-authoring failure surfaced in the agent-fix sprint:

| # | Phase | Failure class | Resolution |
|---|---|---|---|
| 1 | Pilot 2026-04-26 v1 §1 | Wrong Qwen alias (OR bridge regresses to 3.5) | Amendment v2 §2 binding correction |
| 2 | Phase 1 acceptance gate | Mixed-methodology baseline (trio-strict 33.5% vs self-judge 74% conflated under one "v6 baseline" label) | Option C waiver + dual-methodology spec at Phase 2 gate |
| 3 | Phase 2.3 brief | Scope-discovery failure (PM brief assumed 4 cells; cells.ts has 7) | Option A scope-discovery halt + Extension 3 |
| 4 | Phase 2.3 brief | Cell semantics preservation didn't include prompt strictness equivalence | Extension 4 in feedback memory |
| 5 | Phase 2 acceptance gate | σ-aware ranges not pre-registered (±5 pp at N=20 inappropriate; true σ=10.6 pp at p=0.335) | Extension 5: σ-aware ranges binding for future gates |

All 5 are **same root pattern** — brief authoring inherits config / parameters / ranges from a different context without verifying applicability. Extensions 1-5 in `feedback_config_inheritance_audit.md` codify the rule: **read the source-of-record + verify applicability before authoring.** Both PM (brief authoring) and CC-1 (script authoring) bound by this rule going forward.

---

## Audit chain

```
amendment_v2_doc_sha256        = 1ab5082ff773538a26b3c3294f7fbee4e30063a8d994bdb3753bdc9dd6d6cd99
amendment_v1_doc_sha256        = 3946d3e00fbb1996fb7e63096ecef51abf1e209e5ff166fd0d8758e9a3a14aad
cc1_brief_sha256               = 9805adae478333178d36d71b88795afc37f8fb543c2ebccaecb7b01faf06afee
v6_manifest_yaml_sha256        = 5d5c1023421cd1a79f4913bb4c0a59415e21f50797255bff7dfec8e16b68e3ed
phase_1_gate_doc               = decisions/2026-04-26-phase-1-acceptance-gate-results.md
phase_2_gate_d3_doc_sha256     = 520582283ae91d3d1c921cb44a7ce232fcedcdc763b780511ead00913f5fa97d
phase_2_head_sha               = 61743df (Phase 2.3 close)
sprint_plan_doc                = decisions/2026-04-26-agent-fix-sprint-plan.md
v6_accuracy_rule_source        = benchmarks/harness/src/runner.ts:427 + metrics.ts:57
v6_oracle_jsonl_path           = benchmarks/results/raw-locomo-2026-04-24T21-49-17-592Z.jsonl
phase_2_smoke_records          = benchmarks/results/phase-2-acceptance-gate/smoke-records.jsonl
phase_2_rejudge_records        = benchmarks/results/phase-2-acceptance-gate/rejudge-records.jsonl
```

## Cumulative cost

| Item | Cost |
|---|---|
| Initial smoke (Phase 2 gate run #1) | $0.195 |
| Re-judge with F-mode taxonomy (Path A) | $0.252 |
| D3 inspection (analytical only) | $0 |
| **Phase 2 gate cumulative** | **$0.447** |
| Cap | $2.50 |
| Remaining for future gates / Phase 5 re-pilot | $2.05 |

---

## Phase 3 authorization

PM has authorized Phase 3 (long-task persistence) per sprint plan §"Phase 3 — Long-task persistence (2-3 days)". Commit boundaries (per PM kickoff):

- **Commit 3.1**: `packages/agent/src/long-task/checkpoint.ts` + tests; halt + PM review
- **Commit 3.2**: `packages/agent/src/long-task/recovery.ts` + tests; halt + PM review
- **Commit 3.3**: `packages/agent/src/long-task/context-manager.ts` + tests; halt + PM review
- **Commit 3.4**: `packages/agent/src/agent-loop.ts` integration + tests; halt + PM review
- **Phase 3 acceptance gate**: replay-determinism test (process kill mid-step → resume → identical final output) + 315+ existing tests still green

---

**End of Phase 2 acceptance gate results. Phase 3.1 authorized. Standing GREEN.**