# CC-2 Faza 1 — Amendment 2 (Phase 4.5 retrieval-engagement signal incorporation)

**Date:** 2026-04-28
**Author:** PM
**Status:** RATIFIED, supplements Amendment 1, binding upon paste-into-CC-2
**Predecessor:** `briefs/2026-04-28-cc4-faza1-amendment-1.md`
**Trigger:** Phase 4.5 tools audit (CC-1 commit reference: decision memo `decisions/2026-04-28-phase-4-5-tools-audit-results.md`) surfaced empirical mechanistic signal not visible at Amendment 1 authoring time

---

## §1 — Why this amendment exists

Phase 4.5 produced an empirical, pilot-anchored mechanistic finding that **directly changes GEPA Faza 1 fitness function design**. Without incorporation pre corpus-generation, GEPA risks evolving prompt-shapes that improve trio-judge scores via surface-level mutation while leaving the underlying behavioral gap untouched. That would produce false-positive Faza 1 PASS → Phase 5 GEPA-evolved FAIL — the worst possible outcome (we'd waste Faza 2 expansion + Phase 5 budget on shapes that don't actually rescue H4).

Amendment 2 incorporates the signal as a binding fitness function modification before any GEPA evolution happens.

---

## §2 — The empirical signal (Phase 4.5 §"Pilot retrieval engagement empirical signal")

| Cell | Model | retrieval_calls | steps | loop_exhausted | trio_mean |
|---|---|---|---|---|---|
| Task 1 / B | Opus | 2 | 3 | false | 4.94 |
| Task 1 / D | Qwen | 1 | 2 | false | 4.39 |
| Task 2 / B | Opus | 2 | 3 | **true** | 5.00 |
| Task 2 / D | Qwen | 1 | 2 | false | 3.94 |
| Task 3 / B | Opus | 3 | 4 | **true** | 4.89 |
| Task 3 / D | Qwen | 2 | 4 | false | 4.56 |
| **Mean** | **Opus 2.33 / Qwen 1.33** | | **Opus 67% exhausts** | **Δ=−0.65** |

Three observations from Phase 4.5:

1. Qwen retrieves ~half as often as Opus (1.33 avg vs 2.33 avg) on byte-identical tool surface
2. Opus exhausts maxSteps in 2 of 3 retrieval runs (loop_exhausted=true) — wants more retrievals than 5-turn budget
3. Qwen retrieval scores LOWER than Opus retrieval on every task (Δ mean −0.65)

The H4 score gap mechanistically traces (at least partially) to under-engagement with retrieval, not to format issues — Phase 4.5 verified MULTI_STEP_ACTION_CONTRACT renders identically across all 5 prompt shapes.

**Implication for GEPA:** the failure mode is "Qwen finalizes prematurely with insufficient evidence base." The mutation surface (prompt-shape body) IS where this can be addressed — by evolving instruction phrasing that triggers more retrieval iterations / discourages early finalization on Qwen-targeted shapes.

---

## §3 — Fitness function update (binding)

Brief §3.1 originally specified:
> Fitness = trio-strict accuracy − cost penalty (−0.5pp per $0.10 cost above baseline median)

**UPDATED for Faza 1:**

```
Per-shape fitness function:

For Qwen-targeted shapes (qwen-thinking, qwen-non-thinking):
  fitness = trio_strict_pass_rate
          + retrieval_engagement_bonus
          - cost_penalty

  where retrieval_engagement_bonus = 
    +0.05 (5pp)  if mean retrieval_calls per task ≥ 2.0  (Opus parity proxy)
     0.00        if mean retrieval_calls per task in [1.5, 2.0)
    -0.05 (5pp)  if mean retrieval_calls per task < 1.5  (Qwen baseline behavior penalty)

For non-Qwen shapes (claude, gpt, generic-simple):
  fitness = trio_strict_pass_rate − cost_penalty
  (retrieval engagement signal not weighted; these shapes don't have the gap)

Cost penalty unchanged: −0.5pp per $0.10 cost above per-shape baseline median.
```

**Rationale for Qwen-only weighting:** Phase 4.5 finding is Qwen-specific. Opus shape does NOT have the gap (loop_exhausted=true means Opus engages retrieval aggressively). Applying retrieval-engagement bonus uniformly across all shapes would distort fitness for shapes that don't have the underlying behavioral problem.

**Rationale for ±5pp band:** matches the brief §4 condition 1 "+5pp threshold" — keeps signal magnitudes consistent. Wider band would dominate trio-strict signal; narrower would be noise-floor.

**Rationale for 2.0 threshold:** Opus mean 2.33 is the parity target; 2.0 is a slightly relaxed target acknowledging that Faza 1 shapes are mid-evolution and may not perfectly match Opus. Achievement of 2.0 retrieval mean signals "shape closes the engagement gap to within 14% of Opus" — sufficient signal for fitness ranking.

---

## §4 — Mutation oracle prompt update (binding)

Brief §3.3 mutation oracle prompt is updated to include explicit guidance for Qwen-targeted shape mutations:

```
For qwen-thinking and qwen-non-thinking shape mutations specifically:
- Emphasize multi-turn retrieval over single-shot retrieval
- Discourage premature finalization (e.g., "Continue retrieving until 
  you have evidence from at least 2 distinct queries before finalizing")
- Encourage iterative refinement of retrieval queries based on prior turn results
- Anti-premature-finalization scaffolding (e.g., "Before finalizing, ask: 
  what gap in evidence remains? Issue another retrieval if any gap exists.")
- Preserve cell semantic boundary (per Amendment 1 §6.4 mutation validator)

For claude, gpt, generic-simple shape mutations:
- Standard mutation guidance per original brief §3.3 applies
- No Qwen-specific scaffolding (these shapes don't exhibit the gap)
```

CC-2 implementing this update will fork the mutation oracle prompt template into two paths (Qwen vs non-Qwen) — a deliberate added complexity justified by the empirical Phase 4.5 signal. Document the fork in manifest v7 §mutation_oracle_design block.

---

## §5 — Acceptance criteria update (Faza 1 → Faza 2 gate)

Amendment 1 updated §4 condition 1 to reference trio_strict_pass rate. **Amendment 2 adds a Qwen-shape-specific sub-criterion:**

**§4 condition 1 (UPDATED twice — current binding form):**
"Best GEPA candidate per shape beats NULL-baseline by ≥+5pp on trio_strict_pass rate (where trio_strict_pass = trio_mean ≥ 4.0). For Qwen-targeted shapes (qwen-thinking, qwen-non-thinking), additionally: best candidate must have mean retrieval_calls per task ≥ 1.7 (engagement gap closed by ≥50% relative to Qwen baseline 1.33 → Opus parity 2.33)."

§4 conditions 2-4 unchanged:
- §4.2: ≥3/5 shapes show positive delta on trio_strict_pass
- §4.3: trio judge κ within ±0.05 of canonical 0.7878
- §4.4: zero cell semantic violations

**Additional FAIL condition added:**
- **§4.5 (NEW):** if best Qwen-shape candidate achieves +5pp trio_strict delta WITHOUT closing retrieval engagement gap (mean retrieval_calls < 1.5), this signals false-positive evolution (improvement via mutation-noise rather than mechanistic fix). Result: candidate REJECTED, shape marked FAIL even if other criteria pass. PM ratifies whether to re-run mutation generation with stronger anti-premature-finalization scaffolding or escalate.

---

## §6 — Forward to Phase 5 GEPA-evolved variant (out-of-Faza-1 scope, but recorded)

Phase 4.5 also specifies acceptance criteria for the Phase 5 GEPA-evolved variant (separate from Faza 1):

> If GEPA achieves both:
> - Qwen retrieval_calls ≥ Opus retrieval_calls per task (engagement parity)
> - Qwen H4 trio_mean delta from Opus narrowed by ≥ 0.30 points (score parity proxy)
> 
> the sovereign multiplier teza is rescued.

These are **Phase 5 acceptance criteria, not Faza 1.** Faza 1 acceptance per §5 above is necessary-but-not-sufficient — it validates that GEPA can produce candidates that score better AND engage retrieval more. Phase 5 GEPA-evolved variant validates that the engagement gain translates to score gain at scale (N ≥ 30 per cell, full pilot scenario reproduction).

CC-2 must NOT optimize for Phase 5 criteria during Faza 1 selection. Faza 1 selection is per §5 only. Phase 5 is downstream brief authored by PM post Faza 1 Checkpoint C.

---

## §7 — Cost projection (unchanged from Amendment 1)

Per-shape fitness function complexity does not increase per-call LLM cost — retrieval_calls counter is already telemetry on the agent harness (per pilot 2026-04-26 trace data). No additional API calls.

Total Faza 1 expected: ~$100.50, $100 hard cap, $80 internal halt — all unchanged.

Mutation oracle complexity (forked Qwen vs non-Qwen prompts) also unchanged in cost; the fork happens in oracle prompt construction, single LLM call per mutation regardless.

---

## §8 — Implementation order (binding)

CC-2 incorporates Amendment 2 changes into the manifest v7 + launch decision LOCK at the same time as Amendment 1 ratifications. Both amendments are paste-ratified by PM in single message; CC-2 should treat as conjoined binding contract.

Specifically:
1. Manifest v7 §metric_operationalization adds `retrieval_engagement_bonus` block per §3 above
2. Manifest v7 §mutation_oracle_design adds forked Qwen vs non-Qwen prompt template paths per §4 above
3. Launch decision §A inherited rules also includes Phase 4.5 retrieval-engagement signal as Cumulative Pre-flight Rule §A.9 (extending the original 8 sub-rules)
4. Launch decision §acceptance_criteria reflects updated §5 Qwen-specific sub-criterion
5. GEPA harness scaffold (per Amendment 1 §7 step 7) implements per-shape fitness function with telemetry hook on retrieval_calls counter

Test coverage requirements (Amendment 1 §8): unit tests for the per-shape fitness function (≥80% coverage) MUST include test cases for the Qwen-engagement bonus boundary conditions (1.49 / 1.5 / 1.99 / 2.0 / 2.5 retrieval_calls means).

---

## §9 — Cross-references

- Amendment 1: `briefs/2026-04-28-cc4-faza1-amendment-1.md`
- Original brief: `briefs/2026-04-28-cc4-gepa-tier2-evolution-faza1-brief.md`
- Pre-flight report: `briefs/2026-04-28-cc4-faza1-preflight-report.md`
- Phase 4.5 source: `decisions/2026-04-28-phase-4-5-tools-audit-results.md`
- Phase 4.3 verdict: `decisions/2026-04-28-phase-4-3-rescore-delta-report.md`
- Pilot data: `benchmarks/results/pilot-2026-04-26/pilot-task-{1,2,3}-{B,D}.jsonl`
- MULTI_STEP_ACTION_CONTRACT: `packages/agent/src/prompt-shapes/types.ts` (cell semantic boundary linchpin)

---

**End of Amendment 2. Conjoined with Amendment 1, binding upon single paste-into-CC-2.**