# CC-4 Brief — GEPA Tier 2 Prompt-Shapes Evolution (Faza 1 pilot)

**Date:** 2026-04-28
**Author:** PM
**Status:** Authored, awaiting Marko ratification before paste-into-CC-4
**Sesija type:** CC-4 fresh (paralelno sa CC-1 Phase 4.4/4.5 + CC-3 memory shims)
**Critical path:** Korak 1 → GEPA Faza 1 → Faza 2 (gated) → Phase 5 GEPA-evolved
**Cost cap:** $100 hard, halt at $80
**Wall-clock projection (not trigger):** 3–5 dana CC time, ~2–3 dana wall-clock if no rate-limit blockers

---

## §1 — Context (zašto smo ovde)

Phase 4.3 verdikt (CC-1, commit be8f702→c9bda3d) empirijski potvrdio da H3/H4 fail je **72.2% Tier 2** (reasoning/planning failure koji zahteva prompt evoluciju), samo 5.6% Tier 1 (presentation artifact koji bi se popravio Phase 1.1 normalize). Phase 1.1 normalize delta = 0% kroz svih 12 cells. Worst-case T1 ceiling = 27.8%, ispod 30% threshold-a koji bi nam dao paper claim #2 multiplier signal. H4 je 100% T2 unanimnost.

Implikacija: agent fix sprint Phase 1-4 sam, ma koliko ga doteramo, ne može da spase multiplier tezu. Treba reasoning/planning evolucija nad prompt-ima — što je GEPA (Agrawal et al., genetic evolutionary prompt adaptation).

Ovo je gated work: Faza 1 = pilot proof-of-concept (1 cell, $100 cap). Ako Faza 1 PASS → Faza 2 expansion (sve T2-saturated cells, $200-300 cap, posebna ratifikacija). Ako Faza 1 FAIL → fallback na PHF (PASS-with-honest-framing) per Decision Matrix amendment.

---

## §2 — Scope LOCK (Faza 1)

**Što GEPA evoluira:** prompt-shapes templates u `packages/core/src/prompt-shapes/`. **NE** evoluira system prompts (cell semantics) — ti ostaju lock-ovani na manifest v6 specifikaciju da očuvamo apples-to-apples kontrolu sa Stage 3 v6 N=400 results.

**Razlog:** prompt-shapes evolucija je niži rizik za leakage cross-cell, evolution boundary je čist + auditable. System prompt evolucija bi compoundova confounders i ugrozila reproducibility paper claim #1 substrate (74% > Mem0 66.9%).

**Cell scope Faza 1:** **H3 only** (najjača T2 saturacija per Phase 4.3 verdict). H2 + H4 ulaze u Fazu 2 ako Faza 1 PASS. Zašto H3 prvi: maksimalni signal-to-cost ratio za "does GEPA help at all" gate.

**Prompt shapes scope:** svih 5 (claude / qwen-thinking / qwen-non-thinking / gpt / generic-simple). GEPA evoluira **per-shape**, ne unified. Selection metric = best-per-shape-per-cell (ne aggregate).

**N per evaluation:** 15 instanci per candidate per generation (per shape per cell). Insufficient za publishable paper claim, **dovoljno za GEPA fitness signal** (per Agrawal paper — fitness signal stabilizes around N=10-20 in inner loop).

**Generations:** 2 (initial population + 1 mutation round). Faza 1 = proof-of-concept, ne convergence search. Faza 2 expansion može da poveća na 3-5 generations.

**Population:** 3 candidates per shape per cell. Initial population = current shape (baseline) + 2 LLM-generated mutations (Opus 4.7 as mutation oracle).

---

## §3 — Methodology

### 3.1 — Fitness function

Composite score per candidate:
- **Primary:** trio-strict accuracy (Opus 4.7 + GPT-5.4 + MiniMax M2.7 ensemble, 2/3 must agree, max_tokens=3000 per Stage 3 v6 fix)
- **Cost penalty:** −0.5pp per $0.10 cost above baseline median (encourages efficiency)
- **Tie-breaker:** trio-soft accuracy (any 1/3 agreement)

Aggregate fitness per candidate = trio-strict − cost_penalty.

### 3.2 — NULL-baseline gate (binding)

Pre Faza 1 GEPA run, CC-4 mora da reproducira **NULL-baseline** = current prompt-shape on H3 cell, N=15, trio-judged. Ovo lockuje fitness floor pre evolucije.

Acceptance: NULL-baseline trio-strict mora pasti u predikcionom range-u iz Phase 4.3 (h3 cell est. 18-24% trio-strict baseline). Ako NULL-baseline ispod 15% ili iznad 30%, halt-and-PM (signal da nešto fundamentally drift-ovalo između Phase 4.3 i sad).

### 3.3 — Mutation oracle

Opus 4.7 generates 2 mutations per shape per generation, prompted with:
- Current shape template
- Failure mode summary (top-3 T2 failures from Phase 4.3 categorization)
- Constraint: preserve cell semantics (system prompt boundaries, output format contract)
- Mutation guidance: "modify reasoning scaffold, planning step structure, or chain-of-thought triggers — do not modify task framing or scoring criteria"

### 3.4 — Reproducibility

- **Seed:** GEPA evolution algorithm seeded with `42` (configurable in manifest v7)
- **Selection set:** 15 instances per cell sampled deterministic-stratified iz LoCoMo full corpus (same stratification as Stage 3 v6 manifest)
- **Manifest v7:** new manifest extending v6 sa GEPA section (population seed, mutation oracle SHA, generation count, candidate hashes)
- **Audit trail:** every candidate prompt hashed (SHA256), every evaluation logged sa raw judge outputs

### 3.5 — Substrate dependency

GEPA radi nad **post-Phase 4.6 HEAD** (mind/ + harness sa svih Phase 1-4 fix-eva integrated). Substrate version stays v6 (zero changes). Manifest v7 audit chain references manifest v6 anchor SHA.

---

## §4 — Acceptance criteria (Faza 1 → Faza 2 gate)

Faza 1 PASS conditions (binding, all 4 must hold):

1. **Best GEPA candidate per shape beats NULL-baseline by ≥ +5pp on trio-strict** (per H3 cell)
2. **At least 3/5 shapes show positive delta** (avoids cherry-picking single shape that lucked out)
3. **Trio judge κ remains within ±0.05 of canonical 0.7878** (validates judge ensemble didn't drift mid-run)
4. **Zero cell semantic violations detected** (audit step §6.4)

Faza 1 FAIL conditions (any single condition triggers FAIL verdict):

- Best candidate delta < +5pp on majority shapes
- Best candidate beats NULL-baseline only by overfitting evaluation set (detected via held-out 5 instances per cell)
- κ drift > 0.05 (judge ensemble unreliable)
- Cell semantic violation found

If FAIL → fallback PHF, GEPA work parked, paper claim #2 multiplier tezu reframe-uje na "demonstrated only under V2 retrieval + Tier 1 normalize, GEPA insufficient at this scope" (acceptable academic framing, ne mora se hide).

---

## §5 — Cost & halt

**Faza 1 cost projection (rigorous, not trigger):**
- NULL-baseline: 5 shapes × 15 instances × $0.50/run × 1 cell = **$37.50**
- GEPA Gen 1: 5 shapes × 3 candidates × 15 instances × $0.50 × 1 cell = **$112.50**
- Held-out validation: 5 shapes × top-1 × 5 instances × $0.50 = **$12.50**

Wait — that exceeds $100 cap. Recalc with Gen 1 reduced:

**Revised Faza 1 with $100 cap:**
- NULL-baseline: 5 shapes × 10 instances × $0.50 × 1 cell = **$25**
- GEPA Gen 1: 5 shapes × 3 candidates × 10 instances × $0.50 × 1 cell = **$75**
- Held-out: 5 shapes × top-1 × 5 instances × $0.50 = **$12.50**

**Total: ~$112** — still over. Final cut:

**Faza 1 LOCKED scope ($100 cap, $80 halt):**
- NULL-baseline: 5 shapes × **8 instances** × $0.50 × 1 cell = **$20**
- GEPA Gen 1: 5 shapes × 3 candidates × **8 instances** × $0.50 × 1 cell = **$60**
- Held-out: top-1 per shape × 5 instances × $0.50 = **$12.50**

**Total: $92.50 expected, $100 hard cap.** N=8 per evaluation is at lower bound of GEPA fitness signal — acceptable for proof-of-concept Faza 1, NOT acceptable for Faza 2 expansion (Faza 2 scales N to 20+).

**Halt triggers (any single triggers immediate halt + PM ratify):**
- Cumulative spend > $80
- κ drift detected mid-run (judge sample audit every 20 calls)
- Cell semantic violation detected (audit § 6.4)
- Mutation oracle (Opus 4.7) returns 2 consecutive invalid mutations (e.g. mutates system prompt instead of shape)
- Any LLM API blocker (rate-limit cascade, auth failure) — halt, restart with diagnostic pre-flight

---

## §6 — Pre-flight checks (binding, all 8 sub-rules from feedback memory)

Per `feedback_config_inheritance_audit.md` Extensions 1-6 + sub-rules 6-8, CC-4 mora da verifikuje pre run:

### 6.1 — Config inheritance audit
Eksplicitno specify Qwen + Opus + GPT + MiniMax model strings + reasoning_effort + max_tokens u manifest v7. Ne nasleduj iz manifest v6 implicitly. Naročito: Qwen reasoning mode (thinking vs non-thinking) MUST match per-shape configuration (qwen-thinking shape → reasoning enabled, qwen-non-thinking → disabled).

### 6.2 — Mixed-methodology baseline
NULL-baseline mora prijavljivati **trio-strict + self-judge razdvojeno** (ne shared aggregate). Phase 4.3 koristi trio-strict; pisanje "GEPA delta" mora citirati trio-strict numbers, ne self-judge.

### 6.3 — Scope verification
Pre run, CC-4 verifikuje da H3 cell ima ≥40 instanci u source corpus (potrebno za 8 NULL + 24 GEPA + 5 held-out = 37 instances + buffer). Ako H3 ima <40 instanci, halt-and-PM (signal da scope estimate pogrešan).

### 6.4 — Cell semantics prompt strictness preservation
Audit step pre commit Faza 1 results: za svaki GEPA candidate prompt, diff vs baseline. Diff mora biti samo unutar prompt-shape template body (between defined boundaries u shape file). Diff koji touch-uje cell.system_prompt ili cell.scoring_rubric = automatic INVALID, candidate dropped, mutation oracle re-prompted.

### 6.5 — σ-aware acceptance range
N=8 per cell daje cca CI ± 17pp at 95% (binomial), što je široko. **+5pp acceptance threshold je ne-statistički-rigorozan na N=8** — uzima se kao **fitness signal indicator**, ne kao publishable claim. To je razlog zašto Faza 1 = proof-of-concept, ne paper-ready evidence. Faza 2 scale-up je tek tu za publishable σ-bounded delta.

### 6.6 — Mixed-methodology variant
Trio-strict je primary; self-judge je supplementary diagnostic only. Faza 1 acceptance rule (§4) bazira se na trio-strict, ne self-judge.

### 6.7 — Cost super-linear input growth
GEPA candidates have variable token length (mutations may grow prompts). Cost calculation must use **worst-case 1.5× baseline token count** per candidate (encodes mutation overhead). If actual mid-run cost exceeds projection by >30%, halt.

### 6.8 — Source data structure
Verify H3 source data is **agentic knowledge work format** (not factoid LoCoMo). Phase 4.3 categorization confirms H3 = agentic. CC-4 spot-check 3 random H3 instances pre run, confirm task structure matches pilot 2026-04-26 corpus.

---

## §7 — Halt-and-PM checkpoints

Faza 1 ima 3 mandatory halt-and-PM points:

**Checkpoint A (post NULL-baseline, $25 cumulative):**
- Report NULL-baseline results per shape
- Confirm trio-strict in 18-24% range per shape
- Confirm κ within ±0.05 of canonical
- PM authorize GEPA Gen 1 kick

**Checkpoint B (mid-Gen 1, $50 cumulative):**
- Report intermediate κ from first 30 evaluations
- Report any cell semantic violation
- Report mutation oracle behavior (valid mutation rate)
- PM authorize completion of Gen 1

**Checkpoint C (post Gen 1 + held-out, ~$92 cumulative):**
- Final results per shape (NULL-baseline vs best GEPA candidate)
- κ stability report
- Acceptance rule (§4) verdict
- PM authorize either Faza 2 expansion OR FAIL fallback PHF

---

## §8 — Deliverables

1. **Code:**
   - `packages/core/src/prompt-shapes/gepa-evolved/` — directory sa best-per-shape candidates (5 files)
   - `packages/core/src/prompt-shapes/gepa-evolved/manifest.json` — selection metadata + audit chain
   - `benchmarks/gepa/faza-1/` — run logs + raw judge outputs + κ audit + diff snapshots

2. **Manifest v7:**
   - `benchmarks/preregistration/manifest-v7-gepa-faza1.yaml`
   - Extends v6 sa GEPA section (seed, oracle SHA, candidate hashes, generation count)
   - SHA256 logged u Checkpoint C report

3. **Decisions:**
   - `decisions/2026-04-28-gepa-faza1-launch.md` (LOCK upon paste-into-CC-4)
   - `decisions/2026-04-XX-gepa-faza1-results.md` (post-Checkpoint C)

4. **Test coverage:**
   - GEPA selection logic unit-tested (≥80% coverage)
   - Mutation validator (cell semantic check) unit-tested
   - κ audit utility unit-tested

5. **Memory entry:**
   - `.auto-memory/project_gepa_faza1_results.md` post Checkpoint C

---

## §9 — Out-of-scope (Faza 1)

Explicitly NOT in Faza 1:

- H2 + H4 cells (Faza 2 expansion)
- More than 2 GEPA generations
- Population > 3 candidates per shape
- N > 8 per evaluation
- System prompt evolucija (locked by §2 scope)
- Substrate (mind/) modifications (locked by §3.5)
- Apples-to-apples re-eval against pilot 2026-04-26 (separate Korak 12 work)
- Paper §5.4 framing update (post Phase 5 GEPA-evolved variant complete)

---

## §10 — Sequencing

**Predicates (must be done before CC-4 starts):**
- CC-1 Phase 4.3 verdict ratified ✅
- PM brief landed in `briefs/` ✅ (this file)
- Marko ratifikuje + executes paste-into-CC-4

**Successors (depend on Faza 1 outcome):**
- Faza 1 PASS → Faza 2 expansion brief authoring → CC-4 sledeća sesija
- Faza 1 PASS → Phase 5 GEPA-evolved variant brief (CC-1 sesija, post NULL-baseline)
- Faza 1 FAIL → PHF fallback decision memo + paper §5.4 reframe

**Parallel (CC-1 + CC-4 + CC-3):**
- CC-1: Phase 4.4 (skills sweep) → 4.5 (tools sweep) → Phase 5 NULL-baseline
- CC-4: Faza 1 GEPA pilot
- CC-3: Memory shims monorepo (Wave 1.3 next)

No code path conflicts between CC-1 and CC-4 (CC-1 touches harness around prompt-shapes, CC-4 produces new files in `gepa-evolved/` subdir).

---

## §11 — Cross-references

- Phase 4.3 results: `decisions/2026-04-28-phase-4-3-rescore-delta-report.md`
- Manifest v6 anchor: `benchmarks/preregistration/manifest-v6-preregistration.yaml`
- Stage 3 v6 5-cell summary: `D:\Projects\waggle-os\benchmarks\results\stage3-n400-v6-final-5cell-summary.md`
- Pilot 2026-04-26 result: `decisions/2026-04-26-pilot-verdict-FAIL.md`
- Decision Matrix PHF amendment: `decisions/2026-04-26-decision-matrix-self-judge-reframe.md`
- arxiv §5.4 multiplier framing: `research/2026-04-26-arxiv-paper/01-paper-skeleton.md`
- Feedback memory rules: `.auto-memory/feedback_config_inheritance_audit.md`
- 14-step launch plan: `.auto-memory/project_launch_plan_14_step_2026_04_27.md`

---

## §12 — Open questions for Marko

1. **Cost cap $100** — OK ili treba hard $80? PM rec $100 sa $80 internal halt.
2. **GEPA Faza 2 escalation budget** — predaj sad ili odluči post-Faza-1? PM rec post-Faza-1 (gated decision).
3. **Mutation oracle = Opus 4.7** — OK ili koristimo Sonnet 4.6 za cost reduction? PM rec Opus 4.7 (better mutation quality justifies cost; only 5×3×2 = 30 mutation calls total).
4. **Wall-clock priority** — paralelno sa CC-1 sweep ili sequential? PM rec paralelno (no conflict).