moving

benchmarks/gepa/oracle/faza-1/mutation-prompt-template-non-qwen.md

@@ -0,0 +1,54 @@
+# GEPA Mutation Oracle Prompt — Non-Qwen (Claude / GPT / Generic-Simple)
+
+**Template class:** ###TEMPLATE_CLASS###
+**Shape under mutation:** ###SHAPE_NAME###
+
+---
+
+You are an expert prompt engineer. Your job is to produce a single mutated variant of the prompt-shape file below, evolving the *reasoning scaffold* and *planning step structure* while preserving exact cell semantics.
+
+## Context (non-Qwen branch)
+
+This shape (###SHAPE_NAME###) does **not** exhibit the retrieval-engagement gap that Phase 4.5 surfaced for Qwen. Standard mutation guidance per brief §3.3 applies — no Qwen-specific anti-premature-finalization scaffolding required.
+
+Apply general scaffold improvements: clearer reasoning structure, better chain-of-thought triggers, more explicit planning steps where appropriate for the model class.
+
+## Failure modes from Phase 4.3 (top-3 T2 categories for this shape)
+
+###FAILURE_MODE_SUMMARY###
+
+## Mutation guidance (binding)
+
+Modify only:
+- `systemPrompt(input)` method body (string-building only)
+- `soloUserPrompt(input)` method body
+- `multiStepKickoffUserPrompt(input)` method body
+- `retrievalInjectionUserPrompt(input)` method body
+- `metadata.evidence_link` (you MUST update to point to GEPA Gen 1 results: `benchmarks/results/gepa-faza1/oracle/mutation-prompt-template-non-qwen.md`)
+
+Standard mutation directions:
+1. **Evolve reasoning scaffold** — restructure the implicit reasoning prompts (e.g., "think step by step" variants, planning bullets)
+2. **Refine chain-of-thought triggers** — adjust where the model is prompted to articulate its reasoning before producing output
+3. **Improve planning step structure** — make multi-step task decomposition more explicit
+4. **Preserve all cell semantic boundaries** — see prohibitions below
+
+## Prohibited modifications (cell semantic LOCK)
+
+- **DO NOT modify** `MULTI_STEP_ACTION_CONTRACT` — this constant lives in `packages/agent/src/prompt-shapes/types.ts` and its bytes are SHA-pinned
+- **DO NOT modify** `metadata.{description,modelClass,defaultThinking,defaultMaxTokens}`
+- **DO NOT modify** the imports block
+- **DO NOT modify** the file structure (must remain a TypeScript module with a single PromptShape export)
+- **DO NOT modify** the JSON action contract
+- **DO NOT modify** task framing (persona/question/materials section labels)
+
+Mutations violating any of the above will be REJECTED by the mutation validator. Two consecutive invalid mutations trigger halt-and-PM per brief §5.
+
+## Baseline shape file (the input to your mutation)
+
+```typescript
+###BASELINE_SHAPE_CONTENT###
+```
+
+## Output format
+
+Output ONLY the mutated TypeScript file content. No prose, no commentary, no code fences. The file should be a drop-in replacement for the baseline shape file.

benchmarks/gepa/oracle/faza-1/mutation-prompt-template-qwen.md

@@ -0,0 +1,64 @@
+# GEPA Mutation Oracle Prompt — Qwen-Targeted Shape
+
+**Template class:** ###TEMPLATE_CLASS###
+**Shape under mutation:** ###SHAPE_NAME###
+
+---
+
+You are an expert prompt engineer. Your job is to produce a single mutated variant of the prompt-shape file below, evolving the *reasoning scaffold* and *retrieval-engagement guidance* while preserving exact cell semantics.
+
+## Empirical context (binding)
+
+Phase 4.5 audit empirically established that Qwen 3.6 35B-A3B (under both `thinking-on` and `thinking-off` configurations) exhibits a **retrieval-engagement gap** vs Opus 4.7 on the same `MULTI_STEP_ACTION_CONTRACT` tool surface:
+
+- Qwen retrieves **1.33 times per task** on average
+- Opus retrieves **2.33 times per task** on average
+- Qwen finalizes prematurely; Opus exhausts maxSteps in 67% of retrieval runs
+- Resulting H4 trio_mean delta: Qwen scores **−0.65 below** Opus on every task
+
+The mutation surface (prompt-shape body) is the lever to address this. **Your mutation must encourage more aggressive multi-turn retrieval engagement on Qwen.**
+
+## Failure modes from Phase 4.3 (top-3 T2 categories for this shape)
+
+###FAILURE_MODE_SUMMARY###
+
+## Mutation guidance (binding)
+
+Modify only:
+- `systemPrompt(input)` method body (string-building only)
+- `soloUserPrompt(input)` method body
+- `multiStepKickoffUserPrompt(input)` method body
+- `retrievalInjectionUserPrompt(input)` method body
+- `metadata.evidence_link` (you MUST update to point to GEPA Gen 1 results: `benchmarks/results/gepa-faza1/oracle/mutation-prompt-template-qwen.md`)
+
+Apply Qwen-specific scaffolding:
+1. **Emphasize multi-turn retrieval** over single-shot retrieval — make repeated retrieval feel mandatory, not optional
+2. **Discourage premature finalization** with explicit phrasing like:
+   - "Continue retrieving until you have evidence from at least 2 distinct queries before finalizing"
+   - "Single retrieval is rarely sufficient for synthesis tasks"
+3. **Encourage iterative refinement** — each retrieval query should build on prior turn's results
+4. **Add anti-premature-finalization scaffolding** in the multiStepKickoff or systemPrompt:
+   - "Before finalizing, ask: what gap in evidence remains? Issue another retrieval if any gap exists."
+   - "A complete answer requires triangulation from multiple retrievals"
+5. **Preserve all cell semantic boundaries** — see prohibitions below
+
+## Prohibited modifications (cell semantic LOCK)
+
+- **DO NOT modify** `MULTI_STEP_ACTION_CONTRACT` — this constant lives in `packages/agent/src/prompt-shapes/types.ts` and its bytes are SHA-pinned
+- **DO NOT modify** `metadata.{description,modelClass,defaultThinking,defaultMaxTokens}`
+- **DO NOT modify** the imports block
+- **DO NOT modify** the file structure (must remain a TypeScript module with a single PromptShape export)
+- **DO NOT modify** the JSON action contract (the `{"action": "retrieve", ...}` / `{"action": "finalize", ...}` shape)
+- **DO NOT modify** task framing (persona/question/materials section labels — the template rendering structure)
+
+Mutations that violate any of the above will be REJECTED by the mutation validator (`mutation-validator.ts`), consume one of two retry tolerances, and trigger halt-and-PM if exceeded twice consecutively per brief §5.
+
+## Baseline shape file (the input to your mutation)
+
+```typescript
+###BASELINE_SHAPE_CONTENT###
+```
+
+## Output format
+
+Output ONLY the mutated TypeScript file content. No prose, no commentary, no code fences. The file should be a drop-in replacement for the baseline shape file.