refactor: remove self-learning and the optimizer package (P1.4)
Self-learning is excluded from the community edition (ADR-0004); the project owner reworks it separately. Cost tracking stays. - agent: delete behaviour evolution (evolve-schema, iterative GEPA, compose-evolution, evolution gates, orchestrator and deploy), improvement and correction detection, AgentLearning, the eval-dataset builder, optimization-capture and the agent_insights tool. Secret detection moves from eval-dataset to secret-patterns. The LLM wiring keeps only the Haiku adapter and judge call used by skill audit and hygiene, so @ax-llm/ax is now declared by agent instead of server. - server: delete /api/evolution/*, the evolution and optimizer services, GEPA expansion of the first chat message, the prompt_optimization and monthly_assessment jobs, the idle-session reviewer and its session-reviewer persona, prompt injection of past corrections and thumbs-down recorded as correction signals. Existing job rows of the removed types are ignored, not deleted. - core: delete OptimizationLogStore; stop re-exporting the improvement signal and evolution run stores. - web: delete the Evolution screen and the two job types in the automation editor. - delete packages/optimizer. Also logs UF-027 (self-learning loops read data nothing collects) and UF-028 (the Settings prompt-shape choice is ignored by chat).
This commit is contained in:
@@ -1,7 +1,8 @@
|
||||
# ADR-0004: Cost tracking is open; self-learning and optimizer are excluded
|
||||
|
||||
- Status: Accepted
|
||||
- Date: 2026-10-14, amended 2026-10-15 (`optimizer` classified as self-learning)
|
||||
- Date: 2026-10-14, amended 2026-10-15 (`optimizer` classified as self-learning) and
|
||||
2026-10-20 (P1.4 done)
|
||||
|
||||
## Context
|
||||
|
||||
@@ -24,3 +25,32 @@ improvement signals, learning from user corrections, judge and eval loops, and t
|
||||
tables `evolution_runs` and `improvement_signals` are not carried into `waggle.db`.
|
||||
- P1.4 decides whether the optimization log (`optimization_log`) is still needed without
|
||||
`optimizer`.
|
||||
|
||||
## Outcome of P1.4 (2026-10-20)
|
||||
|
||||
Removed:
|
||||
- agent: behaviour evolution (evolve-schema, iterative GEPA, compose-evolution, evolution
|
||||
gates, orchestrator and deploy), improvement and correction detection, `AgentLearning`,
|
||||
the eval-dataset builder, `optimization-capture` and the `agent_insights` tool;
|
||||
- server: `/api/evolution/*`, the evolution and optimizer services, GEPA expansion of the
|
||||
first chat message, the nightly `prompt_optimization` and `monthly_assessment` jobs, the
|
||||
idle-session reviewer and its `session-reviewer` persona, "User Corrections" injected
|
||||
into the prompt, and thumbs-down recorded as a correction signal;
|
||||
- core: `OptimizationLogStore` and the re-exports of `ImprovementSignalStore` and
|
||||
`EvolutionRunStore`; `packages/optimizer`;
|
||||
- web: the Evolution screen (Memory → Improvements) and the two job types in the
|
||||
automation editor.
|
||||
|
||||
Kept, because they are not behaviour learning:
|
||||
- cost tracking and execution traces, including traces of workflow-harness phases;
|
||||
- the LLM judge, used by skill audit, skill hygiene and loop jobs, with its Haiku adapter;
|
||||
- thumbs up/down feedback storage and `/api/feedback/stats`;
|
||||
- saving user corrections as ordinary memory facts;
|
||||
- skill tools (promote, auto-extract, retire), the "pattern detected" suggestion and the
|
||||
skill-distillation hint after a long successful turn. These are user-visible skill
|
||||
features; Egzakta may still classify the distillation hint as self-learning.
|
||||
|
||||
`optimization_log` served only self-learning: it held GEPA prompt variants and was read
|
||||
by the GEPA job, the monthly assessment and `agent_insights`. It is not carried into
|
||||
`waggle.db`. Job rows of the two removed types left in an existing data directory are
|
||||
ignored, not deleted, because the directory may be shared with the commercial edition.
|
||||
|
||||
@@ -55,6 +55,8 @@ Baseline: commit `b20b138` of `git.os-dev.tech/oleg.m/waggle-os`.
|
||||
| UF-024 | Workspace packages import packages they do not declare | Fixed | Build, packaging |
|
||||
| UF-025 | Workspace chat writes entities and error-path messages into personal memory | Open | Memory, privacy |
|
||||
| UF-026 | Three memory tools of the agent are never wired | Open | Agent, memory |
|
||||
| UF-027 | Self-learning loops run on data nothing collects | Open | Agent, self-learning |
|
||||
| UF-028 | The Settings prompt-shape choice has no effect on chat | Open | Web UI, chat |
|
||||
|
||||
## Entries
|
||||
|
||||
@@ -264,3 +266,22 @@ Entries UF-010 to UF-019 come from the UI audit `anti-slop/audit-001-2026-10-14.
|
||||
- Cause: `createMindTools` is called without `getAllWorkspaceSearches`, `feedback` and `embedder`, and each tool silently falls back when its dependency is missing.
|
||||
- What we did: nothing in the old code. In the community edition these tools are rebuilt on the memory contract in P1.5.
|
||||
- Recommendation: pass the three dependencies, or remove the tools whose dependency cannot be provided so the agent is not offered a tool that cannot work.
|
||||
|
||||
### UF-027. Self-learning loops run on data nothing collects
|
||||
- Status: Open
|
||||
- Found: 2026-10-20
|
||||
- Where (commit `b734c1d`; these files are removed from the community edition): `packages/agent/src/optimization-capture.ts:33` (`captureInteraction`); `packages/server/src/local/index.ts:753` (`createSkillTools`), `:2044` (`prompt_optimization` cron), `:2138`, `:2366` (`monthly_assessment` cron); `packages/server/src/local/monthly-assessment.ts:60`
|
||||
- Symptom: the nightly prompt-optimization job, the monthly agent assessment and the `agent_insights` tool report empty or zero activity however much the agent is used. Skill promotion, auto-extraction and retirement never leave the improvement signals their tool descriptions promise.
|
||||
- Cause: `captureInteraction`, the only function that writes interactions into `optimization_log`, has no caller; the only write to that table is the GEPA job storing its own prompt variants. The server builds the skill tools without `improvementSignals`, so their signal code never runs.
|
||||
- What we did: nothing in the old code. Self-learning is excluded from the community edition (ADR-0004), so these modules were removed.
|
||||
- Recommendation: when reworking self-learning, call `captureInteraction` after each chat turn and pass the signal store to `createSkillTools`, with a test that a chat turn produces an `optimization_log` row.
|
||||
|
||||
### UF-028. The Settings prompt-shape choice has no effect on chat
|
||||
- Status: Open
|
||||
- Found: 2026-10-20
|
||||
- Where: `apps/web/src/lib/adapter.ts:1060` (`sendMessage` sends `shape`); `packages/server/src/local/routes/chat.ts` (never reads it); `packages/server/src/local/routes/agent-run.ts:253` (the only route that honours a shape)
|
||||
- Symptom: the Prompt Shape dropdown in Settings saves the choice and the client sends it with every chat message, but chat answers do not change. The client comment says the sidecar "ignores `shape` until A3.1 wires it".
|
||||
- Cause: `/api/chat` does not read the `shape` field; only `/api/agent/run` applies a prompt-shape override.
|
||||
- What we did: nothing yet. The setting stays in the community edition until its fate is decided.
|
||||
- Recommendation: pass `shape` into the chat agent loop as `agent-run.ts` does, or hide the dropdown until it works.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user