Files
waggle-os/docs/oss/adr/0004-cost-tracking-in-self-learning-out.md
shumeiko.it 2e1999a3f6 refactor: remove self-learning and the optimizer package (P1.4)
Self-learning is excluded from the community edition (ADR-0004); the
project owner reworks it separately. Cost tracking stays.

- agent: delete behaviour evolution (evolve-schema, iterative GEPA,
  compose-evolution, evolution gates, orchestrator and deploy),
  improvement and correction detection, AgentLearning, the eval-dataset
  builder, optimization-capture and the agent_insights tool. Secret
  detection moves from eval-dataset to secret-patterns. The LLM wiring
  keeps only the Haiku adapter and judge call used by skill audit and
  hygiene, so @ax-llm/ax is now declared by agent instead of server.
- server: delete /api/evolution/*, the evolution and optimizer services,
  GEPA expansion of the first chat message, the prompt_optimization and
  monthly_assessment jobs, the idle-session reviewer and its
  session-reviewer persona, prompt injection of past corrections and
  thumbs-down recorded as correction signals. Existing job rows of the
  removed types are ignored, not deleted.
- core: delete OptimizationLogStore; stop re-exporting the improvement
  signal and evolution run stores.
- web: delete the Evolution screen and the two job types in the
  automation editor.
- delete packages/optimizer.

Also logs UF-027 (self-learning loops read data nothing collects) and
UF-028 (the Settings prompt-shape choice is ignored by chat).
2026-10-20 00:54:58 +02:00

57 lines
2.9 KiB
Markdown

# ADR-0004: Cost tracking is open; self-learning and optimizer are excluded
- Status: Accepted
- Date: 2026-10-14, amended 2026-10-15 (`optimizer` classified as self-learning) and
2026-10-20 (P1.4 done)
## Context
Waggle tracks model costs and budget limits (execution traces, cost reservation, the
`CostTracker` in the agent). It also has a self-learning subsystem: behaviour evolution,
improvement signals, learning from user corrections, judge and eval loops, and the
`packages/optimizer` package for GEPA prompt optimization, called from the chat route through
`packages/server/src/local/services/optimizer-service.ts`.
## Decision
- Cost tracking and budget limits are part of the community edition, under MIT.
- Self-learning is excluded, including `packages/optimizer`. The project owner reworks it
separately.
## Consequences
- P1.2 and P1.5 move cost tracking to `waggle.db`.
- P1.4 removes the self-learning modules, `packages/optimizer` and the optimizer service. The
tables `evolution_runs` and `improvement_signals` are not carried into `waggle.db`.
- P1.4 decides whether the optimization log (`optimization_log`) is still needed without
`optimizer`.
## Outcome of P1.4 (2026-10-20)
Removed:
- agent: behaviour evolution (evolve-schema, iterative GEPA, compose-evolution, evolution
gates, orchestrator and deploy), improvement and correction detection, `AgentLearning`,
the eval-dataset builder, `optimization-capture` and the `agent_insights` tool;
- server: `/api/evolution/*`, the evolution and optimizer services, GEPA expansion of the
first chat message, the nightly `prompt_optimization` and `monthly_assessment` jobs, the
idle-session reviewer and its `session-reviewer` persona, "User Corrections" injected
into the prompt, and thumbs-down recorded as a correction signal;
- core: `OptimizationLogStore` and the re-exports of `ImprovementSignalStore` and
`EvolutionRunStore`; `packages/optimizer`;
- web: the Evolution screen (Memory → Improvements) and the two job types in the
automation editor.
Kept, because they are not behaviour learning:
- cost tracking and execution traces, including traces of workflow-harness phases;
- the LLM judge, used by skill audit, skill hygiene and loop jobs, with its Haiku adapter;
- thumbs up/down feedback storage and `/api/feedback/stats`;
- saving user corrections as ordinary memory facts;
- skill tools (promote, auto-extract, retire), the "pattern detected" suggestion and the
skill-distillation hint after a long successful turn. These are user-visible skill
features; Egzakta may still classify the distillation hint as self-learning.
`optimization_log` served only self-learning: it held GEPA prompt variants and was read
by the GEPA job, the monthly assessment and `agent_insights`. It is not carried into
`waggle.db`. Job rows of the two removed types left in an existing data directory are
ignored, not deleted, because the directory may be shared with the commercial edition.