Self-learning is excluded from the community edition (ADR-0004); the project owner reworks it separately. Cost tracking stays. - agent: delete behaviour evolution (evolve-schema, iterative GEPA, compose-evolution, evolution gates, orchestrator and deploy), improvement and correction detection, AgentLearning, the eval-dataset builder, optimization-capture and the agent_insights tool. Secret detection moves from eval-dataset to secret-patterns. The LLM wiring keeps only the Haiku adapter and judge call used by skill audit and hygiene, so @ax-llm/ax is now declared by agent instead of server. - server: delete /api/evolution/*, the evolution and optimizer services, GEPA expansion of the first chat message, the prompt_optimization and monthly_assessment jobs, the idle-session reviewer and its session-reviewer persona, prompt injection of past corrections and thumbs-down recorded as correction signals. Existing job rows of the removed types are ignored, not deleted. - core: delete OptimizationLogStore; stop re-exporting the improvement signal and evolution run stores. - web: delete the Evolution screen and the two job types in the automation editor. - delete packages/optimizer. Also logs UF-027 (self-learning loops read data nothing collects) and UF-028 (the Settings prompt-shape choice is ignored by chat).
57 lines
2.9 KiB
Markdown
57 lines
2.9 KiB
Markdown
# ADR-0004: Cost tracking is open; self-learning and optimizer are excluded
|
|
|
|
- Status: Accepted
|
|
- Date: 2026-10-14, amended 2026-10-15 (`optimizer` classified as self-learning) and
|
|
2026-10-20 (P1.4 done)
|
|
|
|
## Context
|
|
|
|
Waggle tracks model costs and budget limits (execution traces, cost reservation, the
|
|
`CostTracker` in the agent). It also has a self-learning subsystem: behaviour evolution,
|
|
improvement signals, learning from user corrections, judge and eval loops, and the
|
|
`packages/optimizer` package for GEPA prompt optimization, called from the chat route through
|
|
`packages/server/src/local/services/optimizer-service.ts`.
|
|
|
|
## Decision
|
|
|
|
- Cost tracking and budget limits are part of the community edition, under MIT.
|
|
- Self-learning is excluded, including `packages/optimizer`. The project owner reworks it
|
|
separately.
|
|
|
|
## Consequences
|
|
|
|
- P1.2 and P1.5 move cost tracking to `waggle.db`.
|
|
- P1.4 removes the self-learning modules, `packages/optimizer` and the optimizer service. The
|
|
tables `evolution_runs` and `improvement_signals` are not carried into `waggle.db`.
|
|
- P1.4 decides whether the optimization log (`optimization_log`) is still needed without
|
|
`optimizer`.
|
|
|
|
## Outcome of P1.4 (2026-10-20)
|
|
|
|
Removed:
|
|
- agent: behaviour evolution (evolve-schema, iterative GEPA, compose-evolution, evolution
|
|
gates, orchestrator and deploy), improvement and correction detection, `AgentLearning`,
|
|
the eval-dataset builder, `optimization-capture` and the `agent_insights` tool;
|
|
- server: `/api/evolution/*`, the evolution and optimizer services, GEPA expansion of the
|
|
first chat message, the nightly `prompt_optimization` and `monthly_assessment` jobs, the
|
|
idle-session reviewer and its `session-reviewer` persona, "User Corrections" injected
|
|
into the prompt, and thumbs-down recorded as a correction signal;
|
|
- core: `OptimizationLogStore` and the re-exports of `ImprovementSignalStore` and
|
|
`EvolutionRunStore`; `packages/optimizer`;
|
|
- web: the Evolution screen (Memory → Improvements) and the two job types in the
|
|
automation editor.
|
|
|
|
Kept, because they are not behaviour learning:
|
|
- cost tracking and execution traces, including traces of workflow-harness phases;
|
|
- the LLM judge, used by skill audit, skill hygiene and loop jobs, with its Haiku adapter;
|
|
- thumbs up/down feedback storage and `/api/feedback/stats`;
|
|
- saving user corrections as ordinary memory facts;
|
|
- skill tools (promote, auto-extract, retire), the "pattern detected" suggestion and the
|
|
skill-distillation hint after a long successful turn. These are user-visible skill
|
|
features; Egzakta may still classify the distillation hint as self-learning.
|
|
|
|
`optimization_log` served only self-learning: it held GEPA prompt variants and was read
|
|
by the GEPA job, the monthly assessment and `agent_insights`. It is not carried into
|
|
`waggle.db`. Job rows of the two removed types left in an existing data directory are
|
|
ignored, not deleted, because the directory may be shared with the commercial edition.
|