Files
waggle-os/docs/oss/adr/0004-cost-tracking-in-self-learning-out.md
shumeiko.it 2e1999a3f6 refactor: remove self-learning and the optimizer package (P1.4)
Self-learning is excluded from the community edition (ADR-0004); the
project owner reworks it separately. Cost tracking stays.

- agent: delete behaviour evolution (evolve-schema, iterative GEPA,
  compose-evolution, evolution gates, orchestrator and deploy),
  improvement and correction detection, AgentLearning, the eval-dataset
  builder, optimization-capture and the agent_insights tool. Secret
  detection moves from eval-dataset to secret-patterns. The LLM wiring
  keeps only the Haiku adapter and judge call used by skill audit and
  hygiene, so @ax-llm/ax is now declared by agent instead of server.
- server: delete /api/evolution/*, the evolution and optimizer services,
  GEPA expansion of the first chat message, the prompt_optimization and
  monthly_assessment jobs, the idle-session reviewer and its
  session-reviewer persona, prompt injection of past corrections and
  thumbs-down recorded as correction signals. Existing job rows of the
  removed types are ignored, not deleted.
- core: delete OptimizationLogStore; stop re-exporting the improvement
  signal and evolution run stores.
- web: delete the Evolution screen and the two job types in the
  automation editor.
- delete packages/optimizer.

Also logs UF-027 (self-learning loops read data nothing collects) and
UF-028 (the Settings prompt-shape choice is ignored by chat).
2026-10-20 00:54:58 +02:00

2.9 KiB

ADR-0004: Cost tracking is open; self-learning and optimizer are excluded

  • Status: Accepted
  • Date: 2026-10-14, amended 2026-10-15 (optimizer classified as self-learning) and 2026-10-20 (P1.4 done)

Context

Waggle tracks model costs and budget limits (execution traces, cost reservation, the CostTracker in the agent). It also has a self-learning subsystem: behaviour evolution, improvement signals, learning from user corrections, judge and eval loops, and the packages/optimizer package for GEPA prompt optimization, called from the chat route through packages/server/src/local/services/optimizer-service.ts.

Decision

  • Cost tracking and budget limits are part of the community edition, under MIT.
  • Self-learning is excluded, including packages/optimizer. The project owner reworks it separately.

Consequences

  • P1.2 and P1.5 move cost tracking to waggle.db.
  • P1.4 removes the self-learning modules, packages/optimizer and the optimizer service. The tables evolution_runs and improvement_signals are not carried into waggle.db.
  • P1.4 decides whether the optimization log (optimization_log) is still needed without optimizer.

Outcome of P1.4 (2026-10-20)

Removed:

  • agent: behaviour evolution (evolve-schema, iterative GEPA, compose-evolution, evolution gates, orchestrator and deploy), improvement and correction detection, AgentLearning, the eval-dataset builder, optimization-capture and the agent_insights tool;
  • server: /api/evolution/*, the evolution and optimizer services, GEPA expansion of the first chat message, the nightly prompt_optimization and monthly_assessment jobs, the idle-session reviewer and its session-reviewer persona, "User Corrections" injected into the prompt, and thumbs-down recorded as a correction signal;
  • core: OptimizationLogStore and the re-exports of ImprovementSignalStore and EvolutionRunStore; packages/optimizer;
  • web: the Evolution screen (Memory → Improvements) and the two job types in the automation editor.

Kept, because they are not behaviour learning:

  • cost tracking and execution traces, including traces of workflow-harness phases;
  • the LLM judge, used by skill audit, skill hygiene and loop jobs, with its Haiku adapter;
  • thumbs up/down feedback storage and /api/feedback/stats;
  • saving user corrections as ordinary memory facts;
  • skill tools (promote, auto-extract, retire), the "pattern detected" suggestion and the skill-distillation hint after a long successful turn. These are user-visible skill features; Egzakta may still classify the distillation hint as self-learning.

optimization_log served only self-learning: it held GEPA prompt variants and was read by the GEPA job, the monthly assessment and agent_insights. It is not carried into waggle.db. Job rows of the two removed types left in an existing data directory are ignored, not deleted, because the directory may be shared with the commercial edition.