Files
waggle-os/AGENTS.md
shumeiko.it 2e1999a3f6 refactor: remove self-learning and the optimizer package (P1.4)
Self-learning is excluded from the community edition (ADR-0004); the
project owner reworks it separately. Cost tracking stays.

- agent: delete behaviour evolution (evolve-schema, iterative GEPA,
  compose-evolution, evolution gates, orchestrator and deploy),
  improvement and correction detection, AgentLearning, the eval-dataset
  builder, optimization-capture and the agent_insights tool. Secret
  detection moves from eval-dataset to secret-patterns. The LLM wiring
  keeps only the Haiku adapter and judge call used by skill audit and
  hygiene, so @ax-llm/ax is now declared by agent instead of server.
- server: delete /api/evolution/*, the evolution and optimizer services,
  GEPA expansion of the first chat message, the prompt_optimization and
  monthly_assessment jobs, the idle-session reviewer and its
  session-reviewer persona, prompt injection of past corrections and
  thumbs-down recorded as correction signals. Existing job rows of the
  removed types are ignored, not deleted.
- core: delete OptimizationLogStore; stop re-exporting the improvement
  signal and evolution run stores.
- web: delete the Evolution screen and the two job types in the
  automation editor.
- delete packages/optimizer.

Also logs UF-027 (self-learning loops read data nothing collects) and
UF-028 (the Settings prompt-shape choice is ignored by chat).
2026-10-20 00:54:58 +02:00

315 lines
15 KiB
Markdown

# AGENTS.md — Waggle Community Edition
### Operating contract for every agent and contributor on this branch
> Read this file in full before changing code.
> On this branch it replaces the commercial-product contract. Where older documents under
> `docs/` (audits, briefs, launch and production-readiness records) disagree with this file
> or with the decision records in `docs/oss/adr/`, this file and the ADRs win.
---
## 0. Where We Are
This branch turns the commercial Waggle OS codebase into an open-source **community edition**.
The conversion is in progress: the code still contains team mode and the hive-mind memory
substrate. Work packages remove or replace them one by one.
Sources of truth, in order:
1. The agreed scope ("Waggle OSS Agreed Scope Annex", held by the project owner).
2. The decision records in [`docs/oss/adr/`](docs/oss/adr/README.md).
3. This file.
Work is split into work packages: **0.x** preparation, **P0** unblocking, **P1** main scope,
**P2** hygiene and publication. Name the work package in commit messages when it helps
(for example `refactor(server): remove Stripe routes (P1.1)`).
---
## 1. What the Community Edition Is
Waggle is a workspace-native AI agent platform with persistent memory: a web UI, a local
Fastify sidecar, an agent runtime with personas, skills and connectors.
The community edition:
- runs locally on Windows, macOS and Linux (the exact platform list awaits owner confirmation);
- installs with a Node-based installer (`packages/launcher`) and optional Tauri desktop
packages, without Python, Docker or a compiler on the user's machine;
- has no monetization: no tiers, trials, payments, license checks or upsell (ADR-0001);
- keeps memory on the user's machine through a memory contract backed by local mem0 (ADR-0002);
- sends analytics only after the user opts in (ADR-0006).
### Stack (verified 2026-10-15)
| Layer | Stack |
|---|---|
| Web UI | React 19, TypeScript, Vite 6, Tailwind CSS 3.4, Radix UI primitives (`apps/web`) |
| Sidecar | Fastify on Node.js, run with tsx in development (`packages/server`) |
| Desktop shell | Tauri 2 (`app/src-tauri`), an optional wrapper around the sidecar and web UI |
| Database | SQLite via better-sqlite3 and sqlite-vec |
| Memory | `MemoryStore` from `@waggle/memory-contract`, implemented on local mem0 by `@waggle/memory-mem0` (ADR-0007). Existing callers still use the hive-mind substrate until P1.5 |
| Local models | Waggle-managed Ollama runtime, or the user's own provider keys |
| Tests | Vitest (unit), Playwright (e2e and visual) |
npm workspaces with the root `package-lock.json`. Node `^20.19.0 || >=22.12.0`.
---
## 2. Repository Map and Scope Status
| Path | Status in the community edition |
|---|---|
| `apps/web` | Keep. The main UI; all React code lives here. |
| `app/` | Keep. Tauri shell only (there is no `app/src/`). |
| `packages/agent`, `core`, `server`, `shared`, `sdk`, `weaver`, `marketplace`, `cli`, `launcher` | Keep. |
| `packages/memory-contract`, `packages/memory-mem0` | Keep. The memory contract and its mem0 implementation (ADR-0007). |
| `packages/waggle-dance`, `packages/worker` | Postponed: team mode stays in the repo, is not built or shipped (ADR-0003). |
| `packages/hive-mind-*`, `memory-mcp`, `wiki-compiler` | Excluded once P1.5 moves all memory access to the contract. |
| `packages/admin-web`, `apps/www` | Excluded. |
| `apps/browser-ext` | Not decided. Do not extend it. |
| `sidecar/`, `benchmarks/`, `judging/`, most of `docs/` | Internal history of the commercial product, removed in P2.2. Community-edition docs live in `docs/oss/`. |
### Where things live
```
apps/web/src/
├── components/os/apps/ # per-app UI shells
├── components/os/overlays/ # OnboardingWizard.tsx, PersonaSwitcher.tsx
├── components/ui/ # shared UI primitives
└── lib/ hooks/ providers/ pages/ test/
packages/agent/src/ # most active package; grep before creating anything
agent-loop.ts core execution loop
orchestrator.ts buildSystemPrompt(), recallMemory()
personas.ts, persona-data.ts, custom-personas.ts
behavioral-spec.ts BEHAVIORAL_SPEC rules
tool-filter.ts filterToolsForContext()
injection-scanner.ts scanForInjection()
cost-tracker.ts CostTracker and model pricing (stays, ADR-0004)
feature-flags.ts exists; do not recreate
commands/ connectors/ mcp/ providers/
packages/server/src/local/ # the local sidecar: index.ts, routes/, services/
packages/core/src/ # config, cron-store, file-store, vault, logger, telemetry
packages/shared/src/ # shared types, Zod schemas, MCP catalog
```
Already removed: tiers and Stripe (P0.1); KVARK, enterprise packs, the compliance
subsystem, self-learning and `packages/optimizer` (P1.4, P1.6, ADR-0004). Still to be
removed, so do not build on them: `install-audit.ts`, `team-sync` and `migration` in core
(P1.2), the team server runtime (P1.1).
---
## 3. Conversion Rules
=== CRITICAL ===
1. **No monetization.** Do not add tier checks, trials, prices, payment flows, license
checks, or KVARK and Team upsell. When a work package touches such code, remove it.
The word "tier" also means UserTier (UI complexity), ModelTier, persona tiers, CLI levels
and memory health levels. Leave those alone and never mass-replace the word.
2. **Memory goes through the contract.** New code reaches memory through `MemoryStore`
(`@waggle/memory-contract`), must not import `@waggle/hive-mind-*`, `memory-mcp` or
`wiki-compiler`, and must not run SQL against a memory database (ADR-0007). Until P1.5 moves
the existing callers, keep changes to the old memory code minimal. The knowledge graph is
not in v1.
3. **Team mode is frozen.** Do not add features to team routes, WaggleDance or the worker.
4. **Self-learning is out.** Do not extend evolution, improvement signals, correction
learning, judge or eval loops, or `optimizer`. Cost tracking stays.
5. **Licenses.** Every new dependency must be compatible with MIT distribution. No GPL or
AGPL code in shipped artifacts: this is why Baileys (GPL `libsignal`) is replaced by
`whatsapp-web.js` (ADR-0005). State the license of a new dependency in the commit message.
6. **Analytics are opt-in.** Nothing is sent before the user agrees. The PostHog key comes
only from the build environment (`VITE_POSTHOG_KEY`) and is never committed.
7. **Three operating systems.** Every platform-specific branch needs Windows, macOS and Linux
paths. End users must not need Python, Docker, a compiler or a developer Node install.
8. **Secrets and history.** Never commit secrets. Never push this repository's history to a
public remote: the public release is a fresh repository built from the cleaned tree (P2.6).
9. **Upstream defects.** When you find or fix a defect that also exists in the commercial
product, log it in [`docs/oss/upstream-fixes.md`](docs/oss/upstream-fixes.md) in the same
commit, in English, using the template at the top of that file. Scope removals (tiers,
Stripe, KVARK, team mode) are not defects and are not logged.
=== END CRITICAL ===
---
## 4. Build, Run and Verify
### Setup and run
```bash
npm ci
npm run build:packages # required: workspace packages export dist/, the sidecar fails without it
npm run build # typechecks apps/web, builds it into the root dist/ (emptied first)
npm run dev:server # sidecar on http://localhost:3333, serves the UI from the root dist/
```
- The sidecar registers routes for the files in `dist/` when it starts (`@fastify/static`
with `wildcard: false`), so assets from a later build return 404. Restart the sidecar after
`npm run build`.
- `npm run dev:web` (Vite) cannot authenticate against the sidecar today (UF-003). Use the
UI served by the sidecar.
### Verification (run these, do not claim "it compiles")
```bash
npx tsc --noEmit --project packages/agent/tsconfig.json
npx tsc --noEmit --project packages/server/tsconfig.json # the sidecar runs via tsx, so nothing else typechecks it
npx tsc --noEmit --project app/tsconfig.json
npm run test # root Vitest: packages, tests/, scripts/ (excludes apps/**)
(cd apps/web && npx vitest run) # web UI tests use their own config
npm run lint # root config; ignores apps/web
(cd apps/web && npx eslint .) # web config; 49 errors pre-exist (docs/oss/baseline.md)
npm run check:deps # every imported package is declared in its package.json
```
- Playwright e2e reuses any server already listening on :3333 and its data directory. For an
isolated run set `WAGGLE_E2E_REUSE_EXISTING_SERVER=0` and a free `WAGGLE_E2E_PORT`. Its web
server step runs `npm run build:all`, which rewrites the root `dist/`.
- [`docs/oss/baseline.md`](docs/oss/baseline.md) records the build, typecheck, lint and test
results at the start of the conversion. Compare your results against it and do not let a
green check turn red.
---
## 5. How to Work
### 5.1 Think before coding
**Don't assume. Don't hide confusion. Surface tradeoffs.**
- State your assumptions explicitly. If uncertain, ask.
- If multiple interpretations exist, present them; don't pick silently.
- If a simpler approach exists, say so. Push back when warranted.
- If something is unclear, stop, name what's confusing, and ask.
=== CRITICAL ===
The most expensive failure mode is making a wrong assumption and building 100+ lines on top
of it. The fix costs ten times what the question would have cost. Stop. Ask. Then build.
=== END CRITICAL ===
### 5.2 Simplicity first
Minimum code that solves the problem. No features beyond the request, no abstractions for
single-use code, no configurability nobody asked for, no error handling for impossible
scenarios. If you write 200 lines and it could be 50, rewrite it.
### 5.3 Surgical changes
- Touch only what the task needs. Don't reformat or "improve" adjacent code.
- Match the existing style.
- If you notice unrelated dead code, mention it; don't delete it.
- Remove imports, variables and functions that your own change made unused.
Every changed line should trace to the request. Removals required by a work package are the
request.
### 5.4 Goal-driven execution
Turn tasks into verifiable goals ("fix the bug" becomes "write a test that reproduces it, then
make it pass"). For multi-step work, state a short plan with a check per step. A task is not
done until verification passes.
### 5.5 Context discipline
- Re-read a file before editing it if the conversation has moved on. Do not trust memory.
- Read files over 500 lines in chunks; never assume you saw all of it.
- Tool output over ~50k characters is truncated. If results look sparse, narrow the query.
- On a rename or removal, grep for direct references, types, string literals, dynamic
imports, barrel re-exports and tests. One grep is never enough.
### 5.6 Check before create
Grep before adding a file. If something might already exist, it probably does (see section 9).
### 5.7 Output discipline
- Long specs, plans and reports go into files under `docs/oss/`, not into chat.
- Implement large work in phases: commit per phase, report briefly, then continue.
- Never hide failures. Report failing tests and skipped checks with their output.
---
## 6. Pre-Work Protocol
- Before a structural refactor of a file over 300 lines, remove dead code in a separate
commit: `chore(scope): dead code removal — <file>`.
- Work in phases of at most five files: change, verify, then continue.
- If the architecture is flawed, state is duplicated, or patterns are inconsistent, say so
and propose a fix. Standard: what would a senior engineer reject in review?
---
## 7. Personas and Onboarding
- `packages/agent/src/persona-data.ts` holds the canonical `PERSONAS` array (pure data);
`personas.ts` holds the `AgentPersona` interface and logic.
- The PersonaSwitcher (`apps/web/src/components/os/overlays/PersonaSwitcher.tsx`) groups
personas into universal modes and template-scoped specialists (`apps/web/src/lib/persona-tier.ts`).
- The OnboardingWizard (`apps/web/src/components/os/overlays/OnboardingWizard.tsx`) runs
first-launch → who-are-you → model-gate → memory-import → template → first-task. Templates
and the template-to-persona mapping live in `overlays/onboarding/constants.ts`.
```typescript
interface AgentPersona {
id, name, description, icon, systemPrompt, modelPreference,
tools: string[], workspaceAffinity: string[],
suggestedCommands: string[], defaultWorkflow: string | null,
disallowedTools?: string[] // denylist, wins over tools[]
failurePatterns?: string[]
isReadOnly?: boolean // no write tools after persona tool filtering
tagline?: string
bestFor?: string[]
wontDo?: string
suggestedSkills?: string[]
suggestedConnectors?: string[]
suggestedMcpServers?: string[]
}
```
---
## 8. Security Constraints (Non-Negotiable)
1. **Secrets** belong in the secret store (`packages/core/src/vault.ts` until P1.2 replaces
it with a `SecretStore` interface) or an untracked local `.env`. Never in git. `.env.example`
holds placeholders only.
2. **Injection defense.** Call `scanForInjection()` from `injection-scanner.ts` on all
connector and external input.
3. **No eval and no dynamic require.**
4. **Tauri IPC allowlist** stays explicit in `app/src-tauri/capabilities/`.
5. **Parameterized SQL only.** No string interpolation in queries.
---
## 9. Already Built — Do Not Recreate
| File | What it does |
|---|---|
| `packages/agent/src/injection-scanner.ts` | `scanForInjection()` |
| `packages/agent/src/cost-tracker.ts` | `CostTracker` and model pricing |
| `packages/agent/src/tool-filter.ts` | `filterToolsForContext()` |
| `packages/agent/src/skill-frontmatter.ts` | `parseSkillFrontmatter()` |
| `packages/agent/src/feature-flags.ts` | Feature flags |
| `packages/agent/src/capability-router.ts` | Per-capability routing |
| `packages/agent/src/loop-guard.ts` | Infinite-loop prevention |
| `packages/agent/src/contradiction-detector.ts` | Memory conflict detection |
| `packages/shared/src/mcp-catalog.ts` | MCP server catalog |
| `packages/core/src/telemetry.ts` | Local telemetry pipeline |
| `packages/server/src/local/persona-tool-filter.ts` | `applyPersonaToolFilter`, `filterMcpToolsForPersona` |
---
## 10. Glossary
| Term | Meaning |
|---|---|
| Sidecar | The local Fastify server (`packages/server/src/local`) that the UI talks to |
| Memory contract | `MemoryStore` in `@waggle/memory-contract`: how Waggle remembers, searches, lists, updates and forgets memories (ADR-0007) |
| mem0 | Open-source memory library (`mem0ai`, Apache-2.0) behind the contract, run locally |
| Hive DS | Waggle design system tokens (`waggle-theme.css`) |
| BEHAVIORAL_SPEC | Core agent rules (`packages/agent/src/behavioral-spec.ts`) |
| Weaver | Memory consolidation and session-skill extraction (`packages/weaver`) |
| WaggleDance | Multi-agent team coordination (`packages/waggle-dance`), postponed |
| Work package | A unit of the conversion plan: 0.x, P0.x, P1.x, P2.x |
| UF-NNN | An entry in the upstream fixes log (`docs/oss/upstream-fixes.md`) |