oleg.m/waggle-os
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

moving
Some checks failed
Installer Smoke / installer-smoke (push) Has been cancelled
Details

Browse Source
This commit is contained in:
Oleg Maslov
2026-09-02 10:10:29 +02:00
commit 0c3e2ead3b
3841 changed files with 970576 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

147
docs/backend-map/DIAGRAMS/01-system-architecture.md Normal file
View File

@@ -0,0 +1,147 @@
# System Architecture (Layers)
This diagram is the cross-cutting runtime map for a Lovable rebuild of the Waggle OS frontend. It traces one request from the Tauri Rust shell, through the `apps/web` React SPA, into the Fastify **Local Sidecar** (port 3333) that serves the SPA and exposes all `/api/*` routes, down through the workspace packages that own each responsibility, into the data stores (per-workspace SQLite `*.mind` files via better-sqlite3 + sqlite-vec, plus team/cloud Postgres + Redis via the separate Cloud Server on port 3100), and out to the LiteLLM router and the LLM providers. Every node and ownership label is grounded in the section files. Key facts to anchor on: the frontend origin **is** the sidecar (same-origin, two-step Bearer-token auth), the sidecar uses local SQLite while the optional Cloud Server uses Postgres + Clerk + Redis, and the memory substrate (`mind` + `harvest`) lives in `@waggle/hive-mind-core`, not `@waggle/core`.
## Full Runtime Stack
```mermaid
flowchart TD
subgraph Shell["Desktop shell — app/ (Tauri 2.0, Rust)"]
TAURI["Tauri WebView\nloads apps/web dist from sidecar\norigin = tauri://localhost"]
end
subgraph Web["Frontend — apps/web (React 19 + Vite + Tailwind 4)"]
SPA["SPA shell + assets\nauth-exempt GET"]
AUTH2["Two-step auth\n1. GET /api/auth/session-token\n2. Authorization Bearer on every /api/*"]
APPS["OS apps + overlays\nChat · Memory{Harvest,Evolution,Wiki}\nMarketplace · Connectors · Launcher\nWaggleDance · Room · Mission Control"]
WS["WebSocket client\nGET /ws?token="]
end
subgraph Sidecar["Local Sidecar :3333 — packages/server (Fastify, Node, bundled in Tauri)"]
SEC["securityMiddleware\nHost allowlist · Bearer · CORS · RateLimit · CSP"]
STATIC["@fastify/static + SPA fallback\nserves apps/web dist"]
ROUTES["62 route plugins, flat /api/* paths\nchat · memory · harvest · evolution · skills\nmarketplace · connectors · wiki · tools\nwaggle-dance · stripe · vault · personas"]
WSL["/ws event-bus relay\napprovals · steps · tools · notifications"]
end
subgraph Packages["Workspace packages (responsibilities)"]
AGENT["@waggle/agent\nrunAgentLoop · Orchestrator · personas\ntool-executor · completion gates · cost-tracker\nconnectors · capability/trust · KVARK tools"]
HMC["@waggle/hive-mind-core\nmind: Identity·Awareness·Frames·KnowledgeGraph\nHybridSearch·Cognify·embeddings\nharvest: source adapters + 4-pass pipeline"]
SHARED["@waggle/shared\ntypes · zod schemas · TIERS\nmcp-catalog · tool-detection"]
DANCE["@waggle/waggle-dance\nWaggleMessage protocol\ndispatcher · SignalBus"]
MKT["@waggle/marketplace\ncatalog · installer · SecurityGate · sync"]
WIKI["@waggle/wiki-compiler\nentity/concept/synthesis/index/health pages"]
OPT["@waggle/optimizer\nself-evolution: GEPA + EvolveSchema\njudge · gates · evolution runs"]
end
subgraph LocalData["Per-workspace local data (better-sqlite3 + sqlite-vec)"]
MIND[("*.mind SQLite per workspace\nmemory_frames + _fts + _vec float[1024]\nknowledge_entities/relations · sessions\nharvest_runs · evolution_runs · wiki_pages")]
CFG[("~/.waggle/config.json\ntier · stripe_customer_id")]
VAULT[("Vault\napi keys · connector creds · kvark:connection")]
MKTDB[("marketplace.db SQLite")]
SKILLS[("~/.waggle/skills/*.md\nplugins · hooks.json")]
end
subgraph Cloud["Optional Cloud Server :3100 — packages/server (team/SaaS only)"]
AUTHP["authPlugin — Clerk JWT verify"]
CROUTES["teams · agents · jobs · scout\nsuggestions · messages · audit · analytics"]
WSG["/ws gateway — team chat"]
PG[("Postgres via drizzle\nDATABASE_URL")]
REDIS[("Redis pub/sub\nteam:*:waggle · job:*:progress")]
end
LITELLM["LiteLLM router\nlitellm-config.yaml\nPOST /chat/completions"]
PROVIDERS["LLM providers\nAnthropic · OpenAI · Ollama (local)\nVoyage · others"]
STRIPEAPI["Stripe API"]
KVARK["KVARK sovereign FastAPI\nsearch · ask · actions (TEAMS/ENTERPRISE)"]
TAURI --> SPA
SPA --> AUTH2
AUTH2 --> APPS
APPS -->|"Bearer /api/*"| SEC
WS -->|"?token="| WSL
SPA -.->|"shell + assets"| STATIC
SEC --> ROUTES
ROUTES --> AGENT
ROUTES --> HMC
ROUTES --> DANCE
ROUTES --> MKT
ROUTES --> WIKI
ROUTES --> OPT
ROUTES --> SHARED
WSL --> AGENT
AGENT --> SHARED
HMC --> SHARED
OPT --> AGENT
WIKI --> HMC
MKT --> SKILLS
AGENT -->|"recall/write memory"| HMC
HMC --> MIND
AGENT --> VAULT
ROUTES --> CFG
MKT --> MKTDB
AGENT --> SKILLS
AGENT -->|"POST /chat/completions"| LITELLM
OPT -->|"judge + mutate (Haiku)"| LITELLM
HMC -->|"embeddings (litellm/ollama)"| LITELLM
WIKI -->|"synthesize (Haiku/Ollama)"| LITELLM
LITELLM --> PROVIDERS
ROUTES -->|"stripeRoutes"| STRIPEAPI
STRIPEAPI -->|"webhook -> write tier"| CFG
VAULT -->|"KvarkClient"| KVARK
APPS -. "team mode only" .-> AUTHP
AUTHP --> CROUTES
CROUTES --> PG
CROUTES --> REDIS
WSG <--> REDIS
```
## Request / Auth Path (Local Sidecar)
This is the bootstrap handshake the rebuilt frontend must reproduce. The API root is the page's own origin; there is no separate host to configure.
```mermaid
flowchart LR
A["1. Load shell\nauth-exempt GET"] --> B["2. GET /api/auth/session-token\nsame-origin gated"]
B --> C["token"]
C --> D["3. Authorization Bearer token\non every /api/* call"]
C --> E["4. GET /ws?token=\nWebSocket relay"]
D --> F["Sidecar route plugins\nrate-limited, tier-gated"]
F -->|"403 TIER_INSUFFICIENT"| G["render upgrade prompt\nrequired + upgradeUrl"]
F -->|"429 + Retry-After"| H["backoff and retry"]
```
## Package Ownership Map
Which package owns which responsibility, distilled from the subsystem sections.
```mermaid
flowchart TD
R["Fastify sidecar routes\npackages/server"] --> A
R --> H
R --> D
R --> M
R --> W
R --> O
A["@waggle/agent\nagent loop · orchestrator · 22 personas\ntool execution + 11-step chain\ncompletion gates D3/D4/D1\nconnectors (30) · capability + trust\nautonomy gating · KVARK tools · cost"]
H["@waggle/hive-mind-core\nMIND: identity · awareness · frames\nknowledge graph · hybrid search\nrelevance scoring · cognify · embeddings\nHARVEST: adapters · 4-pass pipeline · dedup"]
D["@waggle/waggle-dance\nWaggleMessage protocol\ndispatcher + combo validation\nSignalBus ring buffer · AI-OS signals"]
M["@waggle/marketplace\nSQLite catalog · installer\nSecurityGate (4 layers) · 9 sync adapters"]
W["@waggle/wiki-compiler\n5 page types from memory\nincremental compile · Obsidian/Notion export"]
O["@waggle/optimizer\nGEPA + EvolveSchema + judge\nevolution gates · run store · deploy"]
S["@waggle/shared\nwire types · zod schemas\n5-tier model + capabilities\nMCP catalog · tool detection"]
A --> S
H --> S
D --> S
M --> S
W --> S
O --> S
```

466
docs/backend-map/DIAGRAMS/02-master-er.md Normal file
View File

@@ -0,0 +1,466 @@
# Master Data Model (two ER diagrams)
Waggle OS keeps data in two physically separate stores. The **per-workspace memory layer** is a single SQLite file (`*.mind`) per workspace — 14 base tables plus 2 virtual search tables — holding one user's private frames, knowledge graph, identity, awareness, and audit. The **team/cloud relational layer** is a shared PostgreSQL database (20 tables, Drizzle ORM) holding users, teams, agents, tasks, jobs, and governance. The two stores share **no cross-database foreign keys**; the only bridge is the `users.mind_path` text column in Postgres, which points at where a given user's local SQLite `*.mind` file lives. The note after the diagrams explains how one workspace mind relates to a user/team row.
---
## Diagram 1 — Per-workspace memory layer (SQLite, one `*.mind` file)
One isolated SQLite file per workspace. Schema version `'1'`. 14 base tables plus `memory_frames_fts` (FTS5 keyword index) and `memory_frames_vec` (vec0, 1024-dim embeddings), both keyed by `rowid = memory_frames.id`. Foreign keys exist only inside this file (frames to sessions, frames self-ref, relations to entities). `ai_interactions` is append-only (DB triggers reject UPDATE/DELETE). The knowledge graph is bitemporal: active rows have `valid_to IS NULL`.
```mermaid
erDiagram
sessions ||--o{ memory_frames : "gop_id FK"
memory_frames ||--o{ memory_frames : "base_frame_id self-FK"
memory_frames ||--|| memory_frames_fts : "rowid=id FTS5"
memory_frames ||--|| memory_frames_vec : "rowid=id vec0"
knowledge_entities ||--o{ knowledge_relations : "source_id FK"
knowledge_entities ||--o{ knowledge_relations : "target_id FK"
meta {
TEXT key PK
TEXT value
}
identity {
INTEGER id PK "CHECK id=1, single row"
TEXT name
TEXT role
TEXT department
TEXT personality
TEXT capabilities
TEXT system_prompt
TEXT created_at
TEXT updated_at
}
awareness {
INTEGER id PK
TEXT category "task action pending flag"
TEXT content
INTEGER priority
TEXT metadata "JSON"
TEXT created_at
TEXT expires_at "nullable"
}
sessions {
INTEGER id PK
TEXT gop_id UK "join key for frames"
TEXT project_id "nullable"
TEXT status "active closed archived"
TEXT started_at
TEXT ended_at "nullable"
TEXT summary "nullable"
}
memory_frames {
INTEGER id PK
TEXT frame_type "I P B"
TEXT gop_id FK
INTEGER t "per-gop ordinal"
INTEGER base_frame_id FK "nullable self"
TEXT content "JSON for B-frames"
TEXT importance "critical..deprecated"
TEXT source "user_stated..system"
INTEGER access_count
TEXT created_at
TEXT last_accessed
}
memory_frames_fts {
TEXT content "FTS5 rowid=id"
}
memory_frames_vec {
FLOAT embedding "float 1024 rowid=id"
}
knowledge_entities {
INTEGER id PK
TEXT entity_type
TEXT name
TEXT properties "JSON"
TEXT valid_from
TEXT valid_to "nullable=active"
TEXT recorded_at
}
knowledge_relations {
INTEGER id PK
INTEGER source_id FK
INTEGER target_id FK
TEXT relation_type
REAL confidence
TEXT properties "JSON"
TEXT valid_from
TEXT valid_to "nullable=active"
TEXT recorded_at
}
procedures {
INTEGER id PK
TEXT name
TEXT model
TEXT template
INTEGER version
REAL success_rate
REAL avg_cost
TEXT created_at
TEXT updated_at
}
improvement_signals {
INTEGER id PK
TEXT category "capability_gap..skill_promotion"
TEXT pattern_key
TEXT detail
INTEGER count
TEXT first_seen
TEXT last_seen
INTEGER surfaced "0 or 1"
TEXT surfaced_at "nullable"
TEXT metadata "JSON"
}
install_audit {
INTEGER id PK
TEXT timestamp
TEXT capability_name
TEXT capability_type "native..marketplace"
TEXT source
TEXT version "nullable"
TEXT risk_level "low medium high"
TEXT trust_source
TEXT approval_class
TEXT action
TEXT initiator "agent user system"
TEXT detail
}
ai_interactions {
INTEGER id PK "APPEND-ONLY"
TEXT timestamp
TEXT workspace_id "nullable"
TEXT session_id "nullable"
TEXT model
TEXT provider
INTEGER input_tokens
INTEGER output_tokens
REAL cost_usd
TEXT tools_called "JSON"
TEXT human_action "nullable"
TEXT risk_context "nullable"
TEXT imported_from "nullable"
TEXT persona "nullable"
TEXT input_text "nullable"
TEXT output_text "nullable"
}
execution_traces {
INTEGER id PK
TEXT session_id "nullable"
TEXT persona_id "nullable"
TEXT workspace_id "nullable"
TEXT model "nullable"
TEXT task_shape "nullable"
TEXT outcome "success..pending"
TEXT trace_json "JSON"
REAL cost_usd
INTEGER duration_ms
TEXT created_at
TEXT finalized_at "nullable"
}
evolution_runs {
INTEGER id PK
TEXT run_uuid UK
TEXT target_kind
TEXT target_name "nullable"
TEXT baseline_text
TEXT winner_text
TEXT winner_schema_json "nullable"
REAL delta_accuracy
TEXT gate_verdict "pass fail"
TEXT gate_reasons_json "JSON"
TEXT status "proposed..failed"
TEXT artifacts_json "nullable"
TEXT user_note "nullable"
TEXT failure_reason "nullable"
TEXT created_at
TEXT decided_at "nullable"
TEXT deployed_at "nullable"
}
harvest_sources {
INTEGER id PK
TEXT source UK
TEXT display_name
TEXT source_path "nullable"
TEXT last_synced_at "nullable"
INTEGER items_imported
INTEGER frames_created
INTEGER auto_sync "0 or 1"
INTEGER sync_interval_hours
TEXT last_content_hash "nullable"
TEXT created_at
}
```
> Standalone tables (no DB-level FKs to others): `meta`, `identity`, `awareness`, `procedures`, `improvement_signals`, `install_audit`, `ai_interactions`, `execution_traces`, `evolution_runs`, `harvest_sources`. Their `session_id` / `workspace_id` columns are plain TEXT, not enforced foreign keys. A `kg_entity_frames` link table is referenced by `frames.delete()` but is not defined in `schema.ts`, so it may be absent — omitted here.
---
## Diagram 2 — Team/cloud relational layer (PostgreSQL + Drizzle)
Shared multi-user store. All primary keys are server-generated UUIDs (`gen_random_uuid()`); all timestamps are `timestamp with time zone`. Every foreign key is `ON DELETE no action` — there are **no cascades**, so deleting a parent is blocked while children reference it. Two junction tables use composite PKs (`team_members`, `agent_group_members`). `users.mind_path` is the only pointer out to the SQLite memory layer.
```mermaid
erDiagram
users ||--o{ teams : "owns owner_id"
users ||--o{ team_members : "is"
teams ||--o{ team_members : "has"
users ||--o{ agents : "owns user_id"
teams ||--o{ agents : "scopes team_id"
users ||--o{ agent_groups : "owns"
agent_groups ||--o{ agent_group_members : "contains"
agents ||--o{ agent_group_members : "joins"
teams ||--o{ tasks : "has"
users ||--o{ tasks : "creates created_by"
users ||--o{ tasks : "assigned assigned_to"
teams ||--o{ messages : "channel"
users ||--o{ messages : "sends sender_id"
teams ||--o{ team_entities : "owns"
users ||--o{ team_entities : "shares shared_by"
teams ||--o{ team_relations : "owns"
team_entities ||--o{ team_relations : "source_id"
team_entities ||--o{ team_relations : "target_id"
teams ||--o{ team_resources : "owns"
users ||--o{ team_resources : "shares"
teams ||--o{ team_capability_policies : "governs"
users ||--o{ team_capability_policies : "updates"
teams ||--o{ team_capability_overrides : "governs"
users ||--o{ team_capability_overrides : "decides"
teams ||--o{ team_capability_requests : "scopes"
users ||--o{ team_capability_requests : "requests-decides"
teams ||--o{ agent_jobs : "owns"
users ||--o{ agent_jobs : "runs"
teams ||--o{ cron_schedules : "owns"
users ||--o{ cron_schedules : "creates"
users ||--o{ scout_findings : "for-user"
teams ||--o{ scout_findings : "for-team"
proactive_patterns ||--o{ suggestions_log : "fires"
users ||--o{ suggestions_log : "receives"
users ||--o{ agent_audit_log : "acts"
teams ||--o{ agent_audit_log : "scopes"
users {
uuid id PK
text clerk_id UK
text display_name
text email UK
text avatar_url "nullable"
text mind_path "nullable to SQLite mind"
timestamptz created_at
timestamptz updated_at
}
teams {
uuid id PK
text name
text slug UK
uuid owner_id FK
timestamptz created_at
}
team_members {
uuid team_id PK_FK
uuid user_id PK_FK
text role "default member"
text role_description "nullable"
jsonb interests "nullable"
timestamptz joined_at
}
agents {
uuid id PK
uuid user_id FK
uuid team_id FK "nullable"
text name
text role "nullable"
text system_prompt "nullable"
text model "default claude-haiku-4-5"
jsonb tools "default empty"
jsonb config "default empty"
timestamptz created_at
}
agent_groups {
uuid id PK
uuid user_id FK
text name
text description "nullable"
text strategy "default parallel"
timestamptz created_at
}
agent_group_members {
uuid group_id PK_FK
uuid agent_id PK_FK
text role_in_group "default worker"
integer execution_order "default 0"
}
tasks {
uuid id PK
uuid team_id FK
text title
text description "nullable"
text status "default open"
text priority "default normal"
uuid created_by FK
uuid assigned_to FK "nullable"
uuid parent_task_id "no FK logical self-ref"
timestamptz created_at
timestamptz updated_at
}
messages {
uuid id PK
uuid team_id FK
uuid sender_id FK
text type
text subtype
jsonb content
uuid reference_id "no FK"
jsonb routing "nullable"
timestamptz created_at
}
team_entities {
uuid id PK
uuid team_id FK
text entity_type
text name
jsonb properties "default empty"
uuid shared_by FK
timestamptz valid_from
timestamptz valid_to "nullable open-ended"
timestamptz created_at
}
team_relations {
uuid id PK
uuid team_id FK
uuid source_id FK
uuid target_id FK
text relation_type
real confidence "default 1.0"
jsonb properties "default empty"
timestamptz created_at
}
team_resources {
uuid id PK
uuid team_id FK
text resource_type
text name
text description "nullable"
jsonb config
uuid shared_by FK
real rating "default 0"
integer use_count "default 0"
timestamptz created_at
}
team_capability_policies {
uuid id PK
uuid team_id FK
text role
jsonb allowed_sources "default empty"
jsonb blocked_tools "default empty"
text approval_threshold "default none"
uuid updated_by FK "nullable"
timestamptz created_at
timestamptz updated_at
}
team_capability_overrides {
uuid id PK
uuid team_id FK
text capability_name
text capability_type
text decision
text reason "default empty"
uuid decided_by FK
timestamptz created_at
timestamptz decided_at
}
team_capability_requests {
uuid id PK
uuid team_id FK
uuid requested_by FK
text capability_name
text capability_type
text justification
text status "default pending"
uuid decided_by FK "nullable"
text decision_reason "nullable"
timestamptz created_at
timestamptz decided_at "nullable"
}
agent_jobs {
uuid id PK
uuid team_id FK
uuid user_id FK
text job_type
text status "default queued"
jsonb input
jsonb output "nullable"
timestamptz started_at "nullable"
timestamptz completed_at "nullable"
timestamptz created_at
}
cron_schedules {
uuid id PK
uuid team_id FK
uuid created_by FK
text name
text cron_expr
text job_type
jsonb job_config "default empty"
boolean enabled "default true"
timestamptz last_run_at "nullable"
timestamptz next_run_at "nullable"
timestamptz created_at
}
scout_findings {
uuid id PK
uuid user_id FK "nullable"
uuid team_id FK "nullable"
text source
text category
text title
text summary "nullable"
real relevance_score "default 0"
text url "nullable"
text status "default new"
timestamptz created_at
}
proactive_patterns {
uuid id PK "config-only no FK"
text name
jsonb trigger
text suggestion_type
text template
boolean enabled "default true"
}
suggestions_log {
uuid id PK
uuid user_id FK
uuid pattern_id FK
jsonb context
text status "default pending"
timestamptz created_at
}
agent_audit_log {
uuid id PK
uuid user_id FK
uuid team_id FK "nullable"
text agent_name
text action_type
text description
jsonb before_state "nullable"
jsonb after_state "nullable"
boolean requires_approval "default false"
boolean approved "nullable tri-state"
uuid approved_by FK "nullable"
timestamptz created_at
}
```
---
## How a workspace mind relates to a user/team row
A **workspace** is, physically, one SQLite `*.mind` file on the local machine (e.g. `personal.mind`). It is self-contained: switching workspace means opening a different file, and nothing inside that file joins across workspaces. There is no `workspace` table in Postgres — workspaces live as files, not relational rows.
The single connection point between the two layers is the **`users.mind_path`** column in Postgres: a nullable `text` pointer to where that user's private SQLite mind file lives. There are **no cross-database foreign keys**; the relationship is resolved only in application code. Concretely:
- A **user** row in Postgres (`users.id`, Clerk-backed via `clerk_id`) owns at most one `mind_path`. Following that path opens the user's personal `*.mind` SQLite file (Diagram 1), whose `identity` table (single CHECK-pinned row) describes that same user from the memory side.
- Inside the mind file, `ai_interactions.workspace_id` and `execution_traces.workspace_id` are plain TEXT labels for the originating workspace — they are **not** foreign keys to anything in Postgres. The same is true of `session_id`: it is a logical label, not an enforced cross-DB reference.
- **Team-shared knowledge** is duplicated in concept but not in storage: each user keeps a private SQLite `knowledge_entities` / `knowledge_relations` graph (Diagram 1), while the team keeps a separate Postgres `team_entities` / `team_relations` graph (Diagram 2). They are distinct tables in distinct engines; promoting a private fact to the team graph is an application-level copy, not a join.
- The relational layer **never stores memory frames**. All `memory_frames`, embeddings, awareness, and harvest tracking stay local in the workspace mind file; Postgres only holds the team/collaboration/governance metadata and the `mind_path` breadcrumb back to each local file.

80
docs/backend-map/DIAGRAMS/03-chat-turn-sequence.md Normal file
View File

@@ -0,0 +1,80 @@
# Chat Turn Sequence
This diagram traces one conversational chat turn end to end: the frontend POSTs to the real endpoint `POST /api/chat`, which is an SSE stream (the server validates, then calls `reply.hijack()` and writes a raw `text/event-stream`). The route handler (`packages/server/src/local/routes/chat.ts`) assembles the layered system prompt via the per-session `Orchestrator` (`buildSystemPrompt` + `recallMemory`, where recall runs `HybridSearch` over the workspace mind), filters tools by persona/context, then calls `runAgentLoop`, which POSTs to LiteLLM at `${litellmUrl}/chat/completions`. Tool calls pass through the 11-step `executeToolCall` middleware chain (governance, hooks, LoopGuard, injection scan); a `TraceRecorder` wires additive trace callbacks. Tokens and tool events stream back to the UI as named SSE events, and after the loop `autoSaveFromExchange` runs the `CognifyPipeline` to write new memory frames. All claims are grounded in `sections/03a-api-chat-agents.md`, `sections/05a-subsystem-agent-runtime.md`, and `sections/05b-subsystem-memory.md`.
```mermaid
sequenceDiagram
autonumber
participant UI as "Frontend (chat UI)"
participant Route as "POST /api/chat\nchat.ts (SSE)"
participant Orch as "Orchestrator\nbuildSystemPrompt + recallMemory"
participant Search as "HybridSearch\n(workspace mind)"
participant Loop as "runAgentLoop\nagent-loop.ts"
participant LLM as "LiteLLM\n/chat/completions"
participant Tools as "executeToolCall\n11-step chain"
participant Trace as "TraceRecorder"
participant Cog as "CognifyPipeline\nautoSaveFromExchange"
participant Disk as "session .jsonl"
UI->>Route: "POST { message, workspace, session, model?, persona?, autonomy? }"
Note over Route: "validate · injection scan score < 0.7 · RBAC · path guard (all pre-hijack)"
alt rejected
Route-->>UI: "400 / 403 JSON error (MESSAGE_TOO_LONG, INJECTION_DETECTED, ...)"
else accepted
Note over Route: "reply.hijack() · write SSE headers · wire AbortController to client close"
Route->>Disk: "persistMessage user turn (append .jsonl)"
Route->>Route: "generateTurnId UUID v4 · resolve model fallback chain"
Route->>Orch: "recallMemory(query, turnId)"
Note over Orch: "catch-up vs semantic · drop temporary/deprecated · injection scan"
Orch->>Search: "search(query) keyword + vector"
Search-->>Orch: "SearchResult[] ranked by finalScore = rrfScore * relevanceScore"
Orch-->>Route: "recalledContext text"
Route-->>UI: "event: step Recalling relevant memories..."
Route-->>UI: "event: tool / tool_result (auto_recall)"
Route->>Orch: "buildSystemPrompt()"
Orch-->>Route: "identity + self-awareness + preloaded context"
Note over Route: "layer profile + runtime facts + activeSpec.rules + skills + Workspace Now + corrections, then composePersonaPrompt"
Note over Route: "filterToolsForContext + filterAvailableTools + persona allow/deny"
Route->>Loop: "runAgentLoop { systemPrompt, tools, messages, stream:true, maxTurns:200, turnId, traceRecording }"
loop "each turn up to maxTurns"
Loop->>LLM: "POST /chat/completions { model, messages, tools, stream }"
Note over Loop,LLM: "signal = client-abort + 300s timeout · 429/5xx -> backoff, turn--, retry"
LLM-->>Loop: "stream chunks: content + tool_calls + usage"
Loop-->>UI: "event: token xN assistant text"
alt "tool_calls present"
Loop-->>UI: "event: tool { name, input }"
Loop->>Tools: "executeToolCall(name, args)"
Note over Tools: "parse args -> onToolUse -> governance blockedTools -> pre:tool -> pre:memory-write -> LoopGuard.check -> execute -> scanForInjection -> onToolResult -> post hooks"
opt "gated tool"
Loop-->>UI: "event: approval_required { requestId, toolName, input }"
UI->>Route: "POST /api/approval/:requestId { approved, always? }"
Route-->>Loop: "resolve(approved) (auto-deny after 5 min)"
end
Tools->>Trace: "record tool call + sanitized result"
Tools-->>Loop: "role:tool result message (sanitized)"
Loop-->>UI: "event: tool_result { name, result, isError }"
else "no tool_calls (final answer)"
Note over Loop: "maybeFireCompletionGate: D3 verification -> D4 phantom-write -> D1 skill-distillation"
alt "a gate fired"
Note over Loop: "push corrective directive, continue one more turn"
else "none fired"
Loop-->>Trace: "finalize trace"
Loop-->>Route: "AgentResponse { content, toolsUsed, usage }"
end
end
end
Note over Route: "cost tracking · KG entity extraction · disclaimers"
Route->>Cog: "autoSaveFromExchange(message, result.content)"
Cog->>Disk: "cognify writes new memory frame (I/P) + FTS + vec index + entities"
Route->>Orch: "commitSurfacedSignals()"
Route->>Disk: "persistMessage assistant turn"
Route-->>UI: "event: done { content, usage, toolsUsed, model, cost? }"
end
Note over Route: "on failure -> event: error { message } (raw turn still persisted) · finally: raw.end()"
```

218
docs/backend-map/DIAGRAMS/04-feature-api-map.md Normal file
View File

@@ -0,0 +1,218 @@
# Feature -> API Map (rebuild blueprint)
This diagram is the rebuild contract for a Lovable reconstruction of the Waggle OS web client. The left column lists every implemented OS app and overlay (windows opened by `appId` plus the modals/rails `Desktop.tsx` renders). The right column lists the backend endpoint GROUPS each one depends on, grouped exactly as the section files split them (03a Chat/Agents, 03b Memory/Knowledge/Wiki/Harvest, 03c Workspace/Team/Persona/Settings/Profile/Pins, 03d Marketplace/Skills/Connectors/Tools/Vault/Providers, 03e Evolution/Governance/Compliance/Costs/Approvals, 03f Realtime/Ops, 03g Billing/Stripe/KVARK). Every app first goes through the single shared `adapter` singleton and `ServiceProvider`; auth and tier-gating are cross-cutting and apply to all calls. Edges are sourced verbatim from section 04 "Feature -> UI Component -> Backend Endpoints" (and §3 endpoint reference). Apps with no backend calls (Voice, Team Governance, Keyboard Shortcuts) are shown wired only to the shared layer.
## 1. Apps and overlays mapped to endpoint groups
```mermaid
flowchart LR
classDef app fill:#1b2330,stroke:#a78bfa,color:#e8e8ef
classDef ovl fill:#23202e,stroke:#e5a000,color:#f3ead0
classDef shared fill:#0f1622,stroke:#7dd3fc,color:#dff1ff
classDef grp fill:#10271b,stroke:#34d399,color:#d7f7e6
%% Shared layer every feature passes through
ADAPTER["adapter singleton\nlib/adapter.ts\n127.0.0.1:3333"]:::shared
SVC["ServiceProvider\nadapter.connect"]:::shared
AUTH["Auth + Health\nGET /api/auth/session-token\nGET /health"]:::shared
TIER["Tier gating bus\n403 TIER_INSUFFICIENT\nwaggle:tier-insufficient"]:::shared
SVC --> ADAPTER
ADAPTER --> AUTH
ADAPTER --> TIER
%% Endpoint GROUPS (right side)
G_CHAT["03a Chat + Agents\n/api/chat SSE\n/api/history /api/agent/*\n/api/sessions/* /api/jobs/*"]:::grp
G_MEM["03b Memory + Knowledge\n/api/memory/* /api/identity\n/api/mind/* /api/documents"]:::grp
G_WIKI["03b Wiki + Harvest + Import\n/api/wiki/* /api/harvest/*\n/api/import/*"]:::grp
G_WS["03c Workspace + Templates\n/api/workspaces/* /api/files/*\n/api/workspace-templates/* /api/browse"]:::grp
G_TEAM["03c Team + Persona + Settings + Profile + Pins\n/api/personas/* /api/agent-groups/*\n/api/team/* /api/settings/* /api/profile/* /api/pins"]:::grp
G_MKT["03d Marketplace + Skills + Capabilities\n/api/marketplace/* /api/skills/*\n/api/capabilities/status"]:::grp
G_CONN["03d Connectors + Vault\n/api/connectors/* /api/vault/*"]:::grp
G_TOOLS["03d AI-OS Tool Launcher\n/api/tools/detect|launch|processes|kill|hooks"]:::grp
G_EVO["03e Evolution + Compliance + Costs + Feedback + Approvals\n/api/evolution/* /api/compliance/*\n/api/cost*|/api/costs /api/feedback /api/approval/*"]:::grp
G_RT["03f Realtime + Ops\n/api/events* SSE /api/notifications/*\n/api/cron/* /api/fleet/* /api/litellm/* /api/local-inference/* /api/backup|restore"]:::grp
G_WD["03f WaggleDance signals\n/api/waggle/signals\n/api/waggle/stream SSE"]:::grp
G_BILL["03g Billing + Tier + GDPR\n/api/tier /api/tier/start-trial\n/api/stripe/* /api/data/erase"]:::grp
G_TEL["03e Telemetry\n/api/telemetry/*"]:::grp
%% ---- Dock apps (left) ----
A_CHAT["Chat"]:::app
A_DASH["Dashboard / Home"]:::app
A_MEM["Memory\nframes/KG/harvest/wiki/evolution"]:::app
A_EVT["Events and Logs"]:::app
A_CAP["Skills and Apps"]:::app
A_CONN["Connectors"]:::app
A_COCK["Cockpit / Command Center"]:::app
A_MC["Mission Control"]:::app
A_WD["Waggle Dance"]:::app
A_AGT["Personas / Agents"]:::app
A_FILE["Files"]:::app
A_CRON["Scheduled Jobs"]:::app
A_MKT["Marketplace"]:::app
A_LAUN["AI Tools / Launcher"]:::app
A_VOICE["Voice\nstatic placeholder"]:::app
A_ROOM["Room\nsub-agent canvas"]:::app
A_APPR["Approvals"]:::app
A_TL["Timeline"]:::app
A_BAK["Backup and Restore"]:::app
A_TEL["Usage and Telemetry"]:::app
A_GOV["Team Governance\nTEAMS tier"]:::app
A_SET["Settings"]:::app
A_VAULT["Vault"]:::app
A_PROF["My Profile"]:::app
%% All apps go through the shared adapter
A_CHAT & A_DASH & A_MEM & A_EVT & A_CAP & A_CONN & A_COCK & A_MC & A_WD & A_AGT & A_FILE & A_CRON & A_MKT & A_LAUN & A_VOICE & A_ROOM & A_APPR & A_TL & A_BAK & A_TEL & A_GOV & A_SET & A_VAULT & A_PROF --> ADAPTER
%% Chat
A_CHAT --> G_CHAT
A_CHAT --> G_MEM
A_CHAT --> G_TEAM
A_CHAT --> G_WIKI
%% Dashboard
A_DASH --> G_MEM
A_DASH --> G_CHAT
%% Memory app
A_MEM --> G_MEM
A_MEM --> G_WIKI
A_MEM --> G_EVO
A_MEM --> G_RT
%% Events
A_EVT --> G_RT
%% Skills and Apps
A_CAP --> G_MKT
%% Connectors
A_CONN --> G_CONN
%% Cockpit + Compliance
A_COCK --> G_RT
A_COCK --> G_EVO
A_COCK --> G_CONN
A_COCK --> G_MKT
A_COCK --> G_WIKI
%% Mission Control
A_MC --> G_RT
A_MC --> G_TEAM
A_MC --> G_TOOLS
%% Waggle Dance
A_WD --> G_WD
%% Personas / Agents
A_AGT --> G_TEAM
A_AGT --> G_CHAT
A_AGT --> G_MKT
%% Files
A_FILE --> G_WS
%% Scheduled Jobs
A_CRON --> G_RT
%% Marketplace
A_MKT --> G_MKT
%% Launcher
A_LAUN --> G_TOOLS
%% Room
A_ROOM --> G_RT
%% Approvals
A_APPR --> G_EVO
%% Timeline
A_TL --> G_RT
%% Backup
A_BAK --> G_RT
%% Telemetry
A_TEL --> G_EVO
A_TEL --> G_TEL
%% Settings
A_SET --> G_TEAM
A_SET --> G_TEL
A_SET --> G_RT
%% Vault
A_VAULT --> G_CONN
%% Profile
A_PROF --> G_TEAM
```
## 2. Overlays mapped to endpoint groups
```mermaid
flowchart LR
classDef ovl fill:#23202e,stroke:#e5a000,color:#f3ead0
classDef grp fill:#10271b,stroke:#34d399,color:#d7f7e6
classDef shared fill:#0f1622,stroke:#7dd3fc,color:#dff1ff
ADAPTER["adapter singleton\nlib/adapter.ts"]:::shared
GO_WS["03c Workspace + Templates\n/api/workspaces/* /api/workspace-templates/*\n/api/browse"]:::grp
GO_TEAM["03c Persona + Settings + Profile\n/api/personas/* /api/agent-groups/*\n/api/settings /api/profile"]:::grp
GO_MEM["03b Memory + Identity\n/api/memory/search /api/memory/stats\n/api/identity"]:::grp
GO_HARV["03b Harvest + Import\n/api/harvest/* /api/import/*"]:::grp
GO_PROV["03d Vault + Providers + Skills\n/api/vault /api/providers\n/api/skills /v1/models"]:::grp
GO_FLEET["03f Fleet + LiteLLM\n/api/fleet/spawn /api/litellm/models"]:::grp
GO_NOTIF["03f Notifications\n/api/notifications/*"]:::grp
GO_BILL["03g Billing + GDPR\n/api/tier/start-trial\n/api/stripe/* /api/data/erase"]:::grp
GO_TEL["03e Telemetry\n/api/telemetry/track"]:::grp
GO_SVC["Connect + Health\n/api/auth/session-token /health"]:::shared
O_ONB["Onboarding wizard\n8 steps"]:::ovl
O_LOGIN["Login briefing"]:::ovl
O_SEARCH["Global search Cmd+K"]:::ovl
O_CWS["Create workspace dialog"]:::ovl
O_PSW["Persona switcher"]:::ovl
O_SPAWN["Spawn agent dialog"]:::ovl
O_WSW["Workspace switcher\nprops-driven"]:::ovl
O_NINBOX["Notification inbox"]:::ovl
O_CRAIL["Context rail"]:::ovl
O_ERASE["Erase data dialog GDPR"]:::ovl
O_UPG["Upgrade modal"]:::ovl
O_TRIAL["Trial expired modal"]:::ovl
O_KB["Keyboard shortcuts help\nno calls"]:::ovl
O_TOUR["Onboarding tooltips\nno calls"]:::ovl
O_ONB & O_LOGIN & O_SEARCH & O_CWS & O_PSW & O_SPAWN & O_ERASE & O_UPG & O_TRIAL & O_NINBOX & O_CRAIL --> ADAPTER
O_ONB --> GO_SVC
O_ONB --> GO_PROV
O_ONB --> GO_HARV
O_ONB --> GO_TEAM
O_ONB --> GO_WS
O_ONB --> GO_TEL
O_LOGIN --> GO_MEM
O_LOGIN --> GO_WS
O_SEARCH --> GO_WS
O_SEARCH --> GO_MEM
O_SEARCH --> GO_PROV
O_CWS --> GO_WS
O_CWS --> GO_TEAM
O_PSW --> GO_TEAM
O_SPAWN --> GO_FLEET
O_SPAWN --> GO_PROV
O_SPAWN --> GO_WS
O_NINBOX --> GO_NOTIF
O_CRAIL --> GO_MEM
O_ERASE --> GO_BILL
O_UPG --> GO_BILL
O_TRIAL --> GO_BILL
```

159
docs/backend-map/DIAGRAMS/05-tier-gating.md Normal file
View File

@@ -0,0 +1,159 @@
# Tier + Trust Gating
Waggle gates every action on **two orthogonal axes**. The **subscription tier** (TRIAL / FREE / PRO / TEAMS / ENTERPRISE) decides whether a feature exists for this user at all — enforced on HTTP routes via `requireTier()` returning `403 TIER_INSUFFICIENT`, and on tool registration (KVARK tools only exist when KVARK is configured). The **autonomy / trust level** (Normal / Trusted / YOLO) decides, per tool call, whether the UI must show an approval prompt — a pure runtime UX lever with a hardcoded critical blacklist that always wins, even at YOLO. The rule of thumb: *tier decides whether the door exists; trust decides whether it needs a key turn each time you walk through.* Always run a stored tier through `getEffectiveTier(tier, trialStartedAt)` before gating, because an expired TRIAL collapses to FREE.
---
## 1. The two gating axes (overview)
```mermaid
flowchart TD
USER["User stored tier plus trialStartedAt"] --> EFF["getEffectiveTier\nTRIAL expired collapses to FREE"]
EFF --> CAPS["getCapabilities\nTierCapabilities flag set"]
CAPS --> AXIS1["AXIS 1 — Tier gate\nfeature exists for this plan?"]
SESSION["Session AutonomyLevel\nnormal / trusted / yolo"] --> AXIS2["AXIS 2 — Trust gate\nclick needed for this call?"]
AXIS1 --> ROUTES["requireTier preHandler\n403 TIER_INSUFFICIENT on fail"]
AXIS1 --> KREG["KVARK tools registered\nonly when KVARK configured"]
AXIS2 --> CONF["needsConfirmationWithAutonomy\nper tool call"]
ROUTES --> GATED["Marketplace, Personas, Admin,\nTeam, Cloud-sync, Enterprise packs"]
KREG --> KVARK["kvark_search, kvark_feedback,\nkvark_action, kvark_ask_document"]
CONF --> TOOLS["Tools, connectors, bash,\ninstall_capability"]
```
---
## 2. The 5 tiers and what each unlocks
```mermaid
flowchart LR
subgraph TRIAL["TRIAL — 0 USD / 15 days"]
T1["All features unlocked\nmirrors TEAMS plus ENTERPRISE\nselfHosted OFF\ndecays to FREE after 15 days"]
end
subgraph FREE["FREE — 0 USD forever"]
F1["5 workspaces, 5 connectors\nspawnAgents ON, memory free\nbuilt-in skills only\nexport txt and md\nauditLog none, kvarkCta subtle"]
end
subgraph PRO["PRO — 19 USD / mo (solo)"]
P1["Unlimited workspaces plus connectors\ncustomSkills ON, marketplace\nexport txt md pdf json\nteamMembersLimit 1, auditLog basic\nno sharedWorkspaces, no cloudSync"]
end
subgraph TEAMS["TEAMS — 49 USD / seat"]
M1["sharedWorkspaces, teamSkillLibrary\ncloudSync, adminPanel\nauditLog full, selfHosted\nmanagedModelPool, priorityModels\nWaggleDance, kvarkCta active"]
end
subgraph ENT["ENTERPRISE — consultative"]
E1["KVARK sovereign on-prem\nall TEAMS capabilities\nselfHosted ON, kvarkCta none\ngovernance permissions route"]
end
TRIAL -->|"trial expires"| FREE
FREE -->|"upgrade 19/mo"| PRO
PRO -->|"upgrade 49/seat"| TEAMS
TEAMS -->|"sales contact"| ENT
```
---
## 3. Tier x Capability matrix (verbatim from `TIER_CAPABILITIES`)
`-1` means **unlimited**. Values quoted from `packages/shared/src/tiers.ts`.
| Capability | TRIAL | FREE | PRO | TEAMS | ENTERPRISE |
|---|---|---|---|---|---|
| `connectorLimit` | -1 | **5** | -1 | -1 | -1 |
| `workspaceLimit` | -1 | **5** | -1 | -1 | -1 |
| `embeddingProviders` | all 6 | inprocess, mock, ollama | + voyage, openai | all 6 | all 6 |
| `embeddingQuotaPerMonth` | -1 | -1 | -1 | -1 | -1 |
| `messageHistoryLimit` | -1 | -1 | -1 | -1 | -1 |
| `spawnAgents` | yes | yes | yes | yes | yes |
| `customSkills` | yes | **no** | yes | yes | yes |
| `teamSkillLibrary` | yes | **no** | **no** | yes | yes |
| `cloudSync` | yes | **no** | **no** | yes | yes |
| `exportFormats` | txt md pdf json | **txt md** | txt md pdf json | txt md pdf json | txt md pdf json |
| `teamMembersLimit` | -1 | **1** | **1** | -1 | -1 |
| `sharedWorkspaces` | yes | **no** | **no** | yes | yes |
| `adminPanel` | yes | **no** | **no** | yes | yes |
| `auditLog` | full | **none** | **basic** | full | full |
| `selfHosted` | **no** | **no** | **no** | yes | yes |
| `managedModelPool` | yes | **no** | **no** | yes | yes |
| `priorityModels` | yes | **no** | **no** | yes | yes |
| `kvarkCta` | subtle | subtle | subtle | **active** | **none** |
| `stripePriceId` | null | null | `STRIPE_PRICE_PRO` | `STRIPE_PRICE_TEAMS` | null |
`all 6` embedding providers = `inprocess, mock, ollama, voyage, openai, litellm`. Tier ordering (`TIER_ORDER`): `FREE 0, PRO 1, TEAMS 2, ENTERPRISE 3, TRIAL 3` — TRIAL ties ENTERPRISE because it has max capabilities, but is time-limited.
---
## 4. Tier-gated HTTP routes (Axis 1)
The frontend must catch `403 TIER_INSUFFICIENT` and show an upgrade prompt using `required` and `upgradeUrl`.
| Method | Path | Min tier |
|---|---|---|
| POST | `/api/marketplace/install` | PRO |
| POST | `/api/marketplace/publish` | PRO |
| GET | `/api/marketplace/enterprise-packs` | ENTERPRISE (plus KVARK configured) |
| POST | `/api/personas` | PRO |
| POST | `/api/personas/generate` | PRO |
| GET | `/api/cost/by-workspace` | TEAMS |
| POST | `/api/cloud-sync/toggle` | TEAMS |
| GET | `/api/admin/overview` | TEAMS |
| POST | `/api/admin/audit-export` | TEAMS |
| POST | `/api/team/connect` | TEAMS |
| GET | `/api/team/governance/permissions` | ENTERPRISE |
KVARK tools (`kvark_search`, `kvark_feedback`, `kvark_action`, `kvark_ask_document`) are registered only when `getKvarkConfig(vault)` returns a connection — 4 tools when configured, 0 otherwise.
---
## 5. The Normal / Trusted / YOLO autonomy gate (Axis 2)
```mermaid
flowchart TD
CALL["Tool call name plus args"] --> NC{"needsConfirmation?\nALWAYS_CONFIRM, connector write,\ndestructive bash"}
NC -- no --> RUN["Run silently"]
NC -- yes --> LVL{"AutonomyLevel"}
LVL -- normal --> PROMPT["Show approval prompt"]
LVL -- "trusted or yolo" --> CRIT{"isCriticalNeverAutopass?"}
CRIT -- yes --> PROMPT
CRIT -- no --> WHICH{"which level?"}
WHICH -- yolo --> AUTO["Auto-approve plus audit step"]
WHICH -- trusted --> TAP{"in TRUSTED_AUTOPASS\nor safe bash?"}
TAP -- yes --> AUTO
TAP -- no --> PROMPT
```
**Level behavior:**
| Level | Behavior |
|---|---|
| `normal` | Gate everything `needsConfirmation` flags (writes, connector writes, git, install, destructive bash). |
| `trusted` | Auto-pass `TRUSTED_AUTOPASS` (`write_file, edit_file, generate_docx, read_other_workspace, read_other_workspace_file`) and non-critical bash. Still gate git push/commit/pr/merge, `install_capability`, connector writes, cross-workspace writes. |
| `yolo` | Auto-pass everything **except** `isCriticalNeverAutopass`. |
**`isCriticalNeverAutopass` (never auto-passes, even at YOLO):** `rm -rf /` or `~` or `$HOME` or `/*`, any `sudo`, `format C:`, `mkfs`, `reg delete`, `dd if=… of=/dev`, forced `git push --force` to main/master/production, fork bomb; `install_capability` with `_riskLevel === 'high'`.
Autonomy override travels in the chat request body `{ "autonomy": { "level": "trusted"|"yolo", "expiresAt": <ms epoch> } }`; if `expiresAt` is absent or past, the server falls back to `normal`.
---
## 6. How tier + trust combine (worked examples)
```mermaid
flowchart TD
A["FREE user — connector_github_create_issue"] --> A1["Tier: no route gate"] --> A2["Trust: write so Normal prompts,\nTrusted gates, YOLO auto-passes"] --> A3["Runs after approval if connected"]
B["FREE user — Install marketplace pack"] --> B1["Tier: 403 needs PRO"] --> B3["Blocked, upgrade prompt"]
C["ENTERPRISE user — kvark_action"] --> C1["Tier: KVARK configured so registered"] --> C2["Trust: requires approval"] --> C3["Governed action runs with audit ref"]
D["Any tier, YOLO — bash sudo rm -rf /"] --> D1["Tier: none"] --> D2["Trust: isCriticalNeverAutopass"] --> D3["Still prompts even at YOLO"]
E["PRO user — install_capability starter-pack"] --> E1["Tier: none"] --> E2["Trust: ALWAYS_CONFIRM,\nclass from _riskLevel"] --> E3["Prompts, critical if high-risk"]
```
| Scenario | Tier check | Trust check | Net result |
|---|---|---|---|
| FREE — `connector_github_create_issue` | none (not a `requireTier` route) | write → Normal prompts; Trusted gates; YOLO auto-passes | Runs after approval if connected |
| FREE — install marketplace pack | `403 TIER_INSUFFICIENT` (needs PRO) | n/a (blocked first) | Blocked → upgrade prompt |
| ENTERPRISE — `kvark_action` (KVARK configured) | tools registered | requires approval | Governed action runs with audit ref |
| Any tier, YOLO — `bash sudo rm -rf /` | none | `isCriticalNeverAutopass` | Still prompts even at YOLO |
| PRO — `install_capability` (starter-pack) | none | in `ALWAYS_CONFIRM` | Prompts (critical if high-risk) |

110
docs/backend-map/DIAGRAMS/06-api-domains.md Normal file
View File

@@ -0,0 +1,110 @@
# API Domain Overview
This is the cross-cutting map of every HTTP/SSE/WebSocket surface a Lovable rebuild of the Waggle frontend must talk to. Almost everything lives in the **Local Fastify Sidecar** (default loopback `:3333`, flat `/api/*` paths, Bearer session-token + same-origin guards); a smaller set of Clerk-authenticated routes live in the separate **Cloud Server** (`:3100`, used only in SaaS/team-server deployments). Each domain node below is annotated with its endpoint count (counts are derived from the route tables in sections `03a`-`03g`; the approvals group is shared between the chat and governance sections, so it is shown once and noted). KVARK has no Fastify routes — it is reached only through the in-process `KvarkClient` (its method count is shown for completeness). Use this as the index; drill into the matching `03*` section for exact request/response shapes.
```mermaid
mindmap
root(("Waggle API\n7 domains"))
("Chat / Agents / Sessions\n~29 endpoints\nsec 03a")
("POST /api/chat SSE + DELETE history\n2")
("GET /api/history\n1")
("Agent status / cost / model / active\n6")
("Slash commands /api/commands/execute\n1")
("Sessions CRUD + search + export + timeline\n8")
("Agent groups CRUD + run\n5")
("POST /api/agent/run one-shot SSE\n1")
("Approvals SSE-paused gate\nshared with 03e")
("Memory / Wiki / Harvest\n~38 endpoints\nsec 03b")
("Memory frames search/CRUD/stats\n7")
("Knowledge graph read /api/memory/graph\n1")
("Wiki pages / compile / export\n8")
("Harvest preview/commit/sources/runs SSE\n11")
("Legacy import preview/commit\n2")
("Identity record + mind context\n5")
("Document version registry\n3")
("GDPR data erase\n1")
("Workspace / Team / Personas\n~76 endpoints\nsec 03c")
("Workspaces lifecycle + storage + context\n16")
("Workspace templates 15 built-in\n5")
("Team remote-proxy + local CRUD\n25")
("Personas catalog + create + generate\n5")
("Settings / tier / cloud-sync / admin\n14")
("User profile + style + brand\n7")
("Pins per workspace\n4")
("Marketplace / Skills / Connectors\n~60 endpoints\nsec 03d")
("Marketplace search/install/sources PRO\n15")
("Skills + plugins + hooks\n28")
("Connectors connect/disconnect/health\n4")
("AI-OS tool launcher detect/launch/hooks\n5")
("OAuth 5 providers\n3")
("Vault list/add/delete/reveal\n4")
("LLM + search providers catalog\n1")
("Evolution / Governance\n~39 endpoints\nsec 03e")
("Evolution runs accept/reject/run SSE\n8")
("Feedback thumbs + stats\n2")
("Telemetry local-only\n6")
("Compliance EU AI Act + templates + PDF\n12")
("Cost dashboard + by-workspace TEAMS\n3")
("Capabilities status + plugin toggles\n3")
("Approvals inbox + grants\n5")
("Real-time / Ops\n~52 endpoints\nsec 03f")
("WaggleDance UI signals + stream SSE\n4")
("WaggleDance v2 protocol bus\n2")
("Audit events + stats + stream SSE\n3")
("Cron schedules + trigger + history\n7")
("Notifications stream SSE + store\n6")
("Offline message queue\n5")
("Backup / restore / metadata\n3")
("Agent fleet spawn/pause/resume/kill\n5")
("LiteLLM control + pricing\n4")
("Local inference hardware/models/pull\n4")
("Anthropic proxy /v1/chat/completions SSE\n2")
("Filesystem browse local-only\n2")
("Browser extension health\n1")
("Telegram outbound push\n4")
("Cloud / Billing / KVARK\n~35 surfaces\nsec 03g")
("Sidecar inline auth/docs/health/ws\n6")
("Stripe checkout/webhook/sync/portal\n4")
("Cloud agents + groups Clerk\n10")
("Cloud jobs queue Clerk\n4")
("Cloud scout findings Clerk\n2")
("Cloud suggestions Clerk\n2")
("KVARK client methods no routes\n5")
("WebSocket sidecar + cloud\n2")
```
## Domains as a graph (server split + auth model)
The same domains, grouped by which server hosts them and what auth each requires. The sidecar serves the SPA and almost every domain; the cloud server is Clerk-gated and optional.
```mermaid
graph LR
FE["Frontend SPA\napps/web served by sidecar"]
subgraph SIDECAR["Local Sidecar :3333 - Bearer session-token + same-origin"]
D1["Chat / Agents / Sessions\n~29"]
D2["Memory / Wiki / Harvest\n~38"]
D3["Workspace / Team / Personas\n~76"]
D4["Marketplace / Skills / Connectors\n~60"]
D5["Evolution / Governance\n~39"]
D6["Real-time / Ops SSE+WS\n~52"]
BILL["Stripe billing\n4"]
BOOT["Auth token / docs / health / ws\n6"]
end
subgraph CLOUD["Cloud Server :3100 - Clerk JWT - optional SaaS"]
C1["Agents + Groups\n10"]
C2["Jobs queue\n4"]
C3["Scout findings\n2"]
C4["Suggestions\n2"]
C5["Team WebSocket gateway\n1"]
end
KVARK["KVARK client\nno routes - vault-credentialed\n5 methods - TEAMS/ENTERPRISE"]
FE -->|"same-origin Bearer"| SIDECAR
FE -.->|"team mode Clerk JWT"| CLOUD
D5 -->|"agent tools"| KVARK
D3 -->|"teamServerUrl proxy"| CLOUD
BILL -->|"writes tier to config.json"| D3
```