39 KiB
Backend API Delta — Waggle OS UX Refactor (Master List)
Purpose. The single, consolidated, de-duplicated, ordered master list of every backend endpoint the UX refactor must build or extend, plus every
.mind/relational schema migration required. This is the build contract for the LOCKED execution model: in-place incremental refactor ofapps/web+ targeted local Fastify sidecar extensions. Every endpoint is built over the existing substrate — no new database, minimal-to-zero SQLite migration (see §M).Sources (all read & grounded): PRD §16 (
docs/.../Waggle_OS_UX_Refactor_PRD.md:1060-1158), PRD §8/§21 phase+sprint structure (:212-253,:1304-1374), PRD §15 data model (:918-1057); the 22 gap cards (docs/ux-refactor/gap-cards/S00–S21); the route/substrate inventories (docs/ux-refactor/_inventory/{backend-routes,substrate-types,frontend}.md); the audited backend-map (docs/backend-map/sections/03a–03g); and spot-verified live source underpackages/server/src/local/routes/*.ts,packages/server/src/local/workspace-state.ts,packages/hive-mind-core/src/{workspace-manager,mind/schema,mind/db}.ts,packages/core/src/{install-audit,cron-store}.ts.Scope. Everything is the Local Sidecar (
packages/server/src/local/index.ts→buildLocalServer(), loopback:3333, flat/api/*, Bearer session-token). The desktop frontend talks ONLY to this server. The Clerk-gated Cloud server (packages/server/src/routes/*.ts) is out of scope — where its routes collide with PRD paths (notably/api/agents/*) the sidecar work is still net-new locally and flagged.
How to read this
Each row carries:
- Method + Path — the PRD/refactor contract path (PRD-literal where §16 names it).
- Disposition —
NET-NEW(no route serves this; build it) ·EXTEND(a real handler exists; add alias/param/field/behavior) ·NET-NEW (thin dispatcher/alias)(new path, delegates wholly to existing handlers, no new logic). - Build target — the exact route file to create or the existing file/handler/builder to extend
(with line where load-bearing). Verified absent:
agents.ts,artifacts.ts,home.ts,command.ts,mcps.ts,automations.ts,quick-capture.tsdo not exist underpackages/server/src/local/routes/(grep-confirmed) — all are net-new files. - Substrate — the store(s) it reads/writes.
- Shape — a 3–5 line request/response sketch.
- Screens — gap-card IDs that consume it.
Counts are in §Counts at the bottom. De-dup note: §16 lists 65 endpoints but several are consumed
by multiple screens (e.g. /api/workspaces/:id/state → S01+S02; /api/skills/* → S06+S19;
/api/agents/* → S09+S18; the cron→automations aliases → S11+S20; connector/MCP → S07+S08+S14+S17;
harvest → S15+S16). This master list states each endpoint once, attributing all consuming screens.
Endpoints that EXIST as-is with zero backend work (e.g. GET /api/workspaces, GET /api/connectors,
GET /api/memory/graph, the team CRUD core, the harvest engine) are excluded — they need only FE
wiring. Only NET-NEW + EXTEND backend work is listed.
Phase 0 — Architecture alignment (PRD §8 Phase 0 / §21 Sprint 1)
No new endpoints. Backend-relevant work is type alignment + shared additive fields that later phases write through. Land these first because Phase 1–5 write paths depend on them.
| Item | Disposition | Build target | Substrate | Notes / shape |
|---|---|---|---|---|
WorkspaceConfig V2 additive fields |
EXTEND (no route, no migration) | packages/hive-mind-core/src/workspace-manager.ts:5-58 (interface) + CreateWorkspaceOptions :60-95 |
workspace.json (file, NOT SQLite) |
Add optional description, type(WorkspaceType), status('active'|'paused'|'archived'), agentIds[], connectorIds[], mcpIds[], updatedAt, lastActiveAt. Default status:'active'; derive type from templateId/group for existing workspaces. No DB migration (JSON file). Consumed by S02/S17. |
Stamp updatedAt on write |
EXTEND | workspace-manager.ts:222 (update() currently stamps nothing but riskClassifiedAt) |
workspace.json |
One-line write-side touch. |
Stamp lastActiveAt |
EXTEND | agent loop / chat route write-back | workspace.json |
Currently lastActive is DERIVED at read from session mtimes (workspace-context.ts:375-387); persist it. |
| Shared frontend type unions | EXTEND (FE only) | apps/web/src/lib/types.ts |
— | Add WorkspaceType, Scope, Confidence, MemoryKind, ArtifactKind, AgentType, AutonomyLevel, ExtensionType (PRD §15.2). Reconcile FE MemoryFrame/Workspace lossy projections against API shapes (substrate-types §e). |
Phase 1 — Core runtime: Home, Workspace Desktop, Command Center (PRD §8 Phase 1 / §21 Sprint 2–3)
1a. Home Cockpit + Quick Capture (S01) — new home.ts + extend memory.ts
| Method + Path | Disposition | Build target | Substrate | Shape | Screens |
|---|---|---|---|---|---|
GET /api/home/briefing |
NET-NEW | new routes/home.ts; fans buildWorkspaceState() (workspace-state.ts:234) / buildWorkspaceNowBlock() (workspace-context.ts:191) over WorkspaceManager.list() and ranks |
memory_frames, session JSONL, awareness, cron_schedules, identity (for name) |
→ { greeting, userName, date, workspaces:[{id,name,rank,summary,pending,nextActions}], suggestedActions[] }. Cross-workspace ranking aggregator (per-workspace builder is the seed; no cross-WS ranker exists today). |
S01 |
GET /api/home/overnight |
NET-NEW | new routes/home.ts; aggregates over since-last-login window |
events/ai_interactions, notifications, cron_execution_history, memory_frames |
?since=<iso> → { memoriesAdded, artifactsCreated, automationsCompleted, failures:[{source,error}], window:{from,to} }. No time-windowed delta exists today. |
S01 |
POST /api/quick-capture |
EXTEND (thin handler delegating to memory write) | routes/memory.ts POST /api/memory/frames (:248) as the write primitive; new thin handler or alias |
memory_frames (personal .mind) + awareness (for kind:task) + POST /api/ingest (for kind:file) |
{ kind:'note'|'task'|'link'|'file', content, workspaceId? } → { frameId }. Defaults to personal mind, stamps source:'quick-capture'; task also writes an awareness row so it surfaces in nextActions. No migration. |
S01 |
1b. Workspace Desktop (S02) — extend workspaces.ts
| Method + Path | Disposition | Build target | Substrate | Shape | Screens |
|---|---|---|---|---|---|
GET /api/workspaces/:id/state |
EXTEND (thin route) | new thin route in workspaces.ts returning the buildWorkspaceState() sub-object already computed inside /context (workspaces.ts:311) |
memory_frames + session JSONL + awareness |
→ WorkspaceState { active, openQuestions, pending, blocked, completed, stale, recentDecisions, nextActions } (workspace-state.ts:38-55). pending+blocked seed the Tasks tab. No migration. |
S01, S02 |
GET /api/workspaces/:id/activity |
EXTEND (thin alias) | new thin route over GET /api/events?workspaceId= (events.ts) |
ai_interactions / execution_traces / audit_events |
?limit= → { events:[{ts,type,actor,summary}] }. Per-workspace audit feed. No migration. |
S02 |
No backend work for S02 Tasks (
tasks.tsCRUD EXISTS), Members (/api/team/membersEXISTS), or status-bar feeds (/api/fleet,/api/cron,/api/capabilities/statusall EXIST — compose client-side; an aggregate/statusroute is optional and deferred). Artifacts tab is S05's scope; S02 ships an interim file-registry view via existingGET /api/workspaces/:id/files.
1c. Command Center (Ctrl+K) (S00, S03) — new command.ts
| Method + Path | Disposition | Build target | Substrate | Shape | Screens |
|---|---|---|---|---|---|
GET /api/command/search?q= |
NET-NEW | new routes/command.ts federating route |
reads /api/memory/search (memory.ts:120), WorkspaceManager.list(), /api/skills, /api/workspaces/:id/sessions/search (sessions.ts) — no new store |
?q=&scope= → { results:[CommandResult{id,kind:'search'|'launch'|'create'|'run'|'navigate'|'extend', objectType, title, subtitle?, score, requiresApproval?, payload?}] }. Federates over ~4 substrates. |
S00, S03 |
POST /api/command/execute |
EXTEND | existing POST /api/commands/execute (note plural, commands.ts) — extend for navigate/create/run/extend dispatch, OR add a singular /command/execute alias |
command runtime + dispatch targets | { command, objectType?, payload?, workspaceId? } → { ok, result? }. Current runs slash-commands with a subset CommandContext; PRD's palette execute is broader. |
S00, S03 |
GET /api/command/recent |
NET-NEW (or client-derive first) | new routes/command.ts reading ai_interactions (or derive from session/event history) |
ai_interactions (read-only) |
→ { recent:[{command,ts,objectType}] }. No schema change. Cheapest v1 = client-side from session history; promote to server when a consumer needs cross-device. |
S03 |
GET /api/command/suggestions |
NET-NEW | new routes/command.ts reusing deriveNextActions (workspace-state.ts:182-218) + folding in /api/skills/suggestions |
read-only over memory_frames/awareness/cron |
→ { suggestions:[CommandResult] }. No migration. |
S03 |
Phase 2 — Work layer: Memory Center, Artifact Center, Workspace Creation, Onboarding (PRD §8 Phase 2 / §21 Sprint 4–5)
2a. Memory Center (S04, S16) — extend memory.ts + ONE optional migration
| Method + Path | Disposition | Build target | Substrate | Shape | Screens |
|---|---|---|---|---|---|
GET /api/memory |
EXTEND (alias) | accept bare path on existing GET /api/memory/frames (memory.ts:188) |
memory_frames |
?scope=&kind=&confidence=&status= → { frames:[...] }. Alias only. |
S04 |
GET /api/memory/:id |
NET-NEW (thin) | new thin read in memory.ts over FrameStore.getById(id) (no GET .../frames/:id exists today) |
memory_frames |
→ { frame }. Drawer detail. No migration. |
S04 |
POST /api/memory |
EXTEND (alias) | alias on POST /api/memory/frames (memory.ts:248) |
memory_frames |
{ kind,title,content,scope,tags? } → { id }. |
S04 |
PATCH /api/memory/:id |
EXTEND | extend PUT /api/memory/frames/:id (memory.ts:448) to accept PATCH + bare :id |
memory_frames (FrameStore.update) |
{ content?, importance?, status?, tags? } → { ok }. |
S04 |
POST /api/memory/:id/archive |
NET-NEW (thin) | new thin route; model archive as FrameStore.update(id, importance:'deprecated') OR status in the new metadata column |
memory_frames |
→ { ok }. No hard delete. |
S04 |
DELETE /api/memory/:id |
EXTEND (alias) | alias bare :id over DELETE /api/memory/frames/:id (memory.ts:551) |
memory_frames (FrameStore.delete) |
→ { ok }. |
S04 |
POST /api/memory/merge |
NET-NEW | new route in memory.ts; real logic (read N frames, synthesize merged content, write one, archive/delete originals) — reuse FrameStore + findDuplicate dedup |
memory_frames |
{ frameIds:[...], strategy?:'concat'|'llm' } → { mergedId, archived:[...] }. Net-new logic, low schema risk. |
S04 |
| Harvest preview/commit confidence + selection | EXTEND | POST /api/harvest/preview (harvest.ts:221) → return ALL items (or paged) + per-item confidence + normalized kind; POST /api/harvest/commit (harvest.ts:243) → accept { selectedIds?:[] } filter before the createIFrame loop (:382-409) |
in-memory parse (preview) / memory_frames (commit) |
Honors the trust-gate AC ("nothing imports without approval"). Preview confidence needs a classifier (LLM or heuristic). Preview-only confidence needs no migration. | S16 |
2b. Artifact Center (S05) — new artifacts.ts (largest net-new domain; aggregation only)
Single largest entity gap: no
Artifacttype, table, or/api/artifacts*route exists anywhere (substrate-types §e). The backend is a thin net-new aggregation/normalization layer over three existing stores — NO new data store. A lightweightartifacts.jsonindex holds title/status/ tags/relations; the bytes stay in the existing file/document/storage stores.
| Method + Path | Disposition | Build target | Substrate | Shape | Screens |
|---|---|---|---|---|---|
GET /api/artifacts |
NET-NEW | new routes/artifacts.ts; normalize 3 stores |
GET /api/workspaces/:id/files (workspaces.ts:594), document versions (documents.ts), storage files (workspaces.ts /storage/files) + artifacts.json index |
?workspaceId=&kind=&status= → { artifacts:[Artifact{id,title,kind,workspaceId,status,mimeType,storagePath,tags,relatedMemoryIds,...}] }. |
S02, S05 |
POST /api/artifacts |
NET-NEW | artifacts.ts; delegates byte-write to POST /api/ingest / files/upload / documents |
file/document stores + artifacts.json |
{ title,kind,workspaceId,content?/file? } → { id }. |
S05 |
GET /api/artifacts/:id |
NET-NEW | artifacts.ts; resolve composite id → normalized Artifact + relations + preview meta |
3 stores + documents.ts versions |
→ { artifact, relatedVersions[], relatedMemoryIds[] }. |
S05 |
PATCH /api/artifacts/:id |
NET-NEW | artifacts.ts; update title/status/tags/relations in artifacts.json (move = re-point storagePath via files/move) |
artifacts.json (+ files/move) |
{ title?, status?, tags?, relatedMemoryIds? } → { ok }. |
S05 |
DELETE /api/artifacts/:id |
NET-NEW (route) | artifacts.ts; remove index entry + optionally backing file via files/delete/storage/delete |
artifacts.json + file stores |
?deleteBacking=bool → { ok }. |
S05 |
GET /api/artifacts/search-related?q= |
NET-NEW | artifacts.ts; lean on memory_frames_fts + wiki search internally |
memory_frames_fts, sessions, tasks, fleet |
?q=&artifactId= → { related:[{type,id,title,score}] }. |
S05 |
2c. Workspace Creation (S17) — extend workspaces.ts write path
| Method + Path | Disposition | Build target | Substrate | Shape | Screens |
|---|---|---|---|---|---|
POST /api/workspaces (extend body) |
EXTEND | workspaces.ts:116-135 + WorkspaceManager.create (workspace-manager.ts:60-95) |
workspace.json |
Accept description, type, status, skills[], agentIds[], connectorIds[], mcpIds[] (the Phase-0 additive fields). No DB migration. |
S17 |
GET /api/workspace-templates (extend shape) |
EXTEND (optional) | workspace-templates.ts:33 |
template store | Add skills[]/mcps[]/type to WorkspaceTemplate so a chosen template pre-populates all 4 suggestion panels. |
S17 |
2d. Onboarding: First Launch, Who-Are-You, Tool Discovery, Memory Import (S12–S15)
| Method + Path | Disposition | Build target | Substrate | Shape | Screens |
|---|---|---|---|---|---|
PUT /api/profile (extend merge) |
EXTEND | profile.ts:167-228 allow-list (:172-196) + UserProfile (:41-94) + DEFAULT_PROFILE (:96-135) |
profile.json (NOT SQLite) |
Add workType, teamSize, goals[] to the merged-fields allow-list + interface. Already mirrors identity → memory P/I frame (:201-224). No migration. |
S13 |
POST /api/harvest/sources/:id/sync |
NET-NEW (thin) | new thin route in harvest.ts; resolve registered source + re-run commit |
harvest_sources + memory_frames |
→ { runId }. Sync today = POST /api/harvest/commit. NOTE: current sources keyed by :source name (not :id) — keep name key or alias. No new substrate. |
S15, S16 |
S12 First Launch + S14 Tool Discovery need ZERO net-new backend — all source catalogs already have routes (
GET /api/connectors,GET /api/tools/detect,GET /api/offline/status,GET /api/workspaces); selections persist client-side inOnboardingState(localStorage), optionally threaded via the extendedPUT /api/profile. The full harvest engine (preview/commit/sources/progress/scan-claude-code/runs/extract-identity+POST /api/ingest) already EXISTS for S15 — only the/sources/:id/syncalias (above) and the S16 confidence/selection extension (Phase 2a) are new.
Phase 3 — Intelligence layer: Agents, Skills, Automations (PRD §8 Phase 3 / §21 Sprint 6)
3a. Agent Center + Agent Builder (S09, S18) — new sidecar agents.ts + agent store
Naming collision:
/api/agents/*CRUD exists ONLY on the Clerk-gated Cloud server (packages/server/src/routes/agents.ts) — NOT the sidecar (confirmed absent). All of §16.7 is net-new locally: a newpackages/server/src/local/routes/agents.tsregistered inlocal/index.ts. Persistence (recommended v1): a{dataDir}/agents.jsonfile store, mirroring the agent-groups JSON precedent (agent-groups.ts:29) — no SQLite migration. (Alternative: anagentstable inmind/schema.tswith SCHEMA_VERSION bump — only if agents must be FTS/relation-queryable. See §M.)
| Method + Path | Disposition | Build target | Substrate | Shape | Screens |
|---|---|---|---|---|---|
GET /api/agents |
NET-NEW | new routes/agents.ts; reads agent store; overlay live status from /api/agents/active + /api/fleet |
agents.json (new) |
→ { agents:[Agent{id,name,type,goal,personaId,model,autonomyLevel,status,lastRunAt,successRate,...}] }. May union saved agents + read-only personas for back-compat. |
S09, S18 |
POST /api/agents |
NET-NEW | agents.ts; persist Agent (§15.5 fields); PRO-tier gate like personas.ts:32 |
agents.json + install_audit (if elevated tools/MCPs claimed) |
{ name,type,goal,model,personaId?,autonomyLevel,memoryScopes,skillIds,connectorIds,mcpIds,permissions } → { id }. |
S18 |
GET /api/agents/:id |
NET-NEW | agents.ts read over store |
agents.json |
→ { agent }. |
S09, S18 |
PATCH /api/agents/:id |
NET-NEW | agents.ts; mirror agent-groups.ts:74-90 PATCH shape |
agents.json |
{ ...partial } → { ok }. |
S18 |
POST /api/agents/:id/run |
EXTEND | resolve agent → call real executor POST /api/fleet/spawn (fleet.ts:66, the only path that runs runAgentLoop :185). Do NOT use agent-groups/:id/run (stub :105). |
fleet/orchestrator + execution_traces via TraceRecorder |
{ input?, workspaceId? } → { sessionId }. Map agent persona/model/memoryScope/workspace onto spawn body. |
S09, S18 |
POST /api/agents/:id/pause |
EXTEND | map agent→active session → POST /api/fleet/:workspaceId/pause (fleet.ts:260) |
fleet | → { ok }. |
S09, S18 |
GET /api/agents/:id/traces |
NET-NEW (route over existing store) | new thin read over execution_traces (mind/schema.ts:199, written by chat.ts/evolution.ts, no HTTP read today), filtered by agent/session; fallback session timeline sessions.ts |
execution_traces + ai_interactions |
?limit= → { traces:[{ts,step,tool,outcome,cost}] }. |
S09, S18 |
3b. Skills Hub + Skill Builder (S06, S19) — extend skills.ts
| Method + Path | Disposition | Build target | Substrate | Shape | Screens |
|---|---|---|---|---|---|
PATCH /api/skills/:id |
EXTEND | add PATCH alias + :id↔:name over PUT /api/skills/:name (skills.ts:506) |
~/.waggle/skills/*.md |
{ content? } → { ok }. No new substrate. |
S06, S19 |
POST /api/skills/:id/test |
EXTEND | add :id path variant routing to existing POST /api/skills/test (body-driven) |
skill file + parseSkillFrontmatter |
{ testInput? } → { wouldInject, frontmatter }. Sandbox/dry-run only; no execution. |
S06, S19 |
POST /api/skills/:id/install |
NET-NEW (thin dispatcher) | new dispatcher over POST /api/skills/starter-pack/:id, capability-packs/:id, marketplace/install (keep requireTier('PRO') for marketplace-sourced) |
marketplace.db + MarketplaceInstaller + SecurityGate + install_audit |
{ source:'starter'|'pack'|'marketplace' } → { installed }. Resolves source + delegates. |
S06 |
Skill create is the existing structured
POST /api/skills/create(skills.ts:431→generateSkillMarkdown+redactSkillContent+ audit + hash) — the Builder's real target, EXISTS. Publish reusesPOST /api/marketplace/publish(PRO). Optional later: extendSkillFrontmatterfor structured inputs/outputs/memoryAccess (Builder Steps 3–4) — open question, not in §16.8. No.mindmigration (skills are flat files; marketplace ismarketplace.db).
3c. Automation Center + Automation Builder (S11, S20) — alias cron as automations
The capability is cron (
cron.ts,/api/cron/*— full CRUD + trigger + history). "Automations" = a rename/alias surface. Zero MISSING, all PARTIAL. Register a real/api/automations/*alias plugin (PRD vocabulary) OR point the new UI at/api/cron. Trigger/condition/actions ride in the existingjob_config TEXTblob → no.mindmigration for v1.
| Method + Path | Disposition | Build target | Substrate | Shape | Screens |
|---|---|---|---|---|---|
GET /api/automations |
EXTEND (alias) | alias of GET /api/cron (cron.ts:94); reshape toResponse to expose trigger/condition/actions/status from job_type+job_config |
cron_schedules |
→ { automations:[...] }. |
S11, S20 |
POST /api/automations |
EXTEND (alias) | alias of POST /api/cron (cron.ts:67); persist trigger/condition/actions into job_config (+ cron_expr for schedule triggers) |
cron_schedules |
{ name,trigger,condition?,actions[],schedule? } → { id }. |
S11, S20 |
PATCH /api/automations/:id |
EXTEND (alias) | alias of PATCH /api/cron/:id (cron.ts:124) |
cron_schedules |
{ ...partial } → { ok }. Note real bug: FE updateCronJob calls PUT /api/cron/:id but only PATCH is registered (adapter.ts:836 vs cron.ts) — fix the adapter. |
S11, S20 |
POST /api/automations/:id/run |
EXTEND (alias) | alias of POST /api/cron/:id/trigger (cron.ts:174; auto-enables + executes + notifies) |
cron_schedules + executor (index.ts:1379) |
→ { runId }. |
S11, S20 |
POST /api/automations/:id/pause |
NET-NEW (thin) / EXTEND | add thin /pause route OR adapter calls PATCH /api/cron/:id { enabled:false } |
cron_schedules.enabled + scheduler |
→ { ok }. Add /pause for PRD contract. |
S11, S20 |
GET /api/automations/:id/logs |
EXTEND (alias) | alias of GET /api/cron/:id/history (in notifications.ts:202 → cronStore.getExecutionHistory) |
cron_execution_history |
?limit= → { logs:[...] }. |
S11, S20 |
POST /api/automations/test |
NET-NEW | new dry-run route (PRD §12.10 "test before activate"); current POST /api/cron/:id/trigger really executes |
cron executor (no-persist mode) | { trigger,actions[] } → { previewResult }. No log/notify side-effects. |
S20 |
Phase 4 — Extend layer: Connectors, MCPs, Marketplace, Install Audit (PRD §8 Phase 4 / §21 Sprint 7)
4a. Connector Hub (S07, S14) — extend connectors.ts
| Method + Path | Disposition | Build target | Substrate | Shape | Screens |
|---|---|---|---|---|---|
POST /api/connectors/:id/sync |
NET-NEW | new route in connectors.ts (WaggleConnector has connect/healthCheck/execute but no sync()) |
vault sub-key connector:<id>:lastSync (or small store) + activity event + install_audit |
→ { lastSyncAt, ok }. Phased: MVP = healthCheck() + stamp lastSyncAt + emit event; full data re-pull is a larger connector-SDK addition. No .mind migration. |
S07 |
POST /api/connectors/:id/revoke |
EXTEND (alias) | alias to POST /api/connectors/:id/disconnect (connectors.ts:107) + write install_audit action:'rejected'/revoke |
vault + install_audit |
→ { ok }. Same intent, PRD verb. |
S07 |
POST /api/connectors/:id/connect (extend) |
EXTEND | connectors.ts:55 — add auditStore.record(...) on success |
vault + install_audit |
(audit-trail enrichment, no shape change). | S07, S14 |
GET /api/connectors (extend payload) |
EXTEND (optional) | connectors.ts:6 — carry category (already on type, types.ts:299) + lastSyncAt so UI drops hardcoded CATEGORIES/sync-shim |
connector registry | (payload enrichment). | S07, S14 |
GET /api/connectors/health (aggregate) |
NET-NEW (optional, mockup) | connectors.ts — fan healthCheck() across connectors |
connector registry | → { connectors:[{id,status,lastSyncAt}], systemHealth }. Optional v1; compose client-side otherwise. |
S07 |
GET /api/connectors/activity |
NET-NEW (or use shared /api/extend/audit?type=connector) |
reads install_audit rows filtered to type:'connector' |
install_audit |
→ { activity:[...] }. Prefer the shared /api/extend/audit (4c) which serves S07+S08+S21 with one route. No migration. |
S07 |
4b. MCP Hub (S08, S17) — new mcps.ts + persisted MCP-config store + runtime population
The deepest backend gap in the Extend layer. Today
mcpRuntimeis empty and never populated (local/index.ts:911); there is noGET /api/mcps, no persisted MCP-config store, no boot-time population. All routes live in a newpackages/server/src/local/routes/mcps.ts. Substrate decision: persist installed MCP configs as a JSON file /.mcp.json(no migration) — confirm dataDir path + multi-workspace scoping (open question). The MCP catalog is static in@waggle/shared mcp-catalog.ts.
| Method + Path | Disposition | Build target | Substrate | Shape | Screens |
|---|---|---|---|---|---|
GET /api/mcps |
NET-NEW | routes/mcps.ts; join static catalog (@waggle/shared mcp-catalog.ts) + installed-state (capabilities/status.mcpServers[], .mcp.json, marketplace mcp-registry.ts) |
catalog + .mcp.json + install_audit |
→ { mcps:[{id,name,status,installed,tools[],scope}] }. Reads existing substrate; no migration. |
S08, S17 |
POST /api/mcps/install |
EXTEND | route through existing marketplace installer POST /api/marketplace/install (already handles installType:'mcp' → writes .mcp.json, installer.ts:580) |
marketplace.db + .mcp.json + install_audit |
{ mcpId } → { installed }. Resolve MCP id → marketplace package → install. Audit already recorded. |
S08 |
POST /api/mcps/:id/test |
NET-NEW | mcps.ts; resolve server, start() if needed, assert isHealthy() (mcp-runtime.ts:94,399) and/or tools/list round-trip |
McpRuntime |
→ { ok, tools[], error? }. Open question: live spawn-and-handshake vs static manifest validation. |
S08 |
POST /api/mcps/:id/revoke |
NET-NEW | mcps.ts; mcpRuntime.removeServer(name) (mcp-runtime.ts:327) + delete persisted config + install_audit action:'revoked' |
McpRuntime + .mcp.json + install_audit |
→ { ok }. |
S08 |
POST /api/mcps (add custom) |
NET-NEW | mcps.ts; persist config + add to runtime (blueprint API line 530) |
.mcp.json + McpRuntime |
{ name, command, args[], env{}, workspaceId? } → { id }. |
S08 |
POST /api/mcps/:id/start · POST /api/mcps/:id/stop |
NET-NEW | mcps.ts; map to McpRuntime start/stop (PRD §12.8 start/stop) |
McpRuntime |
→ { status }. |
S08 |
PATCH /api/mcps/:id/permissions |
NET-NEW | mcps.ts (blueprint API line 530) |
.mcp.json config |
{ scope?, permissions? } → { ok }. |
S08 |
GET /api/mcps/:id/logs |
NET-NEW (phased) | mcps.ts; needs a ring-buffer of stderr/stateChange in McpServerInstance (no log capture today) |
new in-memory ring buffer | → { logs:[...] }. Defer to later phase if log-capture infra not built. |
S08 |
Boot-time runtime population (populate
mcpRuntimefrom the persisted config at startup,local/index.ts:911) is the foundational non-route work item that unblocks all of the above.
4c. Marketplace / Extend + Install Audit (S21, shared S06/S07/S08)
| Method + Path | Disposition | Build target | Substrate | Shape | Screens |
|---|---|---|---|---|---|
GET /api/marketplace |
EXTEND (alias) | bare path = alias of GET /api/marketplace/search with default params (marketplace.ts:56) |
marketplace.db |
→ { results:[...] }. |
S21 |
GET /api/extend/audit (shared governance read) |
EXTEND (param) | the install-audit read route GET /api/audit/installs already EXISTS (skills.ts:685); add ?capability= / ?type= filter exposing getByCapability() (install-audit.ts:125) |
install_audit |
?type=skill|connector|mcp|marketplace&limit= → { entries:[AuditEntry] }. Serves S06+S07+S08+S21 with one route. No migration. Note: substrate-types §d#1 listed this as missing; it is present — the work is the filter param, not a new route. |
S06, S07, S08, S21 |
No
POST /api/sharehere — that is Phase 5 (Team). Per-workspace install scoping (§12.13) is net-new product surface; PRD §22 risk register says "start with catalog + install audit, postpone billing/public marketplace" — defer to Phase 4 polish.
Phase 5 — Team intelligence: Team Workspace, RBAC, Sharing, Audit (PRD §8 Phase 5 / §21 Sprint 8)
Team CRUD core EXISTS in
team.ts(teams.db):GET/POST/PUT/DELETE /api/teams,/:id,/members,/members/:userId,/activity. No.mindmigration for the team core (teams.dbstandalone;team_capability_policies/overrides/requestsexist in migration0001).
| Method + Path | Disposition | Build target | Substrate | Shape | Screens |
|---|---|---|---|---|---|
POST /api/teams/:id/invite |
EXTEND (alias) | thin /invite alias forwarding to POST /api/teams/:id/members (team.ts:581, owner/admin gated) |
team_members (teams.db) |
{ email?, userId?, role? } → { ok }. No new substrate. |
S10 |
GET /api/teams/:id/audit |
EXTEND (alias) | alias /audit → existing GET /api/teams/:id/activity (team.ts:690, reads audit_events via getAuditDb) |
audit_events |
→ { events:[...] }. Or add audit-export shape. No new store. |
S10 |
PATCH /api/teams/:id/members/:memberId (fix gate) |
EXTEND (bug fix) | team.ts:642 (:userId==:memberId); fix PUT(owner-only :615) vs PATCH(owner/admin :642) role-gate inconsistency |
team_members |
(behavior fix, no shape change). | S10 |
POST /api/share |
NET-NEW | new route (no /api/share anywhere — grep-confirmed); shares memory/artifact with role-appropriate perms |
workspace.json teamId linkage + (if frame-level) memory_frames metadata |
{ objectType:'memory'|'artifact'|'workspace', objectId, teamId, role } → { ok }. Open: frame-level scope needs the memory_frames metadata migration (§M); v1 may scope implicitly via workspace teamId. Gate behind TEAMS tier like /api/team/*. |
S05, S10 |
POST /api/artifacts/:id/share |
NET-NEW (blueprint) | maps to POST /api/share + artifact scope; gate TEAMS |
artifacts.json + team scope |
{ teamId, role } → { ok }. Blueprint action not in §16.6; defer to Phase 5. |
S05 |
GET /api/teams/:id/governance (optional) |
NET-NEW (optional) | surface team_capability_policies/overrides/requests (migration 0001); Enterprise proxy GET /api/team/governance/permissions (team.ts:418) is the remote analog |
team_capability_policies etc. |
→ { policies[], overrides[], requests[] }. RBAC UI. |
S10 |
§M — Schema migrations required
Headline: the entire refactor needs AT MOST ONE conditional SQLite migration, and it is deferrable. Every other "schema addition" is to a JSON file (
workspace.json,profile.json,agents.json,artifacts.json,.mcp.json) — not a database — so it is a pure additive TypeScript-interface change with no migration. The migration runner already does idempotent additiveADD COLUMNonmemory_frames(precedent: it addedsource,mind/db.ts:116-124; pattern =pragma_table_infoguard +ALTER TABLE … ADD COLUMN).
M1 — memory_frames.metadata (CONDITIONAL — Phase 2/Phase 5) — the only .mind SQLite migration
- What: add one nullable column
metadata TEXT NOT NULL DEFAULT '{}'tomemory_frames(mind/schema.ts:47). Store{kind, title, scope, sourceId, sourceUrl, confidence, tags, evidence, relatedMemoryIds, relatedArtifactIds, status}as JSON (PRD §15.4 fields; PRD endorses metadata-first,:1013). - Why conditional:
memory_framesis the only mind table without a JSON blob column (unlikeawareness.metadata,knowledge_entities.properties, etc.). Needed ONLY when persisted confidence/provenance/scope/status becomes a real query/filter axis (S04 Memory Center filters; S16 persisted-confidence review; S10/S05 frame-level/api/sharescope). NOT needed if S16 review is preview-only (pre-commit) and S04 filtering is in-app over the existing columns. - Risk: low — single additive nullable column; avoids touching the FTS/vec virtual tables and IPB
scoring. Promotion path: if
confidencebecomes a primary indexed filter, a later migration addsconfidence REALas a real column (same ADD-COLUMN pattern). - Screens: S04, S16, S10, S05. Build target:
packages/hive-mind-core/src/mind/db.tsmigration block +mind/schema.ts.
M2 — install_audit risk-level CHECK fix (RECOMMENDED — pre-Phase 4, latent bug)
- What: the TS
AuditRiskLevelincludes'critical'(install-audit.ts:16) but both DDL CHECK constraints allow only('low','medium','high')(install-audit.ts:65ANDschema.ts:130— duplicated DDL that must stay in sync). Arecord({riskLevel:'critical'})throws a CHECK violation. - Why: Phase 4 connector/MCP/skill installs all route through
auditStore.record(...). The marketplace route currently side-steps by mapping CRITICAL→riskLevel:'high'+approvalClass:'blocked'(marketplace.ts:224-319) — but any new Extend caller passing'critical'crashes. - Fix (pick one): (a) widen both CHECK constraints to include
'critical'(additive CHECK migration — needs table rebuild for SQLite CHECK change, or relax to no-CHECK), OR (b) lock the CRITICAL→'high'mapping as the permanent contract and drop'critical'from the TS union. (b) is zero-migration. - Risk: low. Screens: S06, S07, S08, S21 (all Extend installs). Build target:
install-audit.ts:65+mind/schema.ts:130(kept in sync) OR the TS union.
M3 — agents table (OPTIONAL — Phase 3, NOT recommended for v1)
- What: an
agentstable inmind/schema.tswith aSCHEMA_VERSIONbump. - Recommendation: do NOT do this for v1. Persist agents to
{dataDir}/agents.json(file store, mirrors theagent-groups.jsonprecedentagent-groups.ts:29) — no migration, reversible. Only add the table if agents must be FTS/relation-queryable. PRD §14.4 non-goal favors minimal backend. - Screens: S09, S18.
Non-migrations (additive JSON-file / interface changes only — listed for completeness, NOT migrations)
WorkspaceConfigV2 fields →workspace.json(Phase 0; substrate-types §a).UserProfileworkType/teamSize/goals→profile.json(S13).Agententity →agents.json(Phase 3, M3 alt).Artifactindex →artifacts.json(Phase 2b).- MCP installed configs →
.mcp.json(Phase 4b). - Automation
trigger/condition/actions→ existingjob_config TEXTblob (Phase 3c; no schema change). cron_schedules/cron_execution_history/notificationstables already exist with lazy creation (cron-store.ts:135-157) — no migration for Automations.
Counts
Counted as distinct backend endpoints (each method+path = 1). Endpoints that EXIST as-is and need only frontend wiring are excluded. Phase-0 non-route work (V2 fields, FE type unions, write- side stamps) is counted separately under "interface/field extensions", not as endpoints.
- Total endpoints requiring backend work: 53 (NET-NEW + EXTEND, de-duplicated).
- NET-NEW endpoints: 35
- Home ×2 (
/home/briefing,/home/overnight) - Command ×3 (
/command/search,/command/recent,/command/suggestions) - Memory ×3 (
/memory/:id,/memory/:id/archive,/memory/merge) - Artifacts ×6 (GET, POST,
/:id, PATCH/:id, DELETE/:id,/search-related) - Agents ×5 (GET, POST,
/:id, PATCH/:id,/:id/traces) - Skills ×1 (
/skills/:id/install) - Automations ×1 (
/automations/test) - Connectors ×3 (
/:id/sync,/connectors/health,/connectors/activity) - MCPs ×8 (
GET /mcps,/:id/test,/:id/revoke,POST /mcpscustom,/:id/start,/:id/stop,PATCH /:id/permissions,/:id/logs) - Team ×3 (
POST /api/share,POST /artifacts/:id/share,GET /teams/:id/governance) - (Several MCP/connector/team items are blueprint-implied beyond the §16 literal list;
/api/automations/:id/pauseis counted under EXTEND as a thin alias over the cronenabledflag.)
- Home ×2 (
- EXTEND endpoints: 18 (distinct backend touch-points; an EXTEND may be an alias, an added param, or added behavior)
/quick-capture(delegates to memory write)/workspaces/:id/state,/workspaces/:id/activity(thin routes over existing builders/events)/command/execute(broaden dispatch)/memoryGET,/memoryPOST,/memory/:idPATCH,/memory/:idDELETE (aliases over/memory/frames*)/harvest/preview+/harvest/commit(confidence +selectedIdsselection)/harvest/sources/:id/sync(thin re-commit alias)/workspacesPOST (richer body)/agents/:id/run,/agents/:id/pause(delegate to fleet)/skills/:idPATCH,/skills/:id/test(:idvariants)- 6×
/automations/*aliases over/cron/*(GET, POST, PATCH, run, pause, logs) /connectors/:id/revoke,/connectors/:id/connect(+audit),/connectorsGET (payload)/mcps/install(via marketplace installer)/marketplace(bare-path alias)/extend/auditfilter param (over existing/audit/installs)/teams/:id/invite,/teams/:id/audit,/teams/:id/members/:memberId(alias + role-gate fix)
- Phase-0 interface/field extensions (NOT endpoints): 5 —
WorkspaceConfigV2 fields +updatedAt/lastActiveAtwrite-stamps (workspace-manager.ts), FE type unions (types.ts),UserProfileworkType/teamSize/goals(profile.ts),WorkspaceTemplateshape,Connectorinterface fields. - PRD §16 cross-reference (from
_inventory/backend-routes.md): of 65 §16-literal endpoints — 16 EXIST as-is (FE wiring only), 30 PARTIAL (→ EXTEND), 19 MISSING (→ NET-NEW). This master list adds ~16 blueprint-implied endpoints (MCP start/stop/logs/permissions/custom, connector health/activity, automations/test, team governance, artifact-share, extend/audit) beyond the §16 literal set. - Schema migrations: 1 conditional SQLite (M1
memory_frames.metadata) + 1 recommended CHECK fix (M2install_auditrisk-level) + 1 optional/deferred (M3agentstable — recommend NOT doing in v1). Net likely-to-ship: 1 (memory_frames.metadata); 0 strictly required if S16 confidence stays preview-only and/api/sharescopes implicitly via workspaceteamId. - New sidecar route files: 5 (
home.ts,command.ts,artifacts.ts,agents.ts,mcps.ts)- 1 alias plugin (
automations.ts→ cron). New JSON file stores: 2 (agents.json,artifacts.json); plus reuse of.mcp.json,workspace.json,profile.json.
- 1 alias plugin (