This commit is contained in:
Oleg Maslov
305
docs/superpowers/specs/2026-05-23-waggle-os-ux-design.md
Normal file
305
docs/superpowers/specs/2026-05-23-waggle-os-ux-design.md
Normal file
@@ -0,0 +1,305 @@
|
||||
# Waggle OS — UX Design v1
|
||||
|
||||
**Date:** 2026-05-23
|
||||
**Status:** Brainstorm output. Vision spec. Pick what to ship, defer the rest.
|
||||
**Anchor metaphor:** Refined Desktop OS (A) with Honeycomb grafts (B) where signal-flow / cells / hive-amber tell the story better than a list or grid.
|
||||
**Inputs:** existing UI surface (24 apps + 15 overlays in `apps/web/src/components/os/`), Hive DS tokens (`honey #e5a000` / `hive-950 #08090c` / `accent #a78bfa`), 8-cluster brainstorm + metaphor confirmation 2026-05-23.
|
||||
|
||||
> Effort tiers used below: **S** = ≤3 days, **M** = ≤2 weeks, **L** = multi-week. Tiers are best-guess and assume one focused engineer + Marko driving design review.
|
||||
|
||||
---
|
||||
|
||||
## 0. The anchor metaphor (and what it isn't)
|
||||
|
||||
Waggle stays a **desktop OS** in chrome: top status bar, bottom dock, free-floating `AppWindow` instances, overlays as modals/sheets. The familiar mental model is preserved (no learning curve) and the existing six months of `Desktop.tsx` / `Dock.tsx` / `AppWindow.tsx` investment is preserved.
|
||||
|
||||
The **honeycomb** is grafted in eight specific places — where the visual metaphor tells the story better than its alternative:
|
||||
|
||||
| # | Graft | Where it lives | Why hex beats the alternative |
|
||||
|---|---|---|---|
|
||||
| 1 | Dock items as hex tiles | `Dock.tsx` | A live "bee tile that glows when an agent is dancing" is the Waggle brand. Drop-in skin. |
|
||||
| 2 | Status-bar crew presence | `StatusBar.tsx` | Tiny hex avatars showing who's working RIGHT NOW. Click → jump-to-window. |
|
||||
| 3 | WaggleDance as literal honeycomb | `WaggleDanceApp.tsx` | Signal flow ON THE EDGES is the original honeybee dance — what was the metaphor *for* in the first place. |
|
||||
| 4 | Mission Control hex grid | `MissionControlApp.tsx` | Agents-as-cells, glow=working, hover=status. Beats a list of "agent #3 / agent #7". |
|
||||
| 5 | Spawn-agent persona tiles | `SpawnAgentDialog.tsx` / `PersonaSwitcher.tsx` | Persona = a "role-cell." Tile feel works. |
|
||||
| 6 | MCP catalog as hex grid | `MarketplaceApp.tsx` / `connectors/BrandTile.tsx` | Connectors are nodes in a graph the user is building. Hex grid > rectangular tile grid. |
|
||||
| 7 | Memory peripheral inspector | New right-rail in `ChatApp.tsx` | Hex mini-graph of what was recalled for this turn. "Why did I get this answer?" answerable at a glance. |
|
||||
| 8 | Honey-amber "active" accent across the OS | `waggle-theme.css` | Universal: `honey` = live/working, `violet` = memory-touched, neutral = idle. Color semantics unified. |
|
||||
|
||||
**Stays as plain Desktop:** free-floating windows (parallelism preserved), `AppWindow` chrome, dialogs as dialogs, settings as a form. The OS does not force-cell everything — it cells the things **about agents and signal**, which is where the metaphor earns rent.
|
||||
|
||||
---
|
||||
|
||||
## 1. Spatial chrome & window choreography
|
||||
|
||||
**Today (in `components/os/`):** `Desktop.tsx` (551 LOC), `Dock.tsx` (170 LOC), `DockTray.tsx`, `AppWindow.tsx`, `BootScreen.tsx`, `StatusBar.tsx`, `WorkspaceBriefing.tsx`, `ContextMenu.tsx`. Free-floating windows with z-order management. Bottom dock with app icons. Top status bar.
|
||||
|
||||
**What's missing for "OS-feel":**
|
||||
- No **snap zones / window tiling** — a real OS lets you halve, quadrant, full-screen via drag or keyboard.
|
||||
- No **multi-monitor** support (Tauri supports it; the web shell doesn't address it).
|
||||
- No **focus modes** — "agents working, leave me alone" / "I'm in a presentation" / "deep work" presets that dim/hide ambient surfaces.
|
||||
- No **virtual desktops** that aren't workspaces — sometimes you want two views of the same workspace (chat + WaggleDance vs Files + Wiki) without swapping context.
|
||||
- No **ambient "agent is running" peripheral indicator** — the dock bee icon doesn't pulse, the window edge doesn't honey-glow.
|
||||
- No **OS-level transitions** between scenes (snap, slide, zoom-into-app).
|
||||
|
||||
**Recommended adds:**
|
||||
1. **Hex dock skin** + active-cell honey glow + agent-progress ring around the active tile **[S]**. (Visible foundation for grafts 1+8 above.)
|
||||
2. **Snap zones** (drag window to screen edge → halve/quadrant) using existing `AppWindow` resize handles **[S]**.
|
||||
3. **Focus modes** — three presets in `StatusBar` ("Working / Focus / Presenting") that toggle right-rail visibility, dock animation, notification stream **[M]**.
|
||||
4. **Pulse + glow when an agent in this window is doing work** — animated honey-amber border on `AppWindow`, decay-out when idle **[S]**.
|
||||
5. **Virtual sub-views per workspace** (later, after cluster 5 — workspaces-as-scenes — lands) **[M]**.
|
||||
6. **Multi-monitor support** via Tauri window-detach (open a window in a separate Tauri WebView on a second display) **[L]**.
|
||||
|
||||
---
|
||||
|
||||
## 2. The command surface — the omnibar
|
||||
|
||||
**Today:** `GlobalSearch.tsx` overlay + `LauncherApp.tsx`. Search is search-only. Launcher is a tool-launching surface. Neither is a true command palette.
|
||||
|
||||
**What's missing:** Marko's OS has 24+ apps, hundreds of memory frames, multiple agents, dozens of skills, and a marketplace. There is no single keystroke that gets you to anything. The most-used surface in any modern OS-of-AI (Raycast, Spotlight, Linear ⌘K) is the **fuzzy-routed omnibar**.
|
||||
|
||||
**Recommended:** ship **⌘K Omnibar** as the universal entry point:
|
||||
|
||||
| Verb prefix | What it does |
|
||||
|---|---|
|
||||
| (no prefix) | Fuzzy search across apps, recent windows, files, memory frames, skills |
|
||||
| `>` | Run a command (open app, switch workspace, spawn agent, toggle focus mode) |
|
||||
| `@` | Ask an agent — picks the right persona or routes to the active window's chat |
|
||||
| `?` | Search memory ("what do I know about X") + inline answer card |
|
||||
| `#` | Jump to skill / connector |
|
||||
| `/` | Insert a slash-command into the active chat window |
|
||||
|
||||
Implementation moves:
|
||||
1. **Build the omnibar shell** in `components/os/overlays/Omnibar.tsx`, hotkey `Ctrl/Cmd+K` (free up `GlobalSearch.tsx` for retirement or convert to the `?` namespace inside it) **[S]**.
|
||||
2. **Wire the verb routers** — start with no-prefix + `>` + `?` (highest leverage), defer `@` / `#` / `/` to Phase 2 **[S]**.
|
||||
3. **Persistent omnibar pill** in the status bar — always-visible `⌘K · ask anything` — beats a hidden shortcut for discoverability **[S]**.
|
||||
4. **Fuzzy index** of (apps, workspaces, recent files, top-N memory frames, skills) refreshed on workspace-switch — sub-100ms response **[S/M]**.
|
||||
|
||||
---
|
||||
|
||||
## 3. Agent visibility & multi-agent choreography
|
||||
|
||||
**Today:** `AgentsApp.tsx`, `MissionControlApp.tsx`, `WaggleDanceApp.tsx`, `SpawnAgentDialog.tsx`. You can spawn, see, and run agents. You can observe WaggleDance signals.
|
||||
|
||||
**What's missing — the live picture:**
|
||||
- **No status-bar presence layer.** When 3 agents are working you don't see it from the chrome.
|
||||
- **No "is my agent stuck?" glance state.** You have to open MissionControl to know.
|
||||
- **WaggleDance is a list, not a dance.** The literal honeycomb-with-signal-edges is the missing payoff.
|
||||
- **No interrupt / hand-off / "stop and ask me" model** — agents either run autonomously or you cancel.
|
||||
- **No agent-to-agent comm visualization.** When the orchestrator delegates to a sub-agent, you have to read logs.
|
||||
|
||||
**Recommended adds:**
|
||||
1. **Status-bar crew presence row** — tiny hex avatars per running agent, honey-glow = working, violet-pulse = waiting on memory, red = stuck/errored. Click → jump-to-window. **[S]**
|
||||
2. **Convert `MissionControlApp` to hex-grid view** — every running agent is a hex cell; cell color = state; hover = current tool; click = drill in. List view stays as a sub-tab. **[M]**
|
||||
3. **Make `WaggleDanceApp` an actual honeycomb canvas** with signal-flow on edges (animated path when one agent broadcasts to another). This is where graft #3 finally pays. **[M]**
|
||||
4. **Interrupt control on every running-agent window** — `Pause / Resume / Ask me / Cancel`. The "Ask me" puts the agent into the Approvals app and shows a notification. **[S]** (already partially wired via Approvals; surface the controls.)
|
||||
5. **Hand-off UI in chat** — when persona A is wrong for a question, an inline card "Pass to persona B?" with a one-click migrate. **[M]**
|
||||
6. **Agent-to-agent comm trace** — when a parent spawns a child, draw the edge in WaggleDance and link the windows; child window has a back-arrow to parent. **[S]**
|
||||
|
||||
---
|
||||
|
||||
## 4. Memory as first-class fabric
|
||||
|
||||
**Today:** `MemoryApp.tsx` with five tabs (Wiki / KnowledgeGraph / Harvest / Weaver / Evolution). Memory is **a destination** — you go to the Memory app to look at it.
|
||||
|
||||
**What's missing — memory as fabric:**
|
||||
- **No peripheral memory inspector in chat.** Every chat turn fetches memory; the user can't see what was retrieved.
|
||||
- **No "what does Waggle know about X" inspector** at the OS level — pin a noun, see the graph.
|
||||
- **No provenance overlay** — when an agent answers, you can't see which frames backed each claim.
|
||||
- **No time-travel** — "what did my memory look like last week?" is not a thing.
|
||||
- **Memory isn't a verb** in the omnibar (cluster 2 fixes this).
|
||||
|
||||
**Recommended adds (this is your moat — bias toward shipping):**
|
||||
1. **Right-rail memory peripheral inspector in `ChatApp`** — hex mini-graph showing which frames were recalled for the current turn, with hover-to-preview. Default-collapsed so it doesn't intrude. **[M]**
|
||||
2. **Provenance citations in agent answers** — small `[1]` markers in the response, hover = the actual frame. Wire to the existing `KnowledgeGraphViewer`. **[M]**
|
||||
3. **Memory inspector overlay** — pin a noun anywhere (selection menu → "What does Waggle know about this?") → mini-overlay with related frames + entity + concept. **[M]**
|
||||
4. **Time-travel slider in MemoryApp** — a horizontal scrubber across the top: "show memory as of 2026-04-15". Backed by `bitemporal validity` already in `knowledge.ts`. **[M]**
|
||||
5. **"Memory health" tile in Dashboard** — frames added today, stale clusters, dedup opportunities. Already partially in HarvestTab; promote to surface-level. **[S]**
|
||||
|
||||
---
|
||||
|
||||
## 5. Workspaces as scenes (not folders)
|
||||
|
||||
**Today:** `WorkspaceSwitcher.tsx` overlay + `WorkspaceRail.tsx` in `FilesApp`. Workspaces hold files and memory frames. They're filing cabinets.
|
||||
|
||||
**What's missing — workspaces as desktops:**
|
||||
- **No persisted window layout per workspace** — switching workspaces resets the spatial arrangement.
|
||||
- **No pinned dock per workspace** — same dock everywhere; you can't say "my Research workspace pins Wiki + Memory + Chat, my Founder workspace pins Cockpit + Compliance + Approvals."
|
||||
- **No assigned crew per workspace** — same personas everywhere; no "this workspace's agents are X, Y, Z."
|
||||
- **No memory scope per workspace** — you have to manually filter.
|
||||
- **No autonomy tier per workspace** — Normal/Trusted/YOLO is global, but a "throwaway research" workspace wants YOLO while "production deploy" wants Trusted.
|
||||
|
||||
**Recommended adds (this is where workspaces become *scenes*):**
|
||||
1. **Persist window-arrangement on workspace-switch** — when you leave a workspace, snapshot which apps were open + their positions; restore on return. **[S]**
|
||||
2. **Per-workspace dock pinning** — workspace config stores `pinnedApps: string[]`; the dock renders the workspace's set first, then the global apps. **[S]**
|
||||
3. **Workspace crew** — when you spawn an agent in a workspace, it sticks; the workspace stores `crew: AgentDef[]` and surfaces them in MissionControl + the status-bar presence row. **[M]**
|
||||
4. **Per-workspace memory scope** as the default for in-workspace search (existing search likely supports it; promote to the UX). **[S]**
|
||||
5. **Per-workspace autonomy tier** — `WorkspaceConfig.autonomy: Normal | Trusted | YOLO`; status-bar shows the active tier with a visible badge. **[S]**
|
||||
6. **Workspace gallery overview** — when no workspace is active, show a hex grid of workspaces with thumbnails. New users see this; advanced users skip via `Cmd+1..9`. **[M]**
|
||||
|
||||
---
|
||||
|
||||
## 6. Input modalities & ambient capture
|
||||
|
||||
**Today:** `ChatApp` (text) + `VoiceApp` (voice). Both are destinations you go to.
|
||||
|
||||
**What's missing — ambient input:**
|
||||
- **No always-listening hotkey** — hold `Cmd+Space` and talk, release to send. Doesn't require switching to VoiceApp.
|
||||
- **No drag-anywhere file-routing** — drag a PDF onto the desktop; the OS asks "send to Files / ask agent about it / harvest into memory?"
|
||||
- **No screenshot-to-agent** — `Cmd+Shift+4`, drag a region, the screenshot lands in the active chat or routes to an agent.
|
||||
- **No QR pull-in from mobile** — Waggle doesn't have a real mobile companion, but a QR-scan-to-open-on-Waggle-OS pattern is achievable.
|
||||
- **No "share my screen with my agent"** — for agents that benefit from visual context (UI debugging, design feedback).
|
||||
|
||||
**Recommended adds:**
|
||||
1. **Push-to-talk hotkey** with floating overlay + waveform during capture + auto-route to active window's input or active chat. **[M]**
|
||||
2. **Drag-to-desktop drop-target** with route picker (Files / Chat / Harvest / Skill). **[M]**
|
||||
3. **Screenshot-to-agent** via Tauri global shortcut → image lands in active chat as attachment. **[M]**
|
||||
4. **Mobile QR-handoff** — QR shown in StatusBar, mobile scans, message sent shows up in your active chat (no native app needed; web page + WebSocket). **[L]**
|
||||
5. **Screen-share to agent** — Tauri can grab a screen region; pipe to a vision-capable model in the active window. Tier-gated (Pro+). **[L]**
|
||||
|
||||
---
|
||||
|
||||
## 7. Trust, autonomy & approval choreography
|
||||
|
||||
**Today:** `ApprovalsApp.tsx`, `VaultApp.tsx`, `CapabilitiesApp.tsx`, three-tier autonomy (`Normal / Trusted / YOLO`). Approvals open as modals. Audit trail is a list.
|
||||
|
||||
**What's missing — approvals as peripheral, not modal:**
|
||||
- **Approvals interrupt your work** — modal blocks the active task instead of queueing peripherally.
|
||||
- **No autonomy dial visible per agent** — the tier is global; the user can't see "this agent runs Trusted, this one Normal."
|
||||
- **No "why is this agent doing this?" surface** — when an action surprises the user, there's no one-click "show me the chain of reasoning."
|
||||
- **Audit trail is a flat log** — should be a scrubable replay (timeline of tool calls + memory hits + outputs).
|
||||
|
||||
**Recommended adds:**
|
||||
1. **Approval inbox in the status bar** — pending approvals = a small honey-amber pip with a count; click expands a popover instead of a modal. Block-mode is opt-in per autonomy tier (YOLO never blocks, Normal blocks on critical only). **[S]**
|
||||
2. **Autonomy dial visible per agent** in MissionControl + agent window header. Click to change scope (this run / this session / this workspace / global). **[S]**
|
||||
3. **"Why?" button on every agent action** — opens a side-panel showing the prompt, the tool call, the memory frames retrieved, the reasoning trace. Wire to existing `ExecutionTraces`. **[M]**
|
||||
4. **Audit replay** in `TimelineApp` — a video-scrubber-style timeline of agent actions; play/pause/seek. Pulls from existing trace store. **[M]**
|
||||
5. **"This action is the kind of thing this agent does at Trusted tier" hint** — when an action triggers approval, show the autonomy threshold and a one-click "raise this agent to Trusted." **[S]**
|
||||
|
||||
---
|
||||
|
||||
## 8. Discoverability — the OS teaches you
|
||||
|
||||
**Today:** `OnboardingWizard.tsx` (one-shot, 8 steps) + `OnboardingTooltips.tsx` (post-onboarding hints). Both are first-run; they go silent after.
|
||||
|
||||
**What's missing — an OS keeps teaching:**
|
||||
- **No tip-of-the-day surface** — users plateau at the 10 features they discovered week 1.
|
||||
- **No "did you know your agent can…" prompts** — based on what the user just did.
|
||||
- **No watching TUTOR persona** — surfaces a capability the user almost discovered but didn't.
|
||||
- **No skill / connector recommendations** based on usage patterns.
|
||||
- **No "you haven't tried X in 30 days, here's what's new" reactivation.**
|
||||
|
||||
**Recommended adds:**
|
||||
1. **Tip-of-the-day in the status bar** — small honey-amber pip, click expands. One tip per day, dismissible permanently. Indexed by user action (don't show "spawn an agent" to a user who spawned one 5 minutes ago). **[S]**
|
||||
2. **Capability recommender after every long-running task** — "this agent ran 12 tools — want me to distil it into a skill?" Already partially exists via D1 distillation; surface the prompt to the user. **[S]**
|
||||
3. **TUTOR persona** as a 17th persona — read-only, watches recent user actions + memory + workspace state, periodically surfaces "I noticed you searched for X three times — here's a saved search." Off by default; opt-in. **[L]**
|
||||
4. **Connector / skill recommender in the MarketplaceApp** — based on workspace topic + recent harvest, suggest 3 connectors. **[M]**
|
||||
5. **Reactivation banner** for stale workspaces ("It's been 30 days since you opened 'Q1 Research' — here's what's changed: 12 new memory frames, 2 closed agents."). **[M]**
|
||||
|
||||
---
|
||||
|
||||
## 9. Cross-cutting design principles
|
||||
|
||||
These tie the 8 clusters together. They are the design language; everything new should obey them.
|
||||
|
||||
| Principle | Concrete rule |
|
||||
|---|---|
|
||||
| **Color semantics** | `honey #e5a000` = live/working/active; `accent #a78bfa` = memory-touched/recalled; neutral = idle. NO arbitrary color use. |
|
||||
| **Peripheral-first** | Information about *agents working* / *memory in play* / *signals firing* lives in the right rail or status bar, not in modal interruptions. |
|
||||
| **Motion = signal** | Animation is reserved for actual signal (agent active, memory hit, hand-off). Static UI = idle UI. No decorative motion. |
|
||||
| **One omnibar, one entry** | `Cmd+K` is the universal entry point. Every app should be reachable through it. |
|
||||
| **Hex where it's a signal-node, rectangle where it's content** | Cells of the OS (agents, dock items, personas, connectors) = hex. Content (text, files, dialogs) = rectangle. Never both for the same concept. |
|
||||
| **Approvals are peripheral by default** | The OS interrupts only at YOLO-violations or user-explicit "approve me when..." rules. |
|
||||
| **Memory is a verb everywhere** | `?` prefix in omnibar; `What do I know about X` from any selection; provenance on every agent answer. |
|
||||
| **Workspaces are scenes, not folders** | Every workspace has its own dock, crew, autonomy tier, layout. |
|
||||
|
||||
---
|
||||
|
||||
## 10. Phased rollout suggestion
|
||||
|
||||
The 8 clusters compound. The right order is **the OS-feel pass first** (visible coherence + new omnibar + honeycomb grafts), **agent visibility + memory fabric second** (the differentiation moat), **scenes + ambient + tutor third** (the deepening).
|
||||
|
||||
### Phase 1 — The OS-feel pass (2–3 weeks)
|
||||
Everything **S** that establishes the visual + interaction language:
|
||||
- Hex dock skin + honey-active accent ⟶ §1.1
|
||||
- Window snap zones ⟶ §1.2
|
||||
- Agent-window honey-pulse on active work ⟶ §1.4
|
||||
- ⌘K Omnibar with no-prefix + `>` + `?` verbs ⟶ §2.1–2.3
|
||||
- Status-bar crew presence row ⟶ §3.1
|
||||
- Memory health tile in Dashboard ⟶ §4.5
|
||||
- Per-workspace dock pinning + autonomy tier ⟶ §5.2 + 5.5
|
||||
- Tip-of-the-day in status bar ⟶ §8.1
|
||||
- Approval inbox in status bar ⟶ §7.1
|
||||
- Autonomy dial per agent ⟶ §7.2
|
||||
|
||||
**Outcome:** the OS *feels* like an OS. Differentiation is visible. No new agents, no new pipelines.
|
||||
|
||||
### Phase 2 — The differentiation moat (4–6 weeks)
|
||||
The **M** items that lean into Waggle's unique surface (memory + multi-agent):
|
||||
- WaggleDance as literal honeycomb canvas ⟶ §3.3
|
||||
- Mission Control hex-grid view ⟶ §3.2
|
||||
- Memory peripheral inspector in ChatApp ⟶ §4.1
|
||||
- Provenance citations in agent answers ⟶ §4.2
|
||||
- Memory inspector overlay (pin-a-noun) ⟶ §4.3
|
||||
- Time-travel slider in MemoryApp ⟶ §4.4
|
||||
- "Why?" button + audit replay ⟶ §7.3 + 7.4
|
||||
- Hand-off UI in chat ⟶ §3.5
|
||||
- Workspace crew + memory scope + scene restoration ⟶ §5.1, 5.3, 5.4
|
||||
- Focus modes ⟶ §1.3
|
||||
|
||||
**Outcome:** the OS-of-AI moat is visible — memory as fabric, agents as a coordinated swarm, workspaces as scenes.
|
||||
|
||||
### Phase 3 — The deepening (multi-week, prioritize when Phase 2 is in users' hands)
|
||||
The **L** items:
|
||||
- Multi-monitor support ⟶ §1.6
|
||||
- Push-to-talk + drag-to-desktop + screenshot-to-agent ⟶ §6.1–6.3
|
||||
- Mobile QR-handoff ⟶ §6.4
|
||||
- Screen-share to agent ⟶ §6.5
|
||||
- TUTOR persona ⟶ §8.3
|
||||
- Reactivation banners ⟶ §8.5
|
||||
|
||||
---
|
||||
|
||||
## 11. Open questions / decisions Marko owns
|
||||
|
||||
These are real decisions that affect the build; none have been settled in this brainstorm.
|
||||
|
||||
1. **TUTOR persona placement** — is it a 17th persona in `persona-data.ts`, a system service that any persona can speak through, or an explicit `tutor: true` flag on existing personas (so Research Researcher can have a tutor mode)?
|
||||
2. **Always-listening privacy posture** — push-to-talk only (safe default), or always-listening behind a per-workspace opt-in? Audit/UI implications.
|
||||
3. **Scene workspaces migration** — do existing workspaces become scenes automatically (lift their last window arrangement) or do users opt-in workspace-by-workspace?
|
||||
4. **Approval inbox vs modal threshold** — what's the default for Normal tier? Currently inferred: modal on credential-touching + writes-outside-workspace, peripheral on everything else.
|
||||
5. **Honey-amber agent-active glow intensity** — light pulse OK, but an agent that's been running for 30 minutes shouldn't strobe forever. Decay rule? (Suggestion: glow for first 30s, then steady-amber dot in the window header.)
|
||||
6. **Free-floating windows vs forced tiling** — Phase 1 ships snap zones; does Phase 2 add a tile-everything mode for users who want it, or stay free-floating?
|
||||
7. **Right-rail visibility default** — peripheral memory inspector default open or default closed? Opens-on-first-recall might be the right answer.
|
||||
8. **Mobile companion form-factor** — is it a separate Tauri mobile build, a PWA, or just a web page with WebSocket? (Affects §6.4 effort.)
|
||||
|
||||
---
|
||||
|
||||
## 12. Implementation notes (deferred — for writing-plans)
|
||||
|
||||
Cross-references for the implementation plan author:
|
||||
|
||||
- **Dock skin** — `apps/web/src/components/os/Dock.tsx`, `apps/web/src/lib/persona-tier.ts` for tier styling already exists.
|
||||
- **Omnibar** — net-new file `apps/web/src/components/os/overlays/Omnibar.tsx`; retire or namespace `GlobalSearch.tsx`.
|
||||
- **Status-bar crew presence** — extend `StatusBar.tsx`; data source = WaggleDance signal bus + running-agent registry.
|
||||
- **WaggleDance honeycomb** — `apps/web/src/components/os/apps/WaggleDanceApp.tsx`; signal-flow data already streams from the v2 bus (`packages/server/src/local/routes/waggle-dance.ts`).
|
||||
- **Memory peripheral inspector** — `apps/web/src/components/os/apps/ChatApp.tsx` + memory recall events from `packages/agent/src/orchestrator.ts` `recallMemory()`.
|
||||
- **Workspace scenes** — `packages/core/src/workspace-config.ts` extends with `pinnedApps`, `crew`, `autonomy`, `lastLayout`.
|
||||
- **Time-travel slider** — `packages/core/src/mind/knowledge.ts` already has bitemporal validity (`valid_from` / `valid_to`); UI is the missing piece.
|
||||
|
||||
Each implementation plan will need its own tsconfig project verification (`packages/agent/tsconfig.json`, `apps/web/tsconfig.json`) and a Vitest pass.
|
||||
|
||||
---
|
||||
|
||||
## 13. What this design doesn't try to do
|
||||
|
||||
- It does not propose a redesign of the **landing page** (`apps/www/`) — explicitly out of scope per the brainstorm.
|
||||
- It does not change **billing/tiers** — Stripe wiring is settled.
|
||||
- It does not change **the agent loop** — that's the engine; this is the UI.
|
||||
- It does not propose **a new design system** — Hive DS stays; honey/violet/hive-950 stay; we just *use them more consistently*.
|
||||
- It does not propose **mobile-native ports** — §6.4 is a companion, not a port.
|
||||
- It does not address **the KVARK on-prem enterprise surface** — that's a separate workstream.
|
||||
|
||||
---
|
||||
|
||||
*End of design doc. Awaiting review.*
|
||||
@@ -0,0 +1,209 @@
|
||||
# Design — Hermes opportunistic time-gated compact-on-Stop (OQ-4)
|
||||
|
||||
**Date:** 2026-06-01
|
||||
**Status:** Approved (brainstorming → spec)
|
||||
**Scope:** `packages/hive-mind-hooks-hermes` only. `@waggle/hive-mind-hooks-core` is **not** modified.
|
||||
**Closes:** OQ-4 (handoff `project_session_handoff_0601_s1.md` §"What's still open" → P2) and the
|
||||
CLAUDE.md §10 residual "hermes has no PreCompact" gap.
|
||||
|
||||
---
|
||||
|
||||
## 1. Problem
|
||||
|
||||
Hermes (NousResearch) ships **no PreCompact lifecycle event** — confirmed absent in source
|
||||
(`adapter.ts` sets `HERMES_EVENT_NAME['pre-compact'] = undefined`; `paths.ts` omits the basename).
|
||||
Every other built hook package binds PreCompact to `bridge.cleanupFrames()` (the
|
||||
`cleanup_frames` MCP tool, `mode:'compact'`) so that, before the host truncates context, the
|
||||
mind runs a maintenance pass: prune expired **temporary** frames + reconcile superseded ones.
|
||||
|
||||
Consequence for a hermes CLI user: **that maintenance never runs.** Temporary capture frames
|
||||
(every UserPromptSubmit saves one) accumulate unbounded in their `.mind`. The Wave-2/3 hermes
|
||||
package shipped this as a documented gap (`adapter.ts`: *"the gap is documented, never
|
||||
approximated"*). OQ-4 is the optional follow-up that closes it.
|
||||
|
||||
## 2. Goal / non-goals
|
||||
|
||||
**Goal:** give hermes a faithful, low-cost approximation of "occasional before-compaction
|
||||
maintenance" by running `cleanupFrames()` opportunistically from the per-turn Stop hook —
|
||||
**opt-in, default off**, so OSS consumers see zero behavior change unless they ask for it.
|
||||
|
||||
**Non-goals:** changing the shared `runStopBody`; touching any other tool package; adding a new
|
||||
upstream MCP tool; per-workspace gating (hermes hooks target the personal mind — a single global
|
||||
gate is correct); reconstructing a true PreCompact signal (Hermes has none — this is an
|
||||
approximation, labelled as such).
|
||||
|
||||
## 3. Core constraint — statelessness
|
||||
|
||||
Hermes' Stop is `post_llm_call`, delivered to a **fresh Node subprocess every turn**
|
||||
(stdin-JSON / exit-0 via `runHook`). Nothing survives between turns in memory, so an
|
||||
"every N turns" counter is impossible without persistence. PreCompact means "occasionally";
|
||||
the only per-turn signal we have is Stop. We bridge the two with a **persisted timestamp gate**:
|
||||
compact at most once per time window, tracking the last-compact instant in a small file.
|
||||
|
||||
(Decision record: alternatives considered were *every-Stop* — simplest but runs a maintenance
|
||||
pass far more often than its intent, one extra CLI spawn per turn — and *probabilistic 1/N* —
|
||||
stateless but non-deterministic cadence, can fire twice in a row or never in a short session.
|
||||
Time-gate chosen: faithful to "occasional", bounded cost, deterministic + testable via an
|
||||
injectable clock. Window default **10 min**.)
|
||||
|
||||
## 4. Placement
|
||||
|
||||
**Hermes-package-only.** The shared `runStopBody` / `makeStopHandler` in `hooks-core` — run by
|
||||
codex, cursor, and (via `makeOpenclawHandler`) openclaw — stay **byte-untouched**. Hermes is the
|
||||
only built tool lacking PreCompact, and the gate needs a hermes-specific state path
|
||||
(`~/.hermes/`). Pushing file-IO + a clock + a tool-specific path into the generic core would add
|
||||
blast radius across four packages for zero reuse. (CLAUDE.md §3.3 surgical changes; §3.2 no
|
||||
speculative flexibility.)
|
||||
|
||||
## 5. Design
|
||||
|
||||
### 5.1 New module: `src/compact-on-stop.ts`
|
||||
|
||||
Small, pure, high-cohesion (CLAUDE.md "many small files"). Exports:
|
||||
|
||||
```text
|
||||
compactStatePath(home?): string
|
||||
→ join(resolvePaths({home}).hermesDir, '.hive-mind-last-compact')
|
||||
|
||||
isCompactEnabled(env = process.env): boolean
|
||||
→ truthy WAGGLE_HERMES_COMPACT_ON_STOP, parsed exactly like WAGGLE_SIGNAL_EMIT:
|
||||
flag != null && flag !== '' && flag !== '0' && flag.toLowerCase() !== 'false'
|
||||
|
||||
resolveWindowMs(env = process.env, overrideMs?): number
|
||||
→ overrideMs (test) wins; else parseFloat(WAGGLE_HERMES_COMPACT_WINDOW_MIN) * 60000
|
||||
when finite and > 0; else DEFAULT_WINDOW_MS (600_000 = 10 min)
|
||||
|
||||
readLastCompactTs(path): Promise<number | undefined>
|
||||
→ read file, parseInt; missing / garbage / NaN → undefined (= "never"); fail-open
|
||||
|
||||
writeLastCompactTs(path, ts): Promise<void>
|
||||
→ mkdir(dirname(path), {recursive:true}) then writeFile(path, String(ts)); errors
|
||||
caught + swallowed by caller. (The mkdir matters: if the flag is set on a host where
|
||||
~/.hermes/ doesn't exist yet — e.g. hermes never installed — a bare writeFile would
|
||||
ENOENT every turn and the throttle would silently degrade to every-turn. mkdir-recursive
|
||||
is idempotent and cheap; in a real install the dir already exists.)
|
||||
|
||||
maybeCompactOnStop(ctx: HookContext, opts?: {
|
||||
now?: () => number; home?: string; windowMs?: number;
|
||||
}): Promise<void>
|
||||
```
|
||||
|
||||
`maybeCompactOnStop` body (all wrapped in one try/catch that logs+swallows — never throws,
|
||||
never rejects):
|
||||
|
||||
1. `if (!isCompactEnabled()) return;` — default-off fast path, no IO.
|
||||
2. `const now = opts.now?.() ?? Date.now();`
|
||||
3. `const path = compactStatePath(opts.home);`
|
||||
4. `const last = await readLastCompactTs(path);`
|
||||
5. `const windowMs = resolveWindowMs(process.env, opts.windowMs);`
|
||||
6. `if (last !== undefined && now - last < windowMs) return;` — inside window, skip.
|
||||
7. `await ctx.bridge.cleanupFrames();` — default `mode:'compact'`.
|
||||
8. `await writeLastCompactTs(path, now);` — **on success only** (a failed compact at step 7
|
||||
throws → caught → swallowed → timestamp NOT written → eligible to retry next turn, rather
|
||||
than being locked out for a whole window).
|
||||
|
||||
Turns are sequential subprocesses (process N exits before N+1 starts) → no read/write race.
|
||||
|
||||
### 5.2 Compose into `src/hooks/stop.ts` (save-first)
|
||||
|
||||
`runStop` keeps its current contract but composes the shared handler with the compact step.
|
||||
The base handler is reused as-is; we only wrap its `run`:
|
||||
|
||||
```ts
|
||||
export interface HermesStopOptions extends Partial<HookRunOptions> {
|
||||
now?: () => number; // test clock
|
||||
home?: string; // test $HOME override for the state file
|
||||
compactWindowMs?: number; // test window override
|
||||
}
|
||||
|
||||
export async function runStop(opts: HermesStopOptions = {}): Promise<void> {
|
||||
const { now, home, compactWindowMs, ...runOpts } = opts;
|
||||
const base = makeStopHandler(hermesAdapter);
|
||||
const handler: typeof base = {
|
||||
parse: base.parse,
|
||||
async run(payload, ctx) {
|
||||
await base.run(payload, ctx); // primary save — unchanged
|
||||
await maybeCompactOnStop(ctx, { now, home, windowMs: compactWindowMs });
|
||||
return undefined;
|
||||
},
|
||||
};
|
||||
return runHook(handler, { name: 'stop', loggerPrefix: 'hermes-hooks', ...runOpts });
|
||||
}
|
||||
```
|
||||
|
||||
Ordering + fail-open guarantees:
|
||||
- **Save before compact.** If `base.run` throws (save failed), `maybeCompactOnStop` is skipped
|
||||
and the throw lands in `runHook`'s existing try/catch → `exit(0)`. The capture is the priority;
|
||||
compaction is best-effort maintenance layered after it.
|
||||
- `maybeCompactOnStop` itself never throws, so a compact/file error cannot affect the exit code
|
||||
or the already-completed save.
|
||||
- Use `typeof base` for the handler type to avoid importing the `StopParsed` named type (it may
|
||||
not be re-exported from the `hooks-core` barrel; `ReturnType`-style typing sidesteps that).
|
||||
|
||||
### 5.3 No changes to install / verify / yaml-merger / paths basenames
|
||||
|
||||
This is a **runtime capture** behavior only. Hermes still binds 3 lifecycle events; the gate is
|
||||
not a registered hook. `paths.ts` `HOOK_BASENAMES` stays 3 (no `pre-compact`). `resolvePaths`
|
||||
already exposes `hermesDir` + a `home` override → reused for the state path; no new path API.
|
||||
|
||||
## 6. Tests (TDD — write first, watch fail, then implement)
|
||||
|
||||
### 6.1 Unit — `tests/compact-on-stop.test.ts` (new)
|
||||
|
||||
Drive `maybeCompactOnStop` directly with `makeMockBridge()`, a tmp `home` dir, and an injected
|
||||
`now`. Stub env with `vi.stubEnv` / `vi.unstubAllEnvs` (afterEach), matching the existing stop test.
|
||||
|
||||
1. flag off → `bridge.cleanupFrames` NOT called; no state file written.
|
||||
2. flag on, no prior timestamp file → `cleanupFrames` called once; state file now holds `now`.
|
||||
3. flag on, last = `now - 1min`, window 10min → NOT called (inside window).
|
||||
4. flag on, last = `now - 11min`, window 10min → called; timestamp updated to `now`.
|
||||
5. flag on, `cleanupFrames` rejects → `maybeCompactOnStop` resolves (no throw); state file NOT
|
||||
updated (retry-next-turn).
|
||||
6. flag on, state-file write fails (point `home` at an existing **file**, so `mkdir(~/.hermes)`
|
||||
throws ENOTDIR/EEXIST) → `maybeCompactOnStop` resolves, no throw; `cleanupFrames` was still
|
||||
attempted (compaction is best-effort regardless of whether the timestamp persisted).
|
||||
7. `WAGGLE_HERMES_COMPACT_WINDOW_MIN=5` honored; `compactWindowMs` opt overrides env.
|
||||
8. `isCompactEnabled` truth table: unset/''/'0'/'false'/'FALSE' → false; '1'/'true'/'yes' → true.
|
||||
|
||||
### 6.2 Integration — extend `tests/hooks/stop.test.ts`
|
||||
|
||||
Through `runStop` with the mock bridge + captures + tmp `home` + injected `now`:
|
||||
|
||||
9. **Default-off regression lock:** flag unset → existing behavior intact AND
|
||||
`bridge.cleanupFrames` NOT called (locks "documented-gap default").
|
||||
10. flag on, eligible → save happens AND `cleanupFrames` called; assert **call order**
|
||||
(`saveMemory` invoked before `cleanupFrames`).
|
||||
11. flag on, eligible, `cleanupFrames` rejects → `exits === [0]`, `saveMemory` still called once.
|
||||
12. flag on, but **no `assistant_response`** (no save) → `cleanupFrames` still gate-eligible and
|
||||
may run (compaction is independent of whether this turn had a response) — assert it runs and
|
||||
exits 0. *(Confirms compaction isn't accidentally coupled to the save path.)*
|
||||
|
||||
All existing hermes stop tests must stay green unchanged.
|
||||
|
||||
## 7. Docs to correct (same PR)
|
||||
|
||||
- `src/adapter.ts` — the block comment "There is NO PreCompact event … the gap is documented,
|
||||
**never approximated**." → "…no PreCompact event; the maintenance pass is **approximated
|
||||
opt-in** from Stop (`WAGGLE_HERMES_COMPACT_ON_STOP`, default off) — see `compact-on-stop.ts`."
|
||||
(`eventName['pre-compact']` stays `undefined` — we are NOT registering a hook.)
|
||||
- `src/index.ts` header — note the opt-in approximation; export `maybeCompactOnStop` +
|
||||
`compactStatePath` if useful for consumers (optional).
|
||||
- `README.md` — capture-fidelity hermes row: add the opt-in compact line + the two env vars.
|
||||
- `src/bin/hermes-hooks.ts` — `printInstallSummary` capture-fidelity blurb: one line noting the
|
||||
opt-in flag (so installers learn it exists).
|
||||
- Handoff / CLAUDE.md §10 — move OQ-4 from open (P2) to closed in the next handoff.
|
||||
|
||||
## 8. Risk / rollback
|
||||
|
||||
Additive + opt-in: with the flag unset (default) the only change is one extra `await base.run`
|
||||
indirection that is behavior-identical to today (test #9 locks this). Rollback = revert the
|
||||
hermes commit; no other package is touched. No new dependency. No upstream/CLI surface change
|
||||
(`cleanup_frames` already exists and is exercised by 4 other packages).
|
||||
|
||||
## 9. Success criteria
|
||||
|
||||
- New unit + integration tests pass (12 cases above); full hermes package suite stays green
|
||||
(was 89/89).
|
||||
- `npx tsc --noEmit` clean on `hive-mind-hooks-hermes` (+ `hooks-core`/`shim-core` unaffected).
|
||||
- Default-off behavior byte-identical to pre-change (regression-locked by test #9).
|
||||
- Docs no longer claim "never approximated".
|
||||
87
docs/superpowers/specs/2026-06-01-openclaw-dedup-design.md
Normal file
87
docs/superpowers/specs/2026-06-01-openclaw-dedup-design.md
Normal file
@@ -0,0 +1,87 @@
|
||||
# OQ-6 — Provenance-Insensitive Save-Side Dedup (OpenClaw gateway double-capture)
|
||||
|
||||
**Date:** 2026-06-01
|
||||
**Status:** Approved design, pre-plan (implementation-ready; no code written yet)
|
||||
**Origin:** Wave 2/3 hook ports spec §5.5 open question OQ-6 (deferred fast-follow). See `docs/superpowers/specs/2026-06-01-wave23-hook-stubs-design.md`.
|
||||
**Touches:** `packages/hive-mind-core/src/mind/frames.ts` (+ tests). OSS-mirrored via subtree-split.
|
||||
|
||||
---
|
||||
|
||||
## 1. Summary
|
||||
|
||||
**Problem.** The OpenClaw gateway can drive Claude Code / Codex as *backends*. If those backends also have hive-mind hooks installed, the same conversation turn is captured twice — once by OpenClaw's gateway hook (`message:received` / `message:sent`) and once by the backend's own lifecycle hooks — producing two near-identical memory frames. Wave 2/3 shipped provenance *stamping* (OpenClaw frames carry `openclaw-gateway:<channel>`) but deferred the dedup.
|
||||
|
||||
**Fix.** Make the existing save-side dedup **provenance-insensitive**: compare the semantic turn **body**, not the `[hm …]` metadata prefix that carries `session:`/`src:`. This collapses any two same-body captures of a turn into one stored frame regardless of which source wrote it.
|
||||
|
||||
**Why it's small.** `FrameStore.createIFrame` already dedups every insert via `findDuplicate` — but with an *exact* `sha256(content.trim())`. The double-capture survives for exactly one reason: the two frames' content prefixes differ (`session:`/`src:`). Stripping that prefix before hashing closes the gap with a one-helper change.
|
||||
|
||||
## 2. Decision log
|
||||
|
||||
- **D-locus (approved): save-side, provenance-aware.** Fix in `findDuplicate` (central, `hive-mind-core`) rather than OpenClaw-local suppression (fragile — the gateway may not expose which backend handled a turn) or recall-side dedup (frame-count moat metric still double-counts; every recall pays). Benefits all sources, not just OpenClaw.
|
||||
- **D-scope (approved): provenance-insensitive globally within the existing 500-frame recency window.** Two identical short bodies (e.g. `"continue"`) across recent sessions also merge to one frame + `access_count`. Accepted as correct/desirable; the recency bound keeps it from being unbounded.
|
||||
- **D-match (approved): exact match on the stripped body, not fuzzy.** Conservative — only *byte-identical* bodies collapse. Rejected `trigramSimilarity` (risks false-merging legitimately-similar frames).
|
||||
|
||||
## 3. Mechanism (verified from source)
|
||||
|
||||
- `FrameStore.createIFrame(...)` calls `findDuplicate(content)` first (frames.ts:78); on a hit it bumps `access_count` (`touch`) and returns the existing frame — no new row.
|
||||
- `findDuplicate(content)` (frames.ts:251) hashes `sha256(content.trim())` and compares against the SHA-256 of each of the **last 500** frames' trimmed content. Recency-bounded by design (the docstring notes unbounded dedup would need a dedicated `content_hash` column + index).
|
||||
- The stored `content` is `buildPrefix(frame) + frame.body`, where `buildPrefix` (shim-core `frame-encoder.ts:106-117`) emits `[hm session:<scope> parent:<id> src:<source> event:<type>] ` (tokens present only when set).
|
||||
- For the double-capture: OpenClaw writes `[hm session:openclaw-gateway:<channel> src:openclaw event:stop] <body>`; the backend writes `[hm session:<backend-scope> src:claude-code event:stop] <body>`. **Same `<body>`, different prefix → different exact hash → not deduped.**
|
||||
|
||||
## 4. The change
|
||||
|
||||
Add a prefix-stripping helper and apply it on both sides of the comparison in `findDuplicate`:
|
||||
|
||||
```ts
|
||||
/**
|
||||
* Strip the leading hive-mind metadata prefix `[hm session:… src:… event:…] `
|
||||
* so dedup compares the semantic turn BODY, not the provenance. Content without
|
||||
* the prefix (harvest / ingest / cognify) is returned unchanged — a no-op.
|
||||
*/
|
||||
function stripHmPrefix(content: string): string {
|
||||
return content.replace(/^\[hm [^\]]*\]\s*/, '');
|
||||
}
|
||||
|
||||
findDuplicate(content: string): MemoryFrame | null {
|
||||
const key = createHash('sha256').update(stripHmPrefix(content).trim()).digest('hex');
|
||||
const existing = this.db.getDatabase().prepare(`
|
||||
SELECT * FROM memory_frames ORDER BY id DESC LIMIT 500
|
||||
`).all() as MemoryFrame[];
|
||||
for (const frame of existing) {
|
||||
const frameKey = createHash('sha256').update(stripHmPrefix(frame.content).trim()).digest('hex');
|
||||
if (frameKey === key) {
|
||||
this.touch(frame.id);
|
||||
return frame;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
```
|
||||
|
||||
No signature change; no new column; no migration. `createIFrame` and every other caller are unchanged.
|
||||
|
||||
## 5. Properties
|
||||
|
||||
- **Backward-compatible / minimal blast radius.** The strip is a no-op for any content lacking the `[hm …]` prefix, so harvest/ingest/cognify dedup behavior is byte-for-byte unchanged. Only hook-captured frames (which carry the prefix) change.
|
||||
- **Conservative.** Only *identical bodies* collapse. Genuinely different captures of a turn (e.g. OpenClaw's raw outbound text vs the backend's summarized turn) have different bodies and both survive — the fix never merges distinct content.
|
||||
- **Attribution preserved.** First writer's frame is kept verbatim (with its `src:`); the later duplicate only bumps `access_count`. One frame per turn, attributed to whoever landed first.
|
||||
- **Recency window sufficient.** Gateway and backend fire on the same turn (seconds apart) → both within the recent 500 → reliably caught. No new index needed.
|
||||
- **Generic + OSS-clean.** Lives in `hive-mind-core`, ships in the mirror, helps every tool. No proprietary/KVARK logic; no secrets.
|
||||
|
||||
## 6. Testing
|
||||
|
||||
`frames.ts` unit tests (extend the existing `findDuplicate`/`createIFrame` suite):
|
||||
|
||||
1. **Cross-source collapse:** insert `[hm session:openclaw-gateway:c1 src:openclaw event:stop] BODY` then `[hm session:s2 src:claude-code event:stop] BODY` → second returns the first frame, `access_count` incremented, row count unchanged (= the OQ-6 scenario).
|
||||
2. **Different body → no merge:** same prefixes, different bodies → two distinct frames.
|
||||
3. **Non-prefixed content unchanged (regression):** two identical bodies *without* `[hm …]` prefix still dedup exactly as before; a prefixed vs non-prefixed pair with the same body collapses (strip makes them equal) — assert intended.
|
||||
4. **Recency bound holds:** a duplicate older than the 500-frame window is not found (documents the bound).
|
||||
5. **Prefix-strip helper:** `stripHmPrefix` removes a well-formed prefix, leaves prefix-less content untouched, and does not over-strip a body that merely contains `[` brackets later.
|
||||
|
||||
OpenClaw-level (optional, in `hive-mind-hooks-openclaw` or an integration test): a gateway-captured frame + a synthetic backend frame of the same turn yield one stored frame.
|
||||
|
||||
## 7. Risks + open questions
|
||||
|
||||
- **Over-merge of trivial identical bodies** (e.g. `"continue"` across recent sessions). Accepted per D-scope; the `access_count` bump preserves the multiplicity signal, and such content is low-value memory. If it ever proves wrong, the strip can be narrowed to only drop the `session:`/`src:` tokens while keeping `event:` (so different events never collide) — noted, not implemented.
|
||||
- **Prefix format coupling.** The regex `^\[hm [^\]]*\]\s*` is coupled to `buildPrefix`'s output. If `frame-encoder.ts` changes the prefix wrapper, the strip must track it. Mitigation: a shared constant/test that asserts `stripHmPrefix(buildPrefix(f) + body) === body` would lock the coupling — consider co-locating, but the helper lives in `hive-mind-core` and `buildPrefix` in `shim-core`, so a cross-package assertion test is the pragmatic guard.
|
||||
- **No content_hash column.** Deliberately out of scope; the 500-window covers OQ-6. Unbounded historical dedup remains a separate future concern (already flagged in the frames.ts docstring).
|
||||
@@ -0,0 +1,383 @@
|
||||
# Wave 2/3 Hook Feasibility — Porting the hive-mind claude-code hook package to 6 other AI tools
|
||||
|
||||
**Date:** 2026-06-01
|
||||
**Author:** Synthesis from 6 per-tool feasibility research passes
|
||||
**Status:** Scoping brief — gates a human go/no-go + ordering decision
|
||||
**Reference package:** `packages/hive-mind-hooks-claude-code` (Wave 1, shipped)
|
||||
**Targets:** `cursor`, `claude-desktop`, `codex`, `codex-desktop`, `hermes`, `openclaw` (all currently `export {}` stubs)
|
||||
|
||||
---
|
||||
|
||||
## 0. TL;DR
|
||||
|
||||
Five of six tools have a real, deterministic lifecycle-hook surface and can reuse ~80% of the Wave 1
|
||||
package (shim-core + the four handler bodies + the runHook stdin/stdout/exit-0 contract). The per-tool
|
||||
work is almost entirely confined to **three install-layer modules** — `paths.ts`, `settings-merger.ts`,
|
||||
`install.ts` (+ matching `uninstall.ts`/`verify.ts`) — plus payload-field remapping in the handlers.
|
||||
|
||||
The single outlier is **Claude Desktop**: it has **no lifecycle hook API at all**. It is not a hook port;
|
||||
it is an MCP-server registration that yields only voluntary, model-initiated, on-demand capture — strictly
|
||||
weaker than the every-turn deterministic capture the Wave 1 hook gives. Build it (the MCP server already
|
||||
exists in `packages/memory-mcp`) but label it honestly as "MCP-bridge, partial capture."
|
||||
|
||||
**Codex + Codex Desktop should be ONE package** — they share `~/.codex/` entirely.
|
||||
|
||||
---
|
||||
|
||||
## 1. Comparison Matrix
|
||||
|
||||
| Tool | What it is | Config surface | SessionStart | UserPromptSubmit | Stop | PreCompact | Tier | Confidence |
|
||||
|---|---|---|---|---|---|---|---|---|
|
||||
| **Cursor** | AI-native VS Code-fork editor; first-party Hooks since v1.7 | `~/.cursor/hooks.json` (JSON, `{version,hooks:{event:[...]}}`) | native (`sessionStart`, `additional_context` inject) | native (`beforeSubmitPrompt`; save-only, **no inject**) | native (`stop`; turn read via `transcript_path`) | approx (`preCompact`; observational, can't block) | near-direct port | high |
|
||||
| **Codex CLI** | OpenAI terminal coding agent; stable hooks mirror CC schema | `~/.codex/hooks.json` (JSON, `{hooks:{Event:[{matcher,hooks:[...]}]}}`) | native (`SessionStart`, `additionalContext`) | native (`UserPromptSubmit`, `prompt`) | native (`Stop`, `last_assistant_message`) | native (`PreCompact`, `trigger`) | near-direct port | high |
|
||||
| **Codex Desktop** | OpenAI native GUI app; same runtime + same `~/.codex/` as CLI | shared `~/.codex/hooks.json` (identical to Codex CLI) | native | native | native | native (`PostCompact` bonus) | near-direct port (fold into Codex pkg) | high |
|
||||
| **Hermes Agent** | Nous Research MIT Python self-improving CLI/server agent | `~/.hermes/config.yaml` (**YAML**, `hooks:` block) | native split (`on_session_start` observe + `pre_llm_call` first-turn inject) | native (`pre_llm_call`) | approx (`post_llm_call` per-turn / `on_session_finalize`) | **none** (internal compaction, no hook) | lifecycle-hooks-need-adapter | medium |
|
||||
| **OpenClaw** | Self-hosted Node "Gateway" bridging chat apps to coding agents | `~/.openclaw/openclaw.json` (**JSON5**) + `~/.openclaw/hooks/<name>/{HOOK.md,handler.ts}` (**in-process TS, not stdin/stdout**) | native (`agent:bootstrap` mutable `bootstrapFiles` / `command:new`) | native (`message:received`) | approx (`message:sent` 0..N/turn / `before_agent_finalize`) | native (`session:compact:before`) | lifecycle-hooks-need-adapter | medium |
|
||||
| **Claude Desktop** | Anthropic first-party GUI chat app; only extensibility = MCP | `claude_desktop_config.json` `mcpServers` key (macOS `~/Library/Application Support/Claude/`, Win `%APPDATA%\Claude\`) | **none** (MCP tool/prompt, model-discretion) | **none** (raw prompt never handed to server) | **none** (no turn event) | **none** (no host compaction event) | mcp-bridge-only | high |
|
||||
|
||||
> "near-direct port" = reuse shim-core + handlers as-is; rewrite only the 3 install-layer modules + payload-field renames.
|
||||
> "lifecycle-hooks-need-adapter" = all/most lifecycle events exist but the config format (YAML / JSON5+TS-handler) and/or
|
||||
> registration model differs enough that the install layer is a rewrite and one event is degraded/missing.
|
||||
> "mcp-bridge-only" = no hook surface; replace the hook package with an MCP-server config patcher; partial capture only.
|
||||
|
||||
---
|
||||
|
||||
## 2. Per-Tool Detail
|
||||
|
||||
### 2.1 Cursor — `near-direct port` — confidence: high
|
||||
|
||||
**What it is.** Anysphere's Cursor, an AI-native VS Code-fork desktop app with an Agent/Composer loop and a
|
||||
`cursor-agent` CLI. Since v1.7 (late 2025, expanded through 2026) it ships a real first-party **Hooks**
|
||||
feature that runs scripts on agent-loop lifecycle events.
|
||||
|
||||
**Config surface.** `~/.cursor/hooks.json` (user-global), plus project `<repo>/.cursor/hooks.json` and enterprise
|
||||
paths (Win `C:\ProgramData\Cursor\hooks.json`; mac `/Library/Application Support/Cursor/hooks.json`; linux
|
||||
`/etc/cursor/hooks.json`). Format: `{ "version": 1, "hooks": { "<event>": [ { "command": "node \"<abs>\\dist\\hooks\\<event>.js\"", "type": "command", "timeout": 5 } ] } }`. Each event maps to an **array** of hook
|
||||
entries → additive append/merge works exactly like the Wave 1 settings.json merge. This is a **separate file**
|
||||
from `settings.json` (which Cursor uses for editor prefs) — new path constants, same merge/marker/backup logic.
|
||||
Detection already present: binary candidates + pointer `~/.cursor/hive-mind-install.json` in `HOOK_POINTER_BY_TOOL`.
|
||||
|
||||
**Lifecycle mapping.**
|
||||
- SessionStart → `sessionStart` (new composer conversation); stdout `{additional_context, env}` — directly satisfies recall-and-inject.
|
||||
- UserPromptSubmit → `beforeSubmitPrompt` (stdin `{prompt, attachments}`); save-only. **Caveat: cannot inject** (stdout only `{continue, user_message}`) — fine, hive-mind's UserPromptSubmit only persists.
|
||||
- Stop → `stop` (status `completed|aborted|error`, `loop_count`); completed turn read via base field `transcript_path`, not inline — minor handler change.
|
||||
- PreCompact → `preCompact` (rich usage stats) but **observational only** (stdout `{user_message}`, cannot block/reorder). `compact_memory` runs fire-and-forget; "before host truncates" is best-effort, not guaranteed-before.
|
||||
|
||||
**Recommended approach.** Port nearly verbatim. shim-core reusable as-is. `hooks/_shared.ts` ports verbatim
|
||||
(identical stdin-JSON/stdout-JSON command-hook model). Four handlers port with small field renames (SessionStart
|
||||
returns `{additional_context}`; UserPromptSubmit reads `stdin.prompt`; Stop summarizes via `base.transcript_path`;
|
||||
PreCompact shells `compact_memory` best-effort). Rewrite the 3 install modules: `paths.ts` (`hooks.json` target,
|
||||
keep pointer), `settings-merger.ts` (`HOOK_EVENT_BY_BASENAME = {session-start:'sessionStart',
|
||||
user-prompt-submit:'beforeSubmitPrompt', stop:'stop', pre-compact:'preCompact'}`; wrap with `version:1` +
|
||||
`hooks.<event>` arrays; keep `_hiveMindShim` marker), `install.ts` (create `{version:1,hooks:{}}` skeleton if
|
||||
absent — CC throws if settings.json missing). `type:'command'`, `timeout:5`. ~80% reuse.
|
||||
|
||||
**Blockers.**
|
||||
- `beforeSubmitPrompt` cannot inject context (acceptable — UserPromptSubmit only persists).
|
||||
- `preCompact` observational only — "run compact_memory BEFORE host truncates" is best-effort.
|
||||
- Stop reads turn via `transcript_path` (null if transcripts disabled) — handler must tolerate null and fail open.
|
||||
- `hooks.json` may not exist on fresh install — `install.ts` must create skeleton, not throw.
|
||||
- Windows `.cmd`-shim exec problem applies — thread `--cli-path` to compiled `hive-mind-cli dist/index.js`.
|
||||
- Editing `hooks.json` likely needs a Cursor restart (reload semantics unverified across 1.7.x) — surface in install UX.
|
||||
- Exact `transcript_path` file format undocumented — Stop summarizer must read defensively (don't assume JSONL).
|
||||
|
||||
**Sources.** `cursor.com/docs/hooks` (+ `.md`); `blog.gitbutler.com/cursor-hooks-deep-dive`;
|
||||
`aiengineerguide.com/til/cursor-agent-lifecycle-hooks`; `skywork.ai/blog/how-to-cursor-1-7-hooks-guide`;
|
||||
repo `packages/{agent,shared}/src/tool-detection.ts`, `packages/hive-mind-hooks-claude-code/**`,
|
||||
`packages/hive-mind-hooks-cursor/src/index.ts` (stub).
|
||||
|
||||
---
|
||||
|
||||
### 2.2 Codex CLI — `near-direct port` — confidence: high
|
||||
|
||||
**What it is.** OpenAI's terminal coding agent CLI (`codex`), rooted at `~/.codex/config.toml`. As of May 2026 it
|
||||
ships a **stable** lifecycle-hooks system that deliberately mirrors Claude Code's schema (same event names,
|
||||
stdin JSON, exit-0 contract, `additionalContext` injection).
|
||||
|
||||
**Config surface.** Write a standalone `~/.codex/hooks.json` (JSON), **not** config.toml — keeps us out of the
|
||||
user's TOML and away from protected `notify`/`profile`/`model_providers` keys. Shape mirrors CC's hooks block:
|
||||
`{ "hooks": { "SessionStart": [ { "matcher": "startup|resume", "hooks": [ { "type":"command", "command":"node \"<dist>/hooks/session-start.js\" --cli-path \"...\"" } ] } ], "UserPromptSubmit":[...], "Stop":[...], "PreCompact":[...] } }`.
|
||||
Hooks across config layers are **additive** (no override) → marker-tagged merge not required for correctness but
|
||||
kept for byte-identical reversible uninstall. Pointer `~/.codex/hive-mind-install.json` matches `HOOK_POINTER_BY_TOOL`.
|
||||
|
||||
**Lifecycle mapping.** All four native and field-compatible with the existing handlers:
|
||||
- SessionStart (`startup|resume|clear|compact`, `source`); injects via stdout text or `hookSpecificOutput.additionalContext` (exact shape session-start.ts already emits).
|
||||
- UserPromptSubmit (`prompt`, `turn_id`, `session_id`, `cwd`) — existing handler already parses `prompt`/`session_id`/`cwd`.
|
||||
- Stop (`last_assistant_message`, `turn_id`, `stop_hook_active`).
|
||||
- PreCompact (`trigger` = `manual|auto`).
|
||||
|
||||
**Recommended approach.** Promote stub to near-verbatim CC clone. Reuse as-is: `@waggle/hive-mind-shim-core`
|
||||
entirely (tool-agnostic); `src/hooks/{_shared,session-start,user-prompt-submit,stop,pre-compact}.ts` essentially
|
||||
unchanged (already parse snake_case `session_id`/`prompt`/`cwd`; session-start already emits the
|
||||
`hookSpecificOutput.additionalContext` Codex honors). Minor edits: set `HookEvent.source` from `'claude-code'`
|
||||
→ `'codex'`; add codex `last_assistant_message`/`trigger` keys to the existing multi-key `pickStringFromObject`
|
||||
fallbacks. Rewrite install layer: `paths.ts` (`~/.codex/hooks.json`, pointer `~/.codex/hive-mind-install.json`),
|
||||
`settings-merger.ts` (`{hooks:{Event:[{matcher,hooks:[...]}]}}`, matcher `"startup|resume|clear|compact"` for
|
||||
SessionStart / `""` elsewhere, keep `_hiveMindShim` marker), `install.ts`/`uninstall.ts`/`verify.ts` (create
|
||||
hooks.json if absent; delete-if-created vs restore-backup-if-existed). Keep `--cli-path` Windows quoting verbatim.
|
||||
`HOOK_EVENT_BY_BASENAME` is identical. **Do NOT use `notify`** — it only fires agent-turn-complete (Stop subset),
|
||||
JSON on argv not stdin, user-config-only, strictly weaker. Estimate ~1 day, dominated by install/merge/verify + tests.
|
||||
|
||||
**Blockers.**
|
||||
- Field-name casing confirmed from docs but not from a live payload — add codex keys as fallbacks rather than assuming.
|
||||
- Minimum Codex version shipping stable hooks not pinned ("stable as of May 2026", no version) — verify dynamically; too-old `codex` may silently ignore hooks.json.
|
||||
- CC installer hard-requires a pre-existing config; codex hooks.json is optional/standalone → `install.ts` must create-if-missing (behavioral fork, not a copy).
|
||||
- Admin lockdown `allow_managed_hooks_only = true` (requirements.toml) can suppress user hooks — `verify` must surface this so install doesn't silently no-op.
|
||||
|
||||
**Sources.** `developers.openai.com/codex/{hooks,config-advanced,config-reference}`;
|
||||
`github.com/openai/codex/blob/main/docs/config.md`; `github.com/openai/codex/issues/8189`;
|
||||
repo detection + Wave 1 reference + `packages/hive-mind-shim-core/src/cli-bridge.ts`.
|
||||
|
||||
---
|
||||
|
||||
### 2.3 Codex Desktop — `near-direct port` (fold into Codex pkg) — confidence: high
|
||||
|
||||
**What it is.** OpenAI's native Codex desktop GUI (the "Codex app"), macOS + Windows (Linux waitlisted), running
|
||||
Codex threads in parallel. **Not a separate engine** — same Codex runtime, same shared `~/.codex/config.toml` as
|
||||
CLI + IDE extension.
|
||||
|
||||
**Config surface.** Registers into the **shared** `~/.codex/` root — identical for CLI, IDE, and App. Use
|
||||
`~/.codex/hooks.json` (cleaner additive-JSON merge) or inline `[[hooks.<Event>]]` TOML in config.toml. JSON shape:
|
||||
`{"hooks":{"<Event>":[{"matcher":"*","hooks":[{"type":"command","command":"node \"<dist>/hooks/<event>.js\"","commandWindows":"...","timeout":600}]}]}}`.
|
||||
**Non-managed hooks require one-time user trust via the `/hooks` command** (trust keyed by hook hash) before they execute.
|
||||
|
||||
**Lifecycle mapping.** SessionStart / UserPromptSubmit / Stop / PreCompact all native (PostCompact bonus); JSON on stdin.
|
||||
|
||||
**Recommended approach.** Build **ONE** `@waggle/hive-mind-hooks-codex` package serving **both** `codex` (CLI) and
|
||||
`codex-desktop` (App) — they share `~/.codex/` entirely, so a separate codex-desktop installer is redundant. Make
|
||||
`hive-mind-hooks-codex-desktop` a thin re-export, or key one installer on the shared config root for both ToolIds.
|
||||
Port as in §2.2. **Also fix `tool-detection.ts`:** the current `codexDesktopCandidatePaths` are speculative/wrong
|
||||
(`%LOCALAPPDATA%/OpenAI/Codex.exe`, `/Applications/Codex.app/...`; source comment admits "unreleased at time of
|
||||
writing"). Prefer **config-presence detection at `~/.codex/`** over guessing the binary; correct
|
||||
`HOOK_POINTER_BY_TOOL['codex-desktop']` from `'.config/Codex/...'` → `'.codex/hive-mind-install.json'`. Add a
|
||||
post-install note to run `/hooks` once to trust.
|
||||
|
||||
**Blockers.**
|
||||
- Non-managed hooks are **not auto-trusted** — install is not fully silent; surface "run `/hooks` to trust" (enterprise `requirements.toml` managed hooks bypass trust but that's an org path).
|
||||
- App settings page doesn't mention hooks; only the general `/codex/hooks` doc asserts the App honors them — runtime verification on an actual App install needed (CLI definitely fires).
|
||||
- On-disk desktop paths undocumented (mac `.dmg`, Win Microsoft Store `9PLM9XGG6VKS`, Linux unreleased) — replace hardcoded guesses with config-presence detection.
|
||||
- Stop exit-2 "continues with stderr as a new prompt" (not a hard block) — harmless for fail-open exit-0 capture; ensure ported handler never writes stderr+exit-2.
|
||||
|
||||
**Sources.** `developers.openai.com/codex/{hooks,app/settings,ide/settings,config-reference,app,changelog}`;
|
||||
repo detection (speculative paths, wrong pointer) + `packages/hive-mind-hooks-codex-desktop/src/index.ts` (stub) + Wave 1 reference.
|
||||
|
||||
---
|
||||
|
||||
### 2.4 Hermes Agent — `lifecycle-hooks-need-adapter` — confidence: medium
|
||||
|
||||
**What it is.** Hermes Agent by Nous Research — a real MIT-licensed Python self-improving CLI/server agent (the
|
||||
`hermes` binary, config under `~/.hermes/`). **NOT** the Nous Hermes LLM model. Confirmed target by
|
||||
`docs/addictiveness-audit-2026-05-28/BENCHMARK-hermes.md`.
|
||||
|
||||
**Config surface.** Config-driven YAML shell hooks in `~/.hermes/config.yaml` under a top-level `hooks:` block (no
|
||||
Python required). Each event key → list of `{ command, matcher?, timeout? (default 60, max 300) }`. Scripts get
|
||||
JSON on stdin, return JSON on stdout (`{"context":"..."}` to inject, `{"decision":"block",...}`, or `{}` for no-op)
|
||||
— nearly identical to the hive-mind contract. Hooks live by convention under `~/.hermes/agent-hooks/`. Pointer
|
||||
`~/.hermes/hive-mind-install.json` already expected.
|
||||
|
||||
**Lifecycle mapping.**
|
||||
- SessionStart → **split**: `on_session_start` (observer-only, no injection) for "switch workspace" + `pre_llm_call` with `is_first_turn=true` (returns `{context:...}`) for "inject recalled frames."
|
||||
- UserPromptSubmit → `pre_llm_call` (fires once/turn before tool loop; `user_message`/`conversation_history`/`is_first_turn`) — fire-and-forget save.
|
||||
- Stop → `post_llm_call` (after tool loop, `assistant_response`) for the per-turn important-frame summary; optionally `on_session_finalize` for end-of-session flush.
|
||||
- PreCompact → **none**. Hermes compacts internally (preflight >50% ctx, gateway auto-compress >85%) and flushes memory to disk first, but exposes **no hook** at that boundary. Approximate via opportunistic `compact_memory` from the Stop handler.
|
||||
|
||||
**Recommended approach.** Config-driven YAML shell-hook installer — cleanest target after CC itself. Reuse as-is:
|
||||
`@waggle/hive-mind-shim-core` + the four handler bodies (same stdin-JSON/stdout-JSON, exit-0/fail-open contract;
|
||||
port with payload-field remapping via `pickStringField`, e.g. `user_message`/`is_first_turn`/`session_id`).
|
||||
Rewrite the install trio: `paths.ts` (`~/.hermes/`, `config.yaml`, pointer), `settings-merger.ts` (**YAML** merge
|
||||
keyed by `{on_session_start, pre_llm_call, post_llm_call}`; add a YAML parser dep e.g. `yaml`; additive +
|
||||
marker-tagged + preserve user hooks), `install.ts` (read-or-create — config may not exist; timestamped
|
||||
byte-identical backup + reversible uninstall). Wire: SessionStart→`on_session_start` (+ `pre_llm_call`/first-turn
|
||||
inject), UserPromptSubmit→`pre_llm_call`, Stop→`post_llm_call`. **Drop the 4th hook** (no PreCompact — document the
|
||||
gap, don't invent). Finally add `'hermes'` to `HOOKS_COHORT` in `tool-launcher.ts` and ship a real bin (currently stub).
|
||||
|
||||
**Blockers.**
|
||||
- No PreCompact-equivalent — 4th hook cannot be ported, only approximated by opportunistic `compact_memory` from Stop.
|
||||
- Config is YAML, not JSON — YAML round-trip is not byte-identical for re-serialized output; mitigate by keeping a literal backup for uninstall rather than diff-merge fidelity.
|
||||
- Existing pkg is a binless `export {}` stub and `'hermes'` is excluded from `HOOKS_COHORT` — both must change.
|
||||
- Context-injection asymmetry: `on_session_start` is observer-only; injection rides `pre_llm_call(is_first_turn)` → SessionStart handler must split into two registrations.
|
||||
- **Source-level unverified:** the exact `VALID_HOOKS` allow-list for shell hooks (vs Python plugin hooks); docs-only, not read from implementation. Benchmark doc's star/version figures (170k stars, v0.14.0) read as possibly aspirational, unconfirmed.
|
||||
- Windows is early-beta (Linux/macOS/WSL2 first-class); Python-installed binary → PATH detection + launch UX weaker on Windows.
|
||||
|
||||
**Sources.** `hermes-agent.nousresearch.com/docs/{user-guide/features/hooks, developer-guide/agent-loop,
|
||||
user-guide/features/skills, guides/build-a-hermes-plugin}`; `github.com/NousResearch/hermes-agent` (+ hooks.md);
|
||||
repo detection + launcher + `docs/addictiveness-audit-2026-05-28/BENCHMARK-hermes.md` + Wave 1 reference.
|
||||
|
||||
---
|
||||
|
||||
### 2.5 OpenClaw — `lifecycle-hooks-need-adapter` — confidence: medium
|
||||
|
||||
**What it is.** A self-hosted Node "Gateway" (Peter Steinberger / community, `github.com/openclaw/openclaw`)
|
||||
bridging messaging surfaces (Discord, Slack, Telegram, WhatsApp, iMessage, …) to AI coding agents. CLI +
|
||||
long-running daemon, **not** a CC fork or IDE, but it has its own `openclaw` CLI and a real internal hooks system.
|
||||
Binary name + `~/.openclaw` config dir verified correct; detector carries no model of the hooks subsystem.
|
||||
|
||||
**Config surface.** `~/.openclaw/openclaw.json` (**JSON5** — comments + trailing commas). Register hooks two ways:
|
||||
(1) discovery-based (preferred) — drop `~/.openclaw/hooks/<name>/` with `HOOK.md` + `handler.ts`, or point
|
||||
`hooks.internal.load.extraDirs:["<abs>"]` at an external dir; enable via `openclaw hooks enable <name>`. (2) legacy
|
||||
`hooks.internal.handlers[]` (deprecated). Internal-hooks shape:
|
||||
`{ "hooks": { "internal": { "enabled": true, "entries": { "<name>": {"enabled":true,"env":{...}} }, "load": {"extraDirs":[...]} } } }`.
|
||||
**Handlers are in-process TypeScript** (`export default async (event)=>{...}`) — but MAY shell out
|
||||
(`execFileAsync("openclaw",[...])`), so a hive-mind handler can spawn `hive-mind-cli` like CliBridge does.
|
||||
(Distinct from the HTTP-webhook `hooks.{enabled,token,path,mappings}` block — a different subsystem.)
|
||||
|
||||
**Lifecycle mapping.**
|
||||
- SessionStart → `agent:bootstrap` (fires before workspace bootstrap files injected; exposes a **mutable `bootstrapFiles` array** — first-class injection seam ≈ CC `additionalContext`) or `command:new` (on `/new`); `gateway:startup` is process-level.
|
||||
- UserPromptSubmit → `message:received` (every inbound channel message before agent processes; `from`/`content`/`channelId`/`metadata`) — direct save-prompt analogue.
|
||||
- Stop → **approx**. No single per-turn Stop. `message:sent` fires per delivered outbound reply (0..N/turn); or `before_agent_finalize` inspects the final answer during finalization. Maps onto `message:sent` but turn boundaries are looser.
|
||||
- PreCompact → `session:compact:before` (`messageCount`, `tokenCount`); `session:compact:after` reports before/after — direct PreCompact analogue, ideal `compact_memory` point.
|
||||
|
||||
**Recommended approach.** Real integration via OpenClaw's **internal** hooks (all 4 events exist). Reuse
|
||||
`@waggle/hive-mind-shim-core` as-is (CliBridge/logger/workspace-resolver/prompt-summarizer/importance-classifier/
|
||||
frame-encoder all tool-agnostic). The four handler bodies port **in spirit** but must be **re-authored as OpenClaw
|
||||
handler.ts modules** (`export default async (event) => {...}` switching on `event.type`/`event.action`, not reading
|
||||
stdin JSON). Mapping: `agent:bootstrap` (push recalled frames into mutable `bootstrapFiles`) or `command:new` →
|
||||
SessionStart; `message:received` → save temp frame; `message:sent`/`before_agent_finalize` → summarize+save;
|
||||
`session:compact:before` → `compact_memory`. Rewrite the registration trio: `paths.ts` (`~/.openclaw/` +
|
||||
`~/.openclaw/hooks/<name>/`), `settings-merger.ts` (discovery-based: write hook **directories** with
|
||||
`HOOK.md`+`handler.ts` and/or merge `hooks.internal.load.extraDirs` into JSON5 openclaw.json — **no** command-array
|
||||
to splice), `install.ts` (back up openclaw.json, patch `hooks.internal`, `openclaw hooks enable`, drop the pointer,
|
||||
byte-identical reversible uninstall). Keep fail-open as **try/catch inside the handler** (not `process.exit(0)`).
|
||||
|
||||
**Blockers.**
|
||||
- No per-turn Stop — `message:sent` is 0..N/turn, `before_agent_finalize` is a plugin-hook → need a debounce/dedup or "last message:sent of a turn" heuristic.
|
||||
- Registration model incompatible with the reference: discovery-based TS handler files + JSON5, not a settings.json command-array → install trio is a **rewrite, not a parameterization**.
|
||||
- JSON5 (comments + trailing commas) — naive `JSON.parse/stringify` destroys user comments; need JSON5-aware merge or restrict edits to writing hook dirs + minimal `extraDirs`.
|
||||
- Handlers are in-process TS loaded by the gateway → fail-open contract becomes "default-exported async handler that must not throw," not "thin stdin script, exit 0."
|
||||
- **Unverified:** exact `handler.ts` event TS type, whether `execFileAsync` is reliable under the gateway loop, whether `openclaw hooks enable` is required vs auto-discovery, and whether `bootstrapFiles` mutation is the sanctioned injection path vs `command:new` (docs prose + one example, not source-read).
|
||||
- OpenClaw drives arbitrary backend coding agents (incl. claude-code/codex) → gateway-layer capture may **double-count** if the backend also has hive-mind hooks → needs a provenance/dedup story.
|
||||
|
||||
**Sources.** `docs.openclaw.ai/{automation/hooks, gateway/configuration-reference, cli/agent}`; `openclaw.ai`;
|
||||
`github.com/openclaw/openclaw` (PR #9761 hooks, Issue #3336); milvus.io overview; repo detection + stub.
|
||||
|
||||
---
|
||||
|
||||
### 2.6 Claude Desktop — `mcp-bridge-only` — confidence: high
|
||||
|
||||
**What it is.** Anthropic's first-party GUI desktop chat app for macOS + Windows (no official Linux client as of
|
||||
2026). A desktop app — **not** a CLI or IDE. Its **only** extensibility surface is MCP servers (via
|
||||
`claude_desktop_config.json`) and one-click `.mcpb`/`.dxt` Desktop Extension bundles. **There is no hooks/settings.json surface.**
|
||||
|
||||
**Config surface.** `claude_desktop_config.json` — macOS `~/Library/Application Support/Claude/`, Windows
|
||||
`%APPDATA%\Claude\`. Integration goes under top-level `"mcpServers"`:
|
||||
`{ "mcpServers": { "hive-mind": { "command":"node", "args":["<memory-mcp dist/index.js>"], "env":{"HIVE_MIND_WORKSPACE_ID":"..."} } } }`.
|
||||
stdio transport for local servers; Streamable HTTP for remote. Alternative packaging: an `.mcpb` bundle (manifest
|
||||
spec 0.3) for drag-into-Settings install. **Note:** `HOOK_POINTER_BY_TOOL['claude-desktop'] = '.config/Claude/...'`
|
||||
is **WRONG** — not a real Claude Desktop config dir on any platform; needs fixing if a real installer ships.
|
||||
|
||||
**Lifecycle mapping.** **All four → none.** No SessionStart event (only a startup prompt Claude *may* invoke —
|
||||
model-discretion, non-deterministic). No per-prompt hook (raw prompt never handed to the server on submit, so
|
||||
"save every prompt" is impossible). No Stop/turn-completed event (no deterministic turn summarization). No exposed
|
||||
compaction lifecycle (`compact_memory` only via manual or Waggle cron, never host-driven). The `.mcpb` manifest 0.3
|
||||
exposes only tools/resources/prompts — **no event-handler field** — so a bundle can't register handlers either.
|
||||
|
||||
**Recommended approach.** **Do NOT port the hook shim** — there is nothing to register into. Replace the hooks
|
||||
package with an **MCP-bridge installer**. **Critical reuse:** Waggle already ships the exact server —
|
||||
`packages/memory-mcp` (`src/index.ts` + `tools/{memory,awareness,identity,harvest,ingest,knowledge,wiki,workspace}.ts`
|
||||
+ `resources/memory.ts`) exposes the full hive-mind tool surface (`recall_memory`, `save_memory`, …) as a stdio MCP
|
||||
server. So the integration is a **thin config patcher**, not a new server: (1) port `paths.ts` → resolve the **real**
|
||||
config path (mac `~/Library/Application Support/Claude/claude_desktop_config.json`, Win `%APPDATA%\Claude\...`) —
|
||||
rewrite, don't reuse the wrong `.config/Claude` pointer. (2) New settings-merger → additively merge an entry under
|
||||
`mcpServers` (not `hooks`), marker-tag for reversible uninstall; the immutable-merge + timestamped-backup +
|
||||
pointer-file + byte-identical-uninstall **shape** ports cleanly even though the target key changes. (3) shim-core is
|
||||
**mostly NOT reusable** — CliBridge/runHook/readStdin/hook-handler scaffolding is dead weight; only the generic
|
||||
logger and the backup/pointer filesystem pattern carry over. (4) Optionally also produce an `.mcpb` bundle for
|
||||
one-click install. **Honest §10/README framing:** this captures **memory on-demand** (Claude voluntarily calling
|
||||
`recall_memory`/`save_memory`) — strictly weaker than the every-turn deterministic capture of the CC shim. Mark the
|
||||
package "MCP-bridge, partial capture," do not imply hook parity.
|
||||
|
||||
**Blockers.**
|
||||
- No lifecycle hook/event API — the 4-hook automatic-capture model is fundamentally unportable.
|
||||
- MCP context is model-initiated, not server-pushed — no deterministic SessionStart inject, no raw prompt on submit, no turn on Stop → silent/automatic capture impossible.
|
||||
- `.mcpb` manifest 0.3 exposes only tools/resources/prompts (no lifecycle field) — bundles can't register handlers.
|
||||
- **Existing artifact conflict:** `HOOK_POINTER_BY_TOOL` uses a non-existent `.config/Claude/...` dir → detector's hook-installed check would be wrong; fix if a real installer ships.
|
||||
- **Redundancy risk:** `packages/memory-mcp` already provides the server → a separate hooks-claude-desktop package is mostly a config patcher. **Confirm with PM** whether to keep it as a package or fold the install into the existing launcher/MCP-registration flow before building.
|
||||
- PreCompact/`compact_memory` can't tie to host compaction — needs Waggle-side cron, out of band.
|
||||
|
||||
**Sources.** `support.claude.com/.../local-mcp-servers-on-claude-desktop`;
|
||||
`anthropic.com/engineering/desktop-extensions`; `github.com/modelcontextprotocol/mcpb`;
|
||||
`blog.modelcontextprotocol.io/posts/2025-11-20-adopting-mcpb`; `mcpbundles.com/docs/concepts/mcpb-files`;
|
||||
`code.claude.com/docs/en/hooks` (CC hooks ≠ desktop app); `github.com/desktop/desktop/issues/22138`;
|
||||
repo detection (wrong pointer) + `packages/memory-mcp/**` + stub.
|
||||
|
||||
---
|
||||
|
||||
## 3. Implementation Tiers
|
||||
|
||||
### Tier A — `near-direct port` (reuse shim-core + handlers; rewrite only the 3 install modules + field renames)
|
||||
**Tools:** Codex CLI, Codex Desktop, Cursor.
|
||||
**Rationale:** All three have native first-party lifecycle hooks with a stdin-JSON / stdout-JSON / command-hook /
|
||||
exit-0 model that is structurally identical to the Wave 1 reference. Codex's schema is a deliberate CC clone (events
|
||||
match exactly); Cursor needs only payload-field renames and a separate `hooks.json` path. Codex CLI + Desktop share
|
||||
`~/.codex/` and collapse into one package. ~80% reuse, ~1 day each (Codex CLI is the anchor; Desktop is a thin
|
||||
re-export; Cursor is a parallel clone).
|
||||
|
||||
### Tier B — `lifecycle-hooks-need-adapter` (events exist but config format/registration model is a rewrite; one event degraded/missing)
|
||||
**Tools:** Hermes Agent, OpenClaw.
|
||||
**Rationale:** Both have real lifecycle events and the shim-core + handler *bodies* still reuse, but the install
|
||||
layer is a genuine rewrite, not a parameterization — Hermes is YAML (no byte-identical round-trip; no PreCompact
|
||||
event), OpenClaw is JSON5 + in-process TypeScript handler files (no stdin command-array; fail-open becomes try/catch;
|
||||
looser Stop boundary). Both also have medium confidence (docs-only on key specifics) and require detector/cohort
|
||||
plumbing changes (Hermes must be added to `HOOKS_COHORT`).
|
||||
|
||||
### Tier C — `mcp-bridge-only` (no hook surface; register an MCP server; partial, on-demand capture)
|
||||
**Tools:** Claude Desktop.
|
||||
**Rationale:** No lifecycle hook API of any kind. The deliverable is an MCP-server config patcher wrapping the
|
||||
already-built `packages/memory-mcp`, yielding voluntary model-initiated capture only — strictly weaker than the
|
||||
every-turn deterministic capture of the shim. Must be labeled "partial capture." Open PM question: keep as a package
|
||||
or fold into the existing MCP-registration flow.
|
||||
|
||||
### Tier D — `not-feasible-now`
|
||||
**Tools:** *(none)*.
|
||||
**Rationale:** Every tool has at least a partial integration path. None is blocked outright at this time.
|
||||
|
||||
---
|
||||
|
||||
## 4. Recommended Implementation Order (easiest + highest-value first)
|
||||
|
||||
1. **Codex CLI (`hive-mind-hooks-codex`)** — highest reuse (schema is a CC clone), full 4-hook lifecycle, native `additionalContext` inject. Build this first; it becomes the second reference shape (CC-clone-with-create-if-missing) for the rest of the cohort.
|
||||
2. **Codex Desktop (`hive-mind-hooks-codex-desktop`)** — near-free once Codex CLI lands: thin re-export over the same `~/.codex/` installer. Bundle the `tool-detection.ts` fix (config-presence detection + correct pointer) here.
|
||||
3. **Cursor (`hive-mind-hooks-cursor`)** — full lifecycle, near-verbatim port, only payload-field renames + a separate `hooks.json` path + create-if-missing. High-value (Cursor is a widely used editor) and low-risk.
|
||||
4. **Hermes (`hive-mind-hooks-hermes`)** — first Tier-B: reuse handlers, rewrite install for YAML, drop PreCompact, add to `HOOKS_COHORT`, ship a real bin. Medium confidence — budget a live-payload verification spike.
|
||||
5. **OpenClaw (`hive-mind-hooks-openclaw`)** — most install-layer rework (JSON5 + in-process TS handlers, re-authored fail-open, Stop debounce, provenance/dedup story). Defer until a live OpenClaw install can validate the handler API.
|
||||
6. **Claude Desktop (`hive-mind-hooks-claude-desktop`)** — last, and gate it on a **PM decision** (package vs fold-into-launcher). It is a different deliverable (MCP bridge, partial capture), so don't let it block the four real hook ports. Fix the wrong `HOOK_POINTER_BY_TOOL` entry as part of whatever ships.
|
||||
|
||||
---
|
||||
|
||||
## 5. Key Risks
|
||||
|
||||
1. **Confidence asymmetry.** Tier A (Codex×2, Cursor) is high-confidence and doc-corroborated. Tier B (Hermes, OpenClaw) is **medium** — key specifics (Hermes shell-hook `VALID_HOOKS` allow-list; OpenClaw `handler.ts` event type + `execFileAsync`-under-loop reliability + `bootstrapFiles` injection sanction) are docs-only, not source-verified. Budget a live-payload/handler verification spike before committing Tier B estimates.
|
||||
2. **`tool-detection.ts` carries known-wrong data.** `HOOK_POINTER_BY_TOOL` for **claude-desktop** (`.config/Claude/...`) and **codex-desktop** (`.config/Codex/...`) point at non-existent dirs; codex-desktop binary candidate paths are admitted guesses for an unreleased app. These must be corrected or the "is-it-installed" check lies. Prefer config-presence detection over binary-path guessing for the desktop apps.
|
||||
3. **Create-if-missing is a behavioral fork from the Wave 1 reference.** The CC installer hard-requires a pre-existing config and throws if absent. Codex/Cursor/Hermes config files are optional/standalone and may not exist — every Tier A/B installer needs create-if-absent + the matching "delete-if-we-created vs restore-backup-if-existed" uninstall logic. Easy to get subtly wrong (orphaned files on uninstall).
|
||||
4. **Not-silent installs.** Codex (CLI + Desktop) non-managed hooks require a one-time `/hooks` trust step; Cursor likely needs a restart for `hooks.json` to take effect. Neither is fully silent like CC — install UX must surface the manual step or the hook silently no-ops.
|
||||
5. **Degraded/missing events.** Hermes has **no PreCompact** (approximate via opportunistic `compact_memory`). Cursor PreCompact is observational (can't guarantee before-truncation). Cursor UserPromptSubmit can't inject. OpenClaw Stop is 0..N/turn (needs debounce). Document each gap rather than implying parity.
|
||||
6. **Config round-trip fidelity.** Hermes YAML and OpenClaw JSON5 don't survive naive `parse→stringify` (comments/ordering lost). Rely on literal byte-identical backups for uninstall rather than diff-merge fidelity; for OpenClaw, prefer writing hook directories and touching config minimally.
|
||||
7. **Claude Desktop is a category mismatch.** Marketing/§10 must not imply hook parity — it is on-demand, model-discretion capture. There's also a redundancy decision (vs `packages/memory-mcp` + the existing launcher MCP-registration flow) that should be resolved by PM before any code is written.
|
||||
8. **OpenClaw double-counting.** OpenClaw can drive claude-code/codex as backends; if those backends also have hive-mind hooks installed, the gateway-layer capture double-counts the same conversation. Needs a provenance/dedup story before shipping.
|
||||
|
||||
---
|
||||
|
||||
## 6. OSS-Mirror Implication
|
||||
|
||||
All seven `hive-mind-hooks-*` packages are part of the **hive-mind OSS split** (per `CLAUDE.md` §7.5) — the public
|
||||
mirror at [`marolinik/hive-mind`](https://github.com/marolinik/hive-mind) is **generated from this monorepo via
|
||||
`git subtree split`**, so whatever lands in these packages is byte-identical in the OSS mirror and there is no
|
||||
cross-repo drift to police.
|
||||
|
||||
Implications for this work:
|
||||
- **Everything built here ships publicly.** The Codex/Cursor/Hermes/OpenClaw hook installers and the Claude Desktop
|
||||
MCP bridge become part of the open-source `hive-mind` surface. That is consistent with the strategy (memory +
|
||||
harvest is the free-forever moat; broad AI-tool reach amplifies it) — but it means **no proprietary/KVARK-gated
|
||||
logic** belongs in these packages. The subtree-split filter (`scripts/oss-subtree-split.sh`) already excludes
|
||||
`vault.ts`, `evolution-runs.ts`, `execution-traces.ts`, `improvement-signals.ts`, `compliance/**`; the new hook
|
||||
packages have no such concerns (they only shell to `hive-mind-cli`) — keep it that way.
|
||||
- **`@waggle/hive-mind-shim-core` is the shared, reused-as-is dependency** across all Tier A/B ports — it too lives in
|
||||
the OSS split, so its tool-agnostic CliBridge/logger stay public. Reusing it (rather than per-tool copies) keeps
|
||||
the OSS mirror DRY and the subtree-split clean.
|
||||
- **Per CLAUDE.md §7.5, do not resurrect the deprecated dual-repo sync workflows** — the export is one-directional
|
||||
(monorepo → mirror). Author everything in `packages/hive-mind-hooks-*`; the mirror follows automatically.
|
||||
|
||||
---
|
||||
|
||||
*This brief gates a human scoping decision. Tier A (Codex×2 + Cursor) is the safe, high-value first slice; Tier B
|
||||
needs a verification spike; Claude Desktop needs a PM package-vs-fold decision and an honest "partial capture" label.*
|
||||
783
docs/superpowers/specs/2026-06-01-wave23-hook-stubs-design.md
Normal file
783
docs/superpowers/specs/2026-06-01-wave23-hook-stubs-design.md
Normal file
@@ -0,0 +1,783 @@
|
||||
# Wave 2/3 hive-mind Hook Ports — Design Specification
|
||||
|
||||
**Date:** 2026-06-01
|
||||
**Status:** Approved design, pre-plan (implementation-ready; no code written yet)
|
||||
**Reference package (FROZEN):** `packages/hive-mind-hooks-claude-code` (Wave 1, shipped)
|
||||
**Shared dependency (reused as-is):** `@waggle/hive-mind-shim-core` (`packages/hive-mind-shim-core`)
|
||||
**Author:** Design synthesis over the approved decision set + source-verified Tier B facts (Hermes + OpenClaw real repos)
|
||||
|
||||
> This spec expands the approved design verbatim. The approved scope, the three locked decisions
|
||||
> (D1/D2/D3), and the per-package plan are not relitigated here — they are made implementation-ready
|
||||
> and the source-verified Tier B facts are folded in. Where verification *contradicted* an assumption,
|
||||
> the contradiction is surfaced explicitly in §5 and §9 (and reflected in the structured `designImpact`
|
||||
> fields), never papered over.
|
||||
|
||||
---
|
||||
|
||||
## 1. Summary
|
||||
|
||||
### What
|
||||
|
||||
Build **five** new hive-mind hook packages, composed over **one** new shared package:
|
||||
|
||||
| New package | Tier | Reuse model |
|
||||
|---|---|---|
|
||||
| `@waggle/hive-mind-hooks-codex` | A | CC-clone JSON installer; second reference shape |
|
||||
| `@waggle/hive-mind-hooks-codex-desktop` | A | Thin re-export of the codex installer (shared `~/.codex/`) |
|
||||
| `@waggle/hive-mind-hooks-cursor` | A | JSON installer with field renames + degraded events |
|
||||
| `@waggle/hive-mind-hooks-hermes` | B | Bespoke YAML codec; no PreCompact event |
|
||||
| `@waggle/hive-mind-hooks-openclaw` | B | Bespoke JSON5 config + in-process TS handlers |
|
||||
| `@waggle/hive-mind-hooks-core` | — | NEW shared package; the 5 above consume it |
|
||||
|
||||
**Excluded:** `claude-desktop` (MCP-bridge category mismatch — no hook surface, on-demand/partial capture
|
||||
only). It is **deferred** and out of scope for this spec; the only claude-desktop work in scope is a TODO
|
||||
marker on its known-wrong `tool-detection.ts` pointer (see §6.3).
|
||||
|
||||
### Why
|
||||
|
||||
The Wave 1 claude-code hook package proves the silent-capture pattern (every-turn deterministic capture of
|
||||
SessionStart / UserPromptSubmit / Stop / PreCompact into hive-mind frames via `hive-mind-cli`). Five more AI
|
||||
tools have a real lifecycle-hook surface; porting the pattern widens the free-forever memory+harvest moat
|
||||
(CLAUDE.md §1 "Moat strategy") across the AI-tool ecosystem. All five packages are part of the hive-mind OSS
|
||||
split (CLAUDE.md §7.5) and ship publicly via subtree-split — so they must be generic, shell only to
|
||||
`hive-mind-cli`, and carry no proprietary/KVARK logic.
|
||||
|
||||
The Wave 1 package (`packages/hive-mind-hooks-claude-code`) is **FROZEN** — it is the reference shape; do NOT
|
||||
modify it. The accepted tradeoff (D3) is that its reversible-install logic is duplicated by the new
|
||||
`hooks-core` rather than retrofitted into CC.
|
||||
|
||||
### Shape of the work
|
||||
|
||||
The per-tool work is almost entirely confined to three concerns — **config paths**, **config codec /
|
||||
register-shape**, and **event-name mapping** — composed over shared primitives in `hooks-core`. The four
|
||||
lifecycle handler *bodies* (recall+inject, save-temporary-frame, summarize+save-important-frame,
|
||||
compact_memory) are parameterized by a per-tool `EventAdapter` and reuse shim-core's `runHook` + `CliBridge`
|
||||
unchanged for the stdin-JSON/exit-0 tools (codex, codex-desktop, cursor, hermes). OpenClaw is the exception:
|
||||
its handlers are in-process TypeScript, so it cannot reuse `runHook` as-is and needs a thin in-process wrapper
|
||||
(§5.5).
|
||||
|
||||
---
|
||||
|
||||
## 2. Decision Log
|
||||
|
||||
### D1 — Scope = Tier A + Tier B (the 5 packages)
|
||||
|
||||
**Decision:** Build `hive-mind-hooks-{codex, codex-desktop, cursor, hermes, openclaw}`.
|
||||
**Rationale:** All five have a real, deterministic lifecycle-hook surface and reuse ≥80% of the Wave 1
|
||||
package (shim-core + handler bodies + the runHook contract, except OpenClaw's handler model). Claude Desktop
|
||||
is excluded because it has *no* hook API — it is an MCP-server registration yielding voluntary,
|
||||
model-initiated, on-demand capture (strictly weaker than every-turn deterministic capture), a different
|
||||
deliverable that must not block the four real hook ports and carries a separate PM package-vs-fold decision.
|
||||
|
||||
### D2 — Tier B (hermes, openclaw) = source-verify the public repos FIRST
|
||||
|
||||
**Decision:** Before committing Tier B estimates, read the real Hermes and OpenClaw repositories to confirm
|
||||
config format, event names, handler model, fail-open mechanism, and external-CLI-invocation feasibility.
|
||||
**Rationale:** The feasibility brief rated Tier B *medium* confidence — key specifics (Hermes shell-hook
|
||||
`VALID_HOOKS` allow-list; OpenClaw `handler.ts` event type, `execFileAsync`-under-loop reliability,
|
||||
`bootstrapFiles` injection sanction) were docs-only, not source-read. **Status: this verification is DONE.**
|
||||
Results are folded into §5.4 (Hermes) and §5.5 (OpenClaw) with `confidenceAfterVerify: high` for both. The
|
||||
verification **confirmed** the Hermes design and **required two corrections** to the OpenClaw design (handler
|
||||
model + one event mismap) — see §5.5 and §9.
|
||||
|
||||
### D3 — Architecture = new shared `@waggle/hive-mind-hooks-core`; CC left as-is
|
||||
|
||||
**Decision:** Create a new shared package `@waggle/hive-mind-hooks-core` exporting the reversible-install
|
||||
primitives, the shared lifecycle handler bodies (parameterized by an `EventAdapter`), and a `jsonRegister`
|
||||
helper. The five new packages consume it. The Wave 1 claude-code package is NOT refactored to consume it.
|
||||
**Rationale:** Extracting shared logic into `hooks-core` keeps the five new packages thin (paths + codec +
|
||||
register-shape + bin + tests) and DRY, and keeps the OSS subtree-split clean (one shared package, not five
|
||||
copies). **Accepted tradeoff:** the reversible-install logic now exists in *two* places — the frozen CC
|
||||
package and `hooks-core`. This duplication is deliberate; retrofitting CC is explicitly out of scope and
|
||||
risks regressing a shipped, regression-locked package.
|
||||
|
||||
---
|
||||
|
||||
## 3. Architecture — `@waggle/hive-mind-hooks-core`
|
||||
|
||||
`hooks-core` is a tool-agnostic library package (no `bin`; consumed by the five tool packages). It mirrors the
|
||||
naming and idioms of the Wave 1 CC modules (`install.ts`, `uninstall.ts`, `verify.ts`, `paths.ts`,
|
||||
`settings-merger.ts`, `hooks/_shared.ts`) so the per-package code reads like the reference. It depends on
|
||||
`@waggle/hive-mind-shim-core` and re-uses its `runHook`, `CliBridge`, `createCliBridge`, `encodeFrame`,
|
||||
`summarizeTurn`, `classifyImportance`, `maybeEmitDiscovery`, and `createLogger` (the same surface the CC hooks
|
||||
import today — see `packages/hive-mind-shim-core/src/index.ts`).
|
||||
|
||||
### 3.0 Package metadata
|
||||
|
||||
- `name: "@waggle/hive-mind-hooks-core"`, `version: "0.1.0"`, `type: "module"`, `license: "Apache-2.0"`,
|
||||
`main: "dist/index.js"`, `types: "dist/index.d.ts"`, `engines.node >= 20`, `publishConfig.access: "public"`.
|
||||
- `dependencies`: `@waggle/hive-mind-shim-core: "*"`, plus a YAML parser for the hermes codec and a JSON5
|
||||
parser for the openclaw codec **(see §9 OQ-3 — whether the YAML/JSON5 deps live in `hooks-core` or only in
|
||||
the hermes/openclaw consumer packages is an open question; default: keep codec-specific deps out of
|
||||
`hooks-core` and in the consumers, so `hooks-core` stays codec-agnostic and the JSON tools pull no YAML
|
||||
dep)**.
|
||||
- `peerDependencies`: `@waggle/hive-mind-cli: "*"` (optional, as in CC) — handlers shell to it at runtime.
|
||||
- No `bin`. Exports a barrel (`.`) plus per-primitive subpath exports if needed by consumers.
|
||||
|
||||
### 3.1 Reversible-install primitives (`install-core.ts` / `paths-core.ts`)
|
||||
|
||||
These generalize the Wave 1 CC logic. The CC reference assumes the config file **must pre-exist** and throws
|
||||
if absent (`install.ts` lines 85–90). `hooks-core` must add a **create-if-missing** mode and the matching
|
||||
uninstall semantics (delete-if-we-created vs restore-backup-if-existed), because codex/cursor/hermes config
|
||||
files are optional/standalone and may not exist on a fresh machine.
|
||||
|
||||
```ts
|
||||
// backupByteIdentical — write a byte-identical timestamped backup of an existing file.
|
||||
// Returns the backup path. No-op marker when the source did not exist (createdByUs case).
|
||||
export function backupPathFor(configPath: string, isoTimestamp: string): string;
|
||||
// → `${configPath}.hive-mind-backup.${iso.replace(/[:.]/g, '-')}` (mirrors CC paths.ts:86)
|
||||
|
||||
export async function backupByteIdentical(
|
||||
configPath: string,
|
||||
isoTimestamp: string,
|
||||
): Promise<{ backupPath: string | null; preExisted: boolean }>;
|
||||
// preExisted=false → no backup written; caller records createdByUs=true in the pointer.
|
||||
// preExisted=true → backup written with the exact original bytes (CC install.ts:104-106 idiom).
|
||||
|
||||
// Pointer file — records what we did so uninstall is exact. Superset of CC's pointer.
|
||||
export interface InstallPointer {
|
||||
version: string;
|
||||
installed_at: string;
|
||||
config_path: string;
|
||||
settings_backup: string | null; // null ⇔ created_by_us=true
|
||||
created_by_us: boolean; // NEW vs CC: true if the config file did not pre-exist
|
||||
hooks_dir: string | null; // null for in-process tools (openclaw)
|
||||
installed_hooks: readonly string[];
|
||||
cli_path: string | null;
|
||||
extra?: Record<string, unknown>; // per-tool: e.g. openclaw hook dir names, hermes registered event keys
|
||||
}
|
||||
export async function writePointer(pointerPath: string, pointer: InstallPointer): Promise<void>;
|
||||
export async function readPointer(pointerPath: string): Promise<InstallPointer>; // throws if malformed/absent
|
||||
|
||||
// restoreFromBackup — round-trip-verified restore. Mirrors CC uninstall.ts:71-89:
|
||||
// write backup bytes over configPath, re-read, assert SHA-256 / byte equality, refuse to delete
|
||||
// the backup unless the readback matches.
|
||||
export async function restoreFromBackup(args: {
|
||||
configPath: string;
|
||||
pointer: InstallPointer;
|
||||
cleanupBackup?: boolean; // default true
|
||||
}): Promise<{ restoredFrom: string | null; createdRemoved: boolean; backupRemoved: boolean }>;
|
||||
// created_by_us=true → DELETE the config file we created (never orphan it); restoredFrom=null.
|
||||
// created_by_us=false → restore the backup byte-identically (CC behavior).
|
||||
|
||||
// normalizeCliPath — reject embedded double-quotes (they break `--cli-path "<value>"` quoting).
|
||||
// Lifted verbatim from CC install.ts:142-154.
|
||||
export function normalizeCliPath(input: string | undefined): string | undefined;
|
||||
```
|
||||
|
||||
**Create-if-missing + uninstall semantics (precise):**
|
||||
|
||||
1. **Install, config pre-existed:** write byte-identical backup → merge/register hive entries → write merged
|
||||
config → pointer with `created_by_us=false`, `settings_backup=<backup>`.
|
||||
2. **Install, config absent:** create a minimal valid skeleton (per-tool: `{version:1,hooks:{}}` for cursor,
|
||||
`{hooks:{}}` for codex, `{}` for hermes YAML, JSON5 `{hooks:{internal:{...}}}` for openclaw) containing
|
||||
*only* hive entries → pointer with `created_by_us=true`, `settings_backup=null`.
|
||||
3. **Uninstall, `created_by_us=false`:** restore backup byte-identically (round-trip verified), delete
|
||||
backup + pointer.
|
||||
4. **Uninstall, `created_by_us=true`:** delete the config file we created, delete pointer. **Never orphan a
|
||||
file we created and never leave a backup behind.**
|
||||
5. **Re-install (upgrade) idempotency:** if a pointer already exists, treat as upgrade — replace the
|
||||
marker-tagged hive entries in place (mirrors CC `mergeHiveHooks` replace-in-place, settings-merger.ts:85-93)
|
||||
rather than duplicating, and keep the original backup so uninstall still restores the true pre-install
|
||||
state.
|
||||
|
||||
### 3.2 `EventAdapter` interface (`event-adapter.ts`)
|
||||
|
||||
The four lifecycle handler bodies are tool-agnostic and parameterized by a per-tool `EventAdapter` — field
|
||||
extractors (which incoming payload keys hold the cwd / prompt / response / session id / parent) plus the
|
||||
event-name map. This generalizes what the CC handlers already do inline via `pickStringFromObject` multi-key
|
||||
fallbacks (e.g. `stop.ts:37-42` reads `response | assistant_message | transcript`).
|
||||
|
||||
```ts
|
||||
export type Lifecycle = 'session-start' | 'user-prompt-submit' | 'stop' | 'pre-compact';
|
||||
|
||||
export interface EventAdapter {
|
||||
/** Tool id used for HookEvent.source + logger names. */
|
||||
readonly source: ShimSource; // 'codex' | 'cursor' | 'hermes' | 'openclaw' | ...
|
||||
/** Map our canonical lifecycle name → the tool's native event key. undefined ⇒ event not supported. */
|
||||
readonly eventName: Record<Lifecycle, string | undefined>;
|
||||
/** Field extractors over the opaque incoming payload (returns undefined when absent). */
|
||||
extractCwd(payload: unknown): string | undefined;
|
||||
extractSessionId(payload: unknown): string | undefined;
|
||||
extractPrompt(payload: unknown): string | undefined; // UserPromptSubmit
|
||||
extractResponse(payload: unknown, ctx: { readFile?: (p: string) => Promise<string> }): Promise<string | undefined> | string | undefined;
|
||||
// async because Cursor delivers the turn via base.transcript_path (read off disk), not inline.
|
||||
extractParent(payload: unknown): string | undefined; // Stop parent frame id, if any
|
||||
/** Per-tool shape of the SessionStart inject response (some tools differ). undefined ⇒ no inject. */
|
||||
formatInject?(additionalContext: string): unknown;
|
||||
}
|
||||
```
|
||||
|
||||
`ShimSource` is the existing shim-core union (`hook-event-types.ts`); it already includes `'claude-code'`,
|
||||
`'cursor'`, `'openclaw'` (per the signal-emitter doc comment). **Verify the union includes `'codex'` and
|
||||
`'hermes'`; if not, extend it in shim-core** (small additive change to the OSS shim-core — acceptable; see
|
||||
§9 OQ-2).
|
||||
|
||||
### 3.3 Shared handler bodies (`handlers-core.ts`)
|
||||
|
||||
Four factory functions return shim-core `HookHandler` objects parameterized by an `EventAdapter`. Each mirrors
|
||||
the corresponding CC hook body (`hooks/session-start.ts`, `user-prompt-submit.ts`, `stop.ts`, `pre-compact.ts`)
|
||||
but reads fields through the adapter instead of hardcoded key lists.
|
||||
|
||||
```ts
|
||||
export function makeSessionStartHandler(a: EventAdapter, opts?: { recallLimit?: number }): HookHandler<...>;
|
||||
// run: bridge.recallMemory('', { limit, scope: 'personal' }) → format hits → a.formatInject(text)
|
||||
// (default formatInject = CC's { hookSpecificOutput: { hookEventName, additionalContext } }).
|
||||
|
||||
export function makeUserPromptSubmitHandler(a: EventAdapter): HookHandler<...>;
|
||||
// run: encodeFrame({eventType:'user-prompt-submit', source:a.source, ...}, {importance:'temporary'})
|
||||
// → bridge.saveMemory(frame). No stdout. (mirrors user-prompt-submit.ts)
|
||||
|
||||
export function makeStopHandler(a: EventAdapter, opts?: { summaryBudgetChars?: number }): HookHandler<...>;
|
||||
// run: summarizeTurn(response) → classifyImportance → encodeFrame(importance:'important'|'critical', parent?)
|
||||
// → bridge.saveMemory → opt-in maybeEmitDiscovery on WAGGLE_SIGNAL_EMIT (mirrors stop.ts:46-108).
|
||||
|
||||
export function makePreCompactHandler(a: EventAdapter): HookHandler<...>;
|
||||
// run: bridge.cleanupFrames() (default mode 'compact'). (mirrors pre-compact.ts)
|
||||
```
|
||||
|
||||
These compose shim-core unchanged: `summarizeTurn`, `classifyImportance`, `encodeFrame`, `maybeEmitDiscovery`,
|
||||
`CliBridge.{recallMemory,saveMemory,cleanupFrames}`. The WAGGLE_SIGNAL_EMIT opt-in behavior (off by default,
|
||||
fail-open) is preserved exactly as in CC `stop.ts` so OSS consumers see no behavior change.
|
||||
|
||||
### 3.4 How `runHook` / `CliBridge` are composed
|
||||
|
||||
For the four stdin-JSON/exit-0 tools (codex, codex-desktop, cursor, hermes), each per-tool hook script is a
|
||||
thin entrypoint identical in shape to CC's `hooks/session-start.ts` etc.:
|
||||
|
||||
```ts
|
||||
// e.g. packages/hive-mind-hooks-codex/src/hooks/session-start.ts
|
||||
import { runHook } from '@waggle/hive-mind-shim-core'; // exact CC import
|
||||
import { makeSessionStartHandler } from '@waggle/hive-mind-hooks-core';
|
||||
import { codexAdapter } from '../adapter.js';
|
||||
export async function runSessionStart(opts = {}) {
|
||||
return runHook(makeSessionStartHandler(codexAdapter), { name: 'session-start', ...opts });
|
||||
}
|
||||
// + the CC isMain self-invoke guard verbatim (session-start.ts:91-102)
|
||||
```
|
||||
|
||||
`runHook` (shim-core `hooks/_shared` equivalent — actually re-exported from shim-core; the CC `_shared.ts`
|
||||
wraps it) already provides: stdin read with timeout, `safeJsonParse`, `--cli-path` argv parsing
|
||||
(`parseHookArgs`), `createCliBridge` wiring, **exit-0-always fail-open**, and stdout JSON emission. The
|
||||
per-tool packages reuse it as-is; `hooks-core` only supplies the adapter + handler bodies.
|
||||
|
||||
> Note on `_shared.ts`: the CC `hooks/_shared.ts` is itself a thin wrapper that re-exports/wraps shim-core's
|
||||
> `createCliBridge` + `createLogger` and defines `runHook`, `parseHookArgs`, `readStdinAsString`,
|
||||
> `safeJsonParse`, `pickStringFromObject`. To keep the new packages DRY, `hooks-core` should export an
|
||||
> equivalent `hook-shared.ts` (lifted from CC `_shared.ts`, which is not frozen-by-reference since it's a copy,
|
||||
> not the CC package — but to be safe, re-author it in `hooks-core` rather than import across package
|
||||
> boundaries). The five packages import `runHook`/`parseHookArgs` from `hooks-core`, not from the frozen CC
|
||||
> package.
|
||||
|
||||
### 3.5 `jsonRegister` helper (`json-register.ts`)
|
||||
|
||||
Generalizes CC's `mergeHiveHooks` (settings-merger.ts:72-101) for any JSON-config tool whose event keys map to
|
||||
**arrays of hook groups**. Additive merge + marker tag + dedup/replace-in-place.
|
||||
|
||||
```ts
|
||||
export const HIVE_MIND_MARKER_BASE = '@hive-mind'; // per-tool suffix appended, e.g. '@hive-mind/codex-hooks'
|
||||
|
||||
export interface JsonRegisterSpec {
|
||||
/** Top-level object key holding the per-event map (e.g. 'hooks'). */
|
||||
hooksKey: string;
|
||||
/** Canonical lifecycle → tool event-key map (from the EventAdapter.eventName). */
|
||||
eventName: Record<Lifecycle, string | undefined>;
|
||||
/** Builds the tool-shaped group object for one hook entry (codex uses {matcher,hooks:[...]}; cursor uses {command,type,timeout}). */
|
||||
buildGroup(lifecycle: Lifecycle, command: string, timeout: number): Record<string, unknown>;
|
||||
/** Reads the marker off a group to detect our own entries for replace/remove. */
|
||||
isHiveGroup(group: unknown): boolean;
|
||||
/** Optional wrapper to ensure skeleton (e.g. cursor needs {version:1}). */
|
||||
ensureSkeleton?(root: Record<string, unknown>): Record<string, unknown>;
|
||||
}
|
||||
|
||||
export function jsonRegister(
|
||||
config: Record<string, unknown> | undefined,
|
||||
entries: readonly { lifecycle: Lifecycle; command: string; timeout: number }[],
|
||||
spec: JsonRegisterSpec,
|
||||
): Record<string, unknown>; // returns a NEW object; never mutates input (CC immutability contract)
|
||||
|
||||
export function jsonUnregister(config, spec): Record<string, unknown>; // strips marker-tagged groups
|
||||
export function hasHiveEntries(config, spec): boolean;
|
||||
```
|
||||
|
||||
`jsonRegister`/`jsonUnregister` preserve all non-hive entries verbatim (preserves the user's existing hooks,
|
||||
exactly as CC's merge preserves `gsd-context-monitor.js`), and dedup our own marker-tagged group by
|
||||
`(eventKey, command)` so re-install upgrades in place. Used by codex, codex-desktop (via codex), and cursor.
|
||||
Hermes (YAML) and OpenClaw (JSON5 + dirs) do NOT use `jsonRegister` — they have bespoke codecs (§5.4, §5.5).
|
||||
|
||||
---
|
||||
|
||||
## 4. Lifecycle → tool-event mapping (canonical reference)
|
||||
|
||||
The four canonical lifecycle actions map to native tool events as follows. "—" = no native event (degraded;
|
||||
documented per §6.1). All five packages prove the two invariants in §7.2.
|
||||
|
||||
| Lifecycle (canonical) | claude-code (ref) | codex | codex-desktop | cursor | hermes | openclaw |
|
||||
|---|---|---|---|---|---|---|
|
||||
| **SessionStart** (recall+inject) | `SessionStart` | `SessionStart` | `SessionStart` | `sessionStart` | `on_session_start` (observe) **+** `pre_llm_call` `is_first_turn` (inject) | `agent:bootstrap` (mutate `bootstrapFiles`) |
|
||||
| **UserPromptSubmit** (save temp) | `UserPromptSubmit` | `UserPromptSubmit` | `UserPromptSubmit` | `beforeSubmitPrompt` (save-only) | `pre_llm_call` | `message:received` |
|
||||
| **Stop** (summarize+save) | `Stop` | `Stop` | `Stop` | `stop` (turn via `transcript_path`) | `post_llm_call` | `message:sent` (0..N/turn — debounce) |
|
||||
| **PreCompact** (compact_memory) | `PreCompact` | `PreCompact` | `PreCompact` (+`PostCompact` bonus) | `preCompact` (observational) | **— (none)** | `session:compact:before` (match `event.action==='compact:before'`) |
|
||||
|
||||
---
|
||||
|
||||
## 5. Per-Package Designs
|
||||
|
||||
### 5.1 `@waggle/hive-mind-hooks-codex` (Tier A — second reference shape)
|
||||
|
||||
- **Config surface:** standalone `~/.codex/hooks.json` (JSON). Do NOT touch `~/.codex/config.toml` (keeps us
|
||||
out of the user's TOML and away from protected `notify`/`profile`/`model_providers` keys). Pointer:
|
||||
`~/.codex/hive-mind-install.json`.
|
||||
- **Codec:** plain JSON (`JSON.parse`/`JSON.stringify(_, null, 2) + '\n'`, exactly CC install.ts:116).
|
||||
- **Register/merge shape:** `jsonRegister` with `hooksKey='hooks'`, group shape
|
||||
`{ matcher, hooks: [{ type:'command', command, timeout }], _hiveMindShim }`. `matcher='startup|resume|clear|compact'`
|
||||
for SessionStart, `''` (or omitted) elsewhere. Hooks across config layers are additive; the marker is kept
|
||||
for byte-identical reversible uninstall, not for correctness.
|
||||
- **Event map:** SessionStart→`SessionStart`, UserPromptSubmit→`UserPromptSubmit`, Stop→`Stop`,
|
||||
PreCompact→`PreCompact` (a deliberate CC clone — `HOOK_EVENT_BY_BASENAME` is effectively identical to CC).
|
||||
- **Adapter field extractors:** reuse CC's snake_case keys (`prompt`, `session_id`, `cwd`) plus codex
|
||||
additions: Stop reads `last_assistant_message` (added to the response fallback list); PreCompact reads
|
||||
`trigger` (`manual|auto`); `HookEvent.source='codex'`.
|
||||
- **Degraded/missing events:** none — all four native and field-compatible.
|
||||
- **Install UX (non-silent steps):** non-managed Codex hooks require a **one-time `/hooks` trust step** (trust
|
||||
keyed by hook hash) before they execute. The installer MUST print: "Run `/hooks` in Codex once to trust the
|
||||
hive-mind hooks." Also `verify` must surface admin lockdown `allow_managed_hooks_only = true`
|
||||
(`requirements.toml`) which suppresses user hooks so install doesn't silently no-op.
|
||||
- **Uninstall:** `created_by_us` aware — delete `hooks.json` if we created it, else restore byte-identical
|
||||
backup (§3.1).
|
||||
- **Reuses from core:** `jsonRegister`, all reversible-install primitives, all four handler bodies, the
|
||||
`runHook` entrypoint shape, `--cli-path` Windows quoting (verbatim from CC `hookCommandFor`, paths.ts:74-84).
|
||||
- **Implements locally:** `paths.ts` (codex paths), `adapter.ts` (codex EventAdapter + JsonRegisterSpec),
|
||||
4 hook entrypoints, bin (`codex-hooks`).
|
||||
- **Blocker to note:** field-name casing confirmed from docs, not a live payload — add codex keys as
|
||||
*fallbacks* in the adapter rather than assuming. Minimum codex version shipping stable hooks is unpinned
|
||||
("stable as of May 2026") — `verify` should probe dynamically; a too-old codex silently ignores hooks.json.
|
||||
|
||||
### 5.2 `@waggle/hive-mind-hooks-codex-desktop` (Tier A — thin re-export)
|
||||
|
||||
- **Config surface:** SHARES `~/.codex/hooks.json` with codex CLI — there is no separate codex-desktop config
|
||||
root. Same pointer file `~/.codex/hive-mind-install.json`.
|
||||
- **Codec / register / events / adapter:** identical to codex — this package is a **thin re-export of the
|
||||
codex installer** (`export * from '@waggle/hive-mind-hooks-codex'` plus a bin that delegates). It exists as a
|
||||
distinct package so the dependency graph + subtree-split see the package boundary and so the launcher's
|
||||
`hookPackageFor('codex-desktop')` resolves to `@waggle/hive-mind-hooks-codex-desktop`.
|
||||
- **In-scope fix — `tool-detection.ts` codex-desktop pointer:** `HOOK_POINTER_BY_TOOL['codex-desktop']` in
|
||||
`packages/agent/src/tool-detection.ts` (line 194) is currently `'.config/Codex/hive-mind-install.json'`,
|
||||
which is wrong — codex-desktop shares `~/.codex/`. **Fix to `'.codex/hive-mind-install.json'`** so the
|
||||
"is-it-installed" probe reads the real pointer. (Prefer config-presence detection at `~/.codex/` over the
|
||||
speculative binary-path guesses; the binary candidate paths in detection are admitted guesses for the
|
||||
unreleased app — out of scope to fully rework here, but the pointer fix is in scope.)
|
||||
- **Install UX:** same `/hooks` trust note as codex. Add a note that the desktop App honoring hooks is
|
||||
doc-asserted but needs runtime verification on an actual App install (CLI definitely fires).
|
||||
- **Reuses from core / codex:** everything, transitively via the codex package.
|
||||
- **Implements locally:** package manifest + thin re-export + bin + the `tool-detection.ts` pointer fix + a
|
||||
parity test that asserts it installs into the same `~/.codex/hooks.json` as codex.
|
||||
|
||||
### 5.3 `@waggle/hive-mind-hooks-cursor` (Tier A — JSON with field renames + degraded events)
|
||||
|
||||
- **Config surface:** `~/.cursor/hooks.json` (JSON) — a **separate file** from Cursor's `settings.json`
|
||||
(editor prefs). Pointer: `~/.cursor/hive-mind-install.json` (already present in `HOOK_POINTER_BY_TOOL`).
|
||||
- **Codec:** plain JSON. Skeleton when absent: `{ "version": 1, "hooks": {} }` (`ensureSkeleton` in the
|
||||
`JsonRegisterSpec` adds `version:1`).
|
||||
- **Register/merge shape:** `jsonRegister` with group shape `{ command, type:'command', timeout, _hiveMindShim }`
|
||||
in `hooks.<event>` arrays (note: cursor uses a flat group, not codex's `{matcher,hooks:[]}` wrapper —
|
||||
`buildGroup` differs).
|
||||
- **Event map (field renames):** SessionStart→`sessionStart`, UserPromptSubmit→`beforeSubmitPrompt`,
|
||||
Stop→`stop`, PreCompact→`preCompact`.
|
||||
- **Degraded/missing events (MUST be documented in README per §6.1):**
|
||||
- `beforeSubmitPrompt` is **SAVE-ONLY** — it cannot inject context (stdout only `{continue, user_message}`).
|
||||
Acceptable: hive-mind's UserPromptSubmit only persists. Adapter `formatInject` is undefined for this event.
|
||||
- `preCompact` is **observational only** — it cannot block/reorder, so "run compact_memory BEFORE host
|
||||
truncates" is best-effort, not guaranteed-before. `compact_memory` runs fire-and-forget.
|
||||
- SessionStart `sessionStart` returns `{ additional_context, env }` — adapter `formatInject` returns
|
||||
`{ additional_context: text }` (a rename of CC's `hookSpecificOutput.additionalContext`).
|
||||
- Stop reads the completed turn via base field `transcript_path` (NOT inline) — `extractResponse` is async,
|
||||
reads the file defensively (tolerate null when transcripts are disabled; do not assume JSONL format), and
|
||||
fails open.
|
||||
- **Install UX:** editing `hooks.json` likely needs a **Cursor restart** for the hooks to take effect (reload
|
||||
semantics unverified across 1.7.x) — installer MUST print "Restart Cursor for hive-mind hooks to take
|
||||
effect." Windows `.cmd`-shim exec problem applies — thread `--cli-path` to the compiled
|
||||
`hive-mind-cli dist/index.js`.
|
||||
- **Uninstall:** `created_by_us` aware (cursor `hooks.json` may not exist on fresh install).
|
||||
- **Reuses from core:** `jsonRegister`, reversible-install primitives, three handler bodies as-is; Stop handler
|
||||
via an adapter whose `extractResponse` reads `transcript_path`.
|
||||
- **Implements locally:** `paths.ts`, `adapter.ts` (renames + transcript reader), 4 entrypoints, bin.
|
||||
|
||||
### 5.4 `@waggle/hive-mind-hooks-hermes` (Tier B — YAML; **source-verified, confidence: high**)
|
||||
|
||||
**Verification status:** source-verified against `NousResearch/hermes-agent` (MIT, Python, default branch
|
||||
`main`, pushed 2026-05-31). The verification **CONFIRMS the approved design** with two clarifications that do
|
||||
not break it (folded in below). `confidenceAfterVerify: high`.
|
||||
|
||||
- **Config surface:** `~/.hermes/config.yaml` (the path `hermes_cli/config.py get_config_path` resolves to;
|
||||
the `cli-config.yaml.example` ships the same `hooks:` block). The relevant system is **Shell hooks** — a
|
||||
top-level `hooks:` block in `config.yaml`. Pointer: `~/.hermes/hive-mind-install.json`.
|
||||
|
||||
> **CRITICAL gotcha (folded from verified facts):** Hermes has **THREE** hook systems sharing the name
|
||||
> "hooks". The design targets the **SHELL-HOOKS** system (`config.yaml` `hooks:` block, shell-out via
|
||||
> subprocess). Do NOT confuse it with (a) the directory-based **Gateway hooks**
|
||||
> (`~/.hermes/hooks/<name>/{HOOK.yaml,handler.py}`, colon-style event names `session:start`, gateway-only,
|
||||
> in-process Python) or (b) the in-process **Plugin hooks** (`ctx.register_hook`). Only the shell-hooks
|
||||
> system is in scope. (Source: `agent/shell_hooks.py`.)
|
||||
|
||||
- **Codec:** bespoke YAML. Add a YAML parser dep (e.g. `yaml`) to the hermes consumer package. **YAML
|
||||
round-trip is NOT byte-identical** for re-serialized output (comments/ordering lost), so reversibility relies
|
||||
on the **literal byte-identical backup** written by `backupByteIdentical` (the merged config is what we
|
||||
*write*; uninstall restores the *original bytes*, not a re-serialized merge). This is fully compatible with
|
||||
the §3.1 primitives.
|
||||
- **Register/merge shape:** additive, marker-tagged merge of hive entries into the YAML `hooks:` block,
|
||||
preserving the user's existing hook entries. Each event key → list of
|
||||
`{ command, timeout?, matcher? }`. Our entries set `command` to the per-event hook script invocation
|
||||
(`node "<dist>/hooks/<event>.js" --cli-path "..."` — same Windows-safe quoting as CC) and `timeout` (default
|
||||
60, hard cap 300). The marker is carried as a sentinel comment or a recognizable command prefix so
|
||||
`jsonUnregister`-equivalent YAML logic can strip exactly our entries on re-install/upgrade.
|
||||
- **Event map (verified exact strings — snake_case):**
|
||||
- **SessionStart → SPLIT** (as the design assumed): `on_session_start` (observer-only; return value ignored;
|
||||
fires once per NEW session, `conversation_loop.py` ~L294) registers the observe/no-op side, **and**
|
||||
`pre_llm_call` with `is_first_turn=true` (stdout `{"context":"..."}` is appended to the user message — NOT
|
||||
the system prompt, to preserve prefix cache; `conversation_loop.py` L687-721) carries the recall+inject.
|
||||
Docs explicitly state Claude Code's `UserPromptSubmit` maps to `pre_llm_call`. So the SessionStart adapter
|
||||
registers **two** event keys.
|
||||
- **UserPromptSubmit → `pre_llm_call`** (fires once/turn before the tool loop; `extra` carries
|
||||
`user_message`, `conversation_history`, `is_first_turn`). Fire-and-forget save.
|
||||
- **Stop → `post_llm_call`** (fires once/turn after the loop completes, only if `final_response` and not
|
||||
interrupted; `extra` carries `assistant_response`; `conversation_loop.py` L4566-4583). This is the reliable
|
||||
turn-end signal for a single-shot CLI run. **Note:** `on_session_finalize` is gateway-path only (fires at
|
||||
`/new`/`/reset`/expiry boundaries) — do NOT assume it fires on every CLI invocation; use `post_llm_call` as
|
||||
the dependable Stop analogue (verified).
|
||||
- **PreCompact → NONE.** **CONFIRMED ABSENT** by source read: no compaction hook in `VALID_HOOKS`; no
|
||||
`compact`/`precompact`/`pre_compact` token anywhere in `plugins.py` or `shell_hooks.py`; the only
|
||||
compaction source (`agent/conversation_compression.py`) emits no hook. The design's "NO PreCompact"
|
||||
assumption is correct — there is genuinely nothing to hook. The hermes adapter's
|
||||
`eventName['pre-compact'] = undefined`; document the gap, do NOT invent (optionally approximate via
|
||||
opportunistic `compact_memory` from the Stop handler — see §9 OQ-4).
|
||||
- **Handler model (CONFIRMED shell-command):** each firing spawns the configured `command` as a real OS
|
||||
subprocess — `argv = shlex.split(os.path.expanduser(command))`,
|
||||
`subprocess.run(argv, input=stdin_json, capture_output=True, timeout=..., text=True, shell=False)`. The JSON
|
||||
payload is piped to stdin; stdout is read back as optional JSON. This is **structurally identical** to the
|
||||
shim-core `runHook` stdin-JSON contract, so the four handler bodies + `runHook` reuse as-is.
|
||||
- **External CLI invocation (CONFIRMED — idiomatic):** the `command` is ANY executable via
|
||||
`shlex.split + shell=False`; docs list languages as "Any (Bash, Python, Go binary, …)". A hook command of
|
||||
`node "<dist>/hooks/stop.js"` (which itself shells `hive-mind-cli` via `CliBridge`) works directly — the
|
||||
event JSON arrives on stdin and stdout JSON is read back. **Two real constraints, neither blocking:**
|
||||
(1) `shell=False` means no pipes/redirection in the command string itself — wrap multi-step logic in the JS
|
||||
hook script (we already do). (2) **FIRST-USE CONSENT ALLOW-LIST** — each unique `(event, command)` string
|
||||
must be approved once. Under any **non-TTY / headless launch** (the Waggle launcher), the hook registers
|
||||
ONLY if one of `--accept-hooks`, `HERMES_ACCEPT_HOOKS=1`, or `hooks_auto_accept: true` is set — otherwise it
|
||||
silently stays unregistered with a warning.
|
||||
- **Fail-open mechanism (CONFIRMED, multi-layer):** config parsing warn-and-skips malformed entries (never
|
||||
raises); the subprocess layer catches Timeout/FileNotFound/Permission/Exception and returns None; non-zero
|
||||
exit is logged but stdout is still parsed; the dispatcher wraps each callback in try/except; every runtime
|
||||
firing site wraps `invoke_hook` in try/except. **Behavioral difference to fold in:** unlike Claude Code there
|
||||
is **NO exit-code-2 / special-exit contract** — control flow (block/inject) is expressed purely via
|
||||
**stdout JSON** (`{"action":"block"}` or `{"context":"..."}`; two block-shapes accepted). Our hooks are
|
||||
capture-only (no block, no inject except SessionStart context), so they just emit `{}`/context and the
|
||||
exit-0-always shim contract is fully compatible.
|
||||
- **Install UX (non-silent steps):**
|
||||
- The installer/launcher MUST set `HERMES_ACCEPT_HOOKS=1` (or write `hooks_auto_accept: true` into the
|
||||
config) under headless/gateway launch, or the hive-mind hooks **silently never register**. Surface this
|
||||
clearly. (Allowlist keys on the exact command STRING, not a script hash — editing the target script is
|
||||
silently trusted; only `hermes hooks doctor` surfaces mtime drift.)
|
||||
- `matcher:` is honored ONLY for `pre_tool_call`/`post_tool_call` — on our lifecycle events it is stripped
|
||||
with a warning; do not set it.
|
||||
- **Cohort plumbing (in scope per design):** add `'hermes'` to `HOOKS_COHORT` in
|
||||
`packages/agent/src/tool-launcher.ts` (currently `['claude-code']`, line 62) once the package ships a real
|
||||
bin, so the launcher routes hook install/verify/uninstall for hermes. **Note:** add each Tier-A/B tool to
|
||||
`HOOKS_COHORT` as its package ships a bin (codex, codex-desktop, cursor too) — the design singled out hermes
|
||||
because the feasibility brief flagged it as excluded, but the cohort gate applies to every newly-real
|
||||
package (see §9 OQ-1).
|
||||
- **Reuses from core:** all four-minus-PreCompact handler bodies, `runHook`, reversible-install primitives.
|
||||
- **Implements locally:** `paths.ts`, bespoke YAML `settings-merger`/codec, `adapter.ts` (hermes field
|
||||
extractors + 2-key SessionStart registration), 3 entrypoints (no pre-compact), bin, the `HOOKS_COHORT` edit.
|
||||
- **Re-pin note:** the hermes repo moves fast (verified against HEAD/`main`, pushed 2026-05-31); **re-pin to a
|
||||
commit SHA before relying on the cited line numbers** during implementation.
|
||||
|
||||
### 5.5 `@waggle/hive-mind-hooks-openclaw` (Tier B — JSON5 + in-process TS; **source-verified, two design corrections**)
|
||||
|
||||
**Verification status:** source-verified against `openclaw/openclaw`. The verification **REQUIRES CHANGES** —
|
||||
two material corrections to the approved design's event/handler assumptions (folded in below; also surfaced in
|
||||
§9). `confidenceAfterVerify: high`. The corrections do NOT change the package's scope; they change the
|
||||
host↔handler glue and one event name.
|
||||
|
||||
- **Config surface:** `~/.openclaw/openclaw.json` (**JSON5** — strict `JSON.parse` first, then `JSON5.parse`
|
||||
fallback; supports `$include` merges + `${ENV}` substitution). Hooks are configured under the
|
||||
`hooks.internal.*` tree **inside this one file**; the `hooks/<name>/{HOOK.md,handler.ts}` files are the hook
|
||||
*implementation* (discovered from directories), separate from config. Pointer:
|
||||
`~/.openclaw/hive-mind-install.json`.
|
||||
- **Codec:** bespoke JSON5. **Naive `JSON.parse`/`JSON.stringify` destroys user comments + trailing commas.**
|
||||
Mitigation: write the hive **hook directory** (`~/.openclaw/hooks/hive-mind/{HOOK.md,handler.ts}` — or a
|
||||
managed dir) and touch `openclaw.json` minimally (add our entry under `hooks.internal.entries` and/or
|
||||
`hooks.internal.load.extraDirs`); rely on the literal byte-identical backup for uninstall rather than
|
||||
re-serialization fidelity (compatible with §3.1).
|
||||
- **Register/merge shape:** discovery-based. Install writes a hook directory with `HOOK.md`
|
||||
(frontmatter declaring `metadata.openclaw.events[]`) + `handler.ts` (default export), and patches
|
||||
`hooks.internal.enabled=true` + an `entries["hive-mind"]={enabled:true, env:{...}}` (or `extraDirs`). **Hooks
|
||||
are OFF until opted in** — set `hooks.internal.enabled=true` and/or run `openclaw hooks enable hive-mind`.
|
||||
Record the created dir + config keys in the pointer's `extra` so uninstall removes exactly what we added.
|
||||
- **Event map (VERIFIED — match on `(type, action)` pair, NOT the joined string):**
|
||||
- **SessionStart → `agent:bootstrap`** — `event.context.bootstrapFiles` is a **MUTABLE array**; the handler
|
||||
pushes recalled frames onto it before bootstrap files are injected into the system prompt
|
||||
(`applyBootstrapHookOverrides` reads it back). This is the sanctioned injection seam (≈ CC
|
||||
`additionalContext`). Only recognized basenames load (`AGENTS.md`/`MEMORY.md`/etc.). CONFIRMED.
|
||||
- **UserPromptSubmit → `message:received`** (inbound message from any channel; `event.context` =
|
||||
`{from, content, channelId, ...}`; replyable). The true user-prompt analog. (`command:new` is the
|
||||
`/new`-reset analog, not generic prompt submit.) CONFIRMED.
|
||||
- **Stop → `message:sent`** — fires **ONCE PER OUTBOUND PAYLOAD DELIVERED** (`deliver.ts:1044`), i.e.
|
||||
**0..N per turn**. This is the design's "needs debounce" case; it is **NON-replyable** (pushed
|
||||
`event.messages[]` are ignored). The consumer must debounce/dedupe (e.g. "last `message:sent` of a turn"
|
||||
heuristic or a short timer). CONFIRMED.
|
||||
- **PreCompact → `session:compact:before`** — **but the runtime `event.action` is `'compact:before'`, NOT
|
||||
`'session:compact:before'`** (the `session:` prefix appears only in the `HOOK.md` `events[]` array). A
|
||||
handler matching the full joined string will silently never fire. Match `event.action === 'compact:before'`.
|
||||
CONFIRMED + corrected.
|
||||
|
||||
**CORRECTION 1 — `before_agent_finalize` is the WRONG system.** The approved design listed
|
||||
`before_agent_finalize` alongside the internal events, but it is a **typed PLUGIN hook**
|
||||
(`src/plugins/hook-types.ts` `PluginHookName` union, registered via `api.on('before_agent_finalize', …)`),
|
||||
NOT an internal `HOOK.md`/`handler.ts` event. The internal-hooks event union is exactly
|
||||
`command | session | agent | gateway | message` — it has **no agent-finalize event**, and `command:stop` is
|
||||
explicitly documented as cancellation/command-lifecycle, **not** a finalization gate. **Resolution:** do NOT
|
||||
use `before_agent_finalize`. Use `message:sent` (with debounce) as the Stop analog. If a true finalization
|
||||
gate is ever required, it must be built as an OpenClaw **plugin** (`api.on`), a different subsystem — out of
|
||||
scope for this hook package. (Surfaced in §9.)
|
||||
|
||||
- **Handler model (VERIFIED — in-process TypeScript, NOT stdin/exit-0):** a hook is a directory
|
||||
`hooks/<name>/{HOOK.md, handler.ts}`. The gateway dynamically `import()`s `handler.ts`, grabs the default
|
||||
export, and registers it as an `InternalHookHandler`. Signature:
|
||||
`(event: InternalHookEvent) => Promise<void> | void`, where `InternalHookEvent =
|
||||
{ type, action, sessionKey, context, timestamp, messages }`. Handlers run **inside the gateway Node process**
|
||||
and share its event loop.
|
||||
|
||||
**CORRECTION 2 — shim-core `runHook` is NOT reusable as-is for OpenClaw.** `runHook` is a stdin-in /
|
||||
exit-0-out subprocess model; OpenClaw handlers are in-process functions. **Resolution:** OpenClaw needs a
|
||||
thin **in-process wrapper** — a `handler.ts` whose default export receives the `InternalHookEvent`, maps
|
||||
`event.context` → the payload shape the shared handler bodies expect, then invokes the shared logic. The
|
||||
shared *logic* (recall+inject via mutating `bootstrapFiles`; save-temp-frame; summarize+save;
|
||||
`compact_memory`) still reuses shim-core's `CliBridge`/`encodeFrame`/`summarizeTurn`/`classifyImportance` —
|
||||
only the host↔handler glue changes from stdin/exit to in-process call + `execFile`. So `hooks-core` exports
|
||||
an **openclaw-specific in-process handler factory** (distinct from the `runHook`-based entrypoints the JSON
|
||||
tools use). The four shared handler *bodies* (§3.3) are authored to take an already-extracted payload, so
|
||||
both the `runHook` path and the in-process path can drive them.
|
||||
|
||||
- **External CLI invocation (VERIFIED — fully supported, idiomatic):** a handler imports `node:child_process`
|
||||
and shells out — `execFile('hive-mind-cli', [...])` (promisified + awaited) is the sanctioned pattern (the
|
||||
`gateway:pre-restart` example in `docs/automation/hooks.md` uses exactly `execFileAsync`). So the
|
||||
external-CLI-invocation assumption **HOLDS** — a Waggle handler can `execFile('hive-mind-cli', …)` (or reuse
|
||||
`CliBridge`, which already spawns it). For reliability under the event loop: use the async form and `await`
|
||||
it (the host already awaits + try/catches each handler); for non-blocking emit, use
|
||||
`fireAndForgetBoundedHook` (bounded 16 concurrent, 2s timeout) or just don't await. **No sandbox/allow-list
|
||||
restricts `child_process` from a hook** — managed/workspace hooks are "trusted local code" (the loader logs a
|
||||
trust warning).
|
||||
- **Fail-open mechanism (VERIFIED — try/catch, NOT exit-0):** `triggerInternalHook` wraps each handler in
|
||||
try/catch (logs `Hook error [type:action]`, runs the next handler, the agent flow is unaffected). There is
|
||||
**no exit code** — handlers are JS functions, so "exit 0" does not apply. **Resolution:** the openclaw
|
||||
fail-open contract is "the default-exported async handler must not throw" — wrap the handler body in
|
||||
try/catch and return on error (as every bundled handler does), and do NOT block synchronously (return a
|
||||
promise; let the host await it).
|
||||
- **Degraded/missing events (document in README per §6.1):** Stop is 0..N/turn and non-replyable (debounce
|
||||
required); no single per-turn "agent finished one reply" internal event.
|
||||
- **`handler.ts` loadability caveat (VERIFIED):** `handler.ts` is loaded via dynamic `import()` of the file
|
||||
URL — a user-dropped raw `.ts` in `~/.openclaw/hooks/` relies on the gateway having a TS loader (tsx/bundled).
|
||||
**Resolution:** ship a **compiled `.js`** handler (with matching filename) rather than a raw `.ts`, OR verify
|
||||
the target install's TS loader. Default: ship compiled `.js` from our `dist/` and reference it (this is
|
||||
consistent with how every other package ships compiled hook scripts). (Surfaced in §9 OQ-5.)
|
||||
- **Provenance / dedup story (design-mandated):** OpenClaw can drive claude-code/codex as **backends** — if
|
||||
those backends also have hive-mind hooks installed, the gateway-layer capture double-counts the same
|
||||
conversation. The package MUST stamp a provenance marker on frames it saves (e.g. `source` metadata
|
||||
`openclaw-gateway` + the channel/session id) and the design notes it "can drive CC/codex as backends," so a
|
||||
dedup heuristic (skip frames whose content hashes match a backend-captured frame within a short window) is
|
||||
needed before shipping. (Open question on the exact dedup mechanism — §9 OQ-6.)
|
||||
- **Reuses from core:** the four shared handler *bodies* (§3.3) and shim-core (`CliBridge`, `encodeFrame`,
|
||||
`summarizeTurn`, `classifyImportance`), the reversible-install primitives, the openclaw in-process handler
|
||||
factory.
|
||||
- **Implements locally:** `paths.ts` (`~/.openclaw/` + hook dir), bespoke JSON5-minimal-touch codec, the
|
||||
`HOOK.md` + compiled `handler.ts` template, the in-process event→payload mapping (incl. `bootstrapFiles`
|
||||
mutation for SessionStart and `compact:before` action matching), debounce for `message:sent`, provenance
|
||||
stamping, bin, `HOOKS_COHORT` edit.
|
||||
|
||||
---
|
||||
|
||||
## 6. Cross-Cutting Concerns
|
||||
|
||||
### 6.1 Degraded / missing-event documentation policy
|
||||
|
||||
Every package's README MUST document its degraded or missing events explicitly and MUST NOT imply hook parity
|
||||
with claude-code. Required disclosures:
|
||||
|
||||
- **cursor:** `beforeSubmitPrompt` save-only (no inject); `preCompact` observational only (best-effort, not
|
||||
guaranteed-before-truncation); Stop turn read via `transcript_path` (null when transcripts disabled).
|
||||
- **hermes:** **no PreCompact event at all** (confirmed absent — only 3 hooks ship); SessionStart is split
|
||||
across two events; block/inject is via stdout-JSON not exit codes.
|
||||
- **openclaw:** Stop is `message:sent` 0..N/turn (debounced, non-replyable); no agent-finalize internal event;
|
||||
`before_agent_finalize` is a different (plugin) subsystem.
|
||||
- **codex / codex-desktop:** full parity (no degraded events) — but note the one-time `/hooks` trust step is
|
||||
required for hooks to execute (not a degraded event, an install-UX step).
|
||||
|
||||
A short "Capture fidelity" table in each README (events supported / degraded / absent) is the canonical format.
|
||||
|
||||
### 6.2 Non-silent install UX
|
||||
|
||||
Installers MUST **print** any non-silent manual step (do not assume silent success like CC):
|
||||
|
||||
- **codex / codex-desktop:** "Run `/hooks` in Codex once to trust the hive-mind hooks." `verify` surfaces
|
||||
`allow_managed_hooks_only` lockdown.
|
||||
- **cursor:** "Restart Cursor for hive-mind hooks to take effect."
|
||||
- **hermes:** "Headless/gateway runs require `HERMES_ACCEPT_HOOKS=1` (or `hooks_auto_accept: true`) or the
|
||||
hooks will not register." The launcher should set this env under headless launch.
|
||||
- **openclaw:** "Run `openclaw hooks enable hive-mind` (or set `hooks.internal.enabled=true`) to activate."
|
||||
|
||||
### 6.3 `tool-detection.ts` fixes
|
||||
|
||||
In `packages/agent/src/tool-detection.ts`, `HOOK_POINTER_BY_TOOL` (lines 189-197):
|
||||
|
||||
- **In scope — fix codex-desktop:** change `'codex-desktop': '.config/Codex/hive-mind-install.json'` →
|
||||
`'.codex/hive-mind-install.json'` (codex-desktop shares `~/.codex/`). Ship with the codex-desktop package.
|
||||
- **Out of scope — claude-desktop TODO:** `'claude-desktop': '.config/Claude/hive-mind-install.json'` is also
|
||||
wrong (not a real Claude Desktop config dir on any platform; the real config is mac
|
||||
`~/Library/Application Support/Claude/` / Win `%APPDATA%\Claude\`). **Leave a `// TODO(claude-desktop):`
|
||||
comment** pointing at the deferred MCP-bridge work; do NOT fix it now (claude-desktop is excluded from this
|
||||
spec's scope). Do not modify the speculative codex-desktop binary candidate paths beyond the pointer fix
|
||||
unless trivially co-located.
|
||||
|
||||
### 6.4 OSS / subtree-split
|
||||
|
||||
All five packages are part of the hive-mind OSS split (CLAUDE.md §7.5) — the public mirror
|
||||
(`marolinik/hive-mind`) is generated from this monorepo via `git subtree split`, so whatever lands is
|
||||
byte-identical in the mirror with no cross-repo drift to police. Implications:
|
||||
|
||||
- **No proprietary/KVARK-gated logic** in any of these packages — they only shell to `hive-mind-cli` (the
|
||||
subtree-split filter already excludes `vault.ts`/`evolution-runs.ts`/`execution-traces.ts`/
|
||||
`improvement-signals.ts`/`compliance/**`; these packages have no such concerns — keep it that way).
|
||||
- **`@waggle/hive-mind-shim-core` and the new `@waggle/hive-mind-hooks-core` are the shared, reused-as-is
|
||||
dependencies** and also live in the OSS split — reuse them (do not copy) to keep the mirror DRY.
|
||||
- Author everything in `packages/hive-mind-hooks-*` + `packages/hive-mind-hooks-core`; the mirror follows
|
||||
automatically. Do NOT resurrect the deprecated dual-repo sync workflows (CLAUDE.md §7.5).
|
||||
- **Keep `hooks-core` in the OSS allowlist:** ensure `scripts/oss-subtree-split.sh` includes the new package
|
||||
in the export set (open item — §9 OQ-7).
|
||||
|
||||
---
|
||||
|
||||
## 7. Testing Strategy
|
||||
|
||||
### 7.1 Per-package parity with the CC reference suite
|
||||
|
||||
Each of the five packages mirrors the CC reference test layout (the CC package has install/uninstall/verify/
|
||||
paths/settings-merger tests + per-hook handler tests). Required suites per package:
|
||||
|
||||
- **paths tests:** correct config path, pointer path, hooks-dir resolution, Windows-safe backup path
|
||||
(`:`/`.` → `-`), Windows `--cli-path` quoting.
|
||||
- **register/merge tests:** additive merge preserves existing user entries verbatim; marker-tagged dedup /
|
||||
replace-in-place on re-install; immutability (input config never mutated — mirrors CC settings-merger
|
||||
contract).
|
||||
- **install tests:** config pre-existed → byte-identical backup + merged config + pointer with
|
||||
`created_by_us=false`; config absent → skeleton created + pointer `created_by_us=true`, no backup.
|
||||
- **uninstall tests:** `created_by_us=false` → byte-identical restore (round-trip verified), backup+pointer
|
||||
removed; `created_by_us=true` → config file deleted (no orphan), pointer removed. (Per-tool: hermes/openclaw
|
||||
prove literal-backup restore since codec round-trip is lossy.)
|
||||
- **verify tests:** entries present + point at live hook scripts + `hive-mind-cli --help` probe (CC verify.ts
|
||||
shape); plus per-tool surfacing (codex `allow_managed_hooks_only`; hermes consent/registration state).
|
||||
- **per-hook handler tests:** each lifecycle handler given a representative tool payload → asserts the right
|
||||
`CliBridge` call (recallMemory / saveMemory with correct importance+scope / cleanupFrames) with an injected
|
||||
mock bridge (the CC `runHook` test hooks: `readStdin`/`writeStdout`/`exit`/`bridge` overrides). OpenClaw
|
||||
handler tests drive the in-process handler with a synthetic `InternalHookEvent` (incl. `bootstrapFiles`
|
||||
mutation assertion + `compact:before` action match + `message:sent` debounce).
|
||||
|
||||
### 7.2 `hooks-core` unit tests
|
||||
|
||||
`hooks-core` carries its own unit suite for the shared primitives independent of any tool: `backupByteIdentical`
|
||||
(pre-existed vs absent), `restoreFromBackup` (both branches + round-trip-failure refusal), pointer
|
||||
read/write/malformed, `normalizeCliPath` (double-quote rejection), `jsonRegister`/`jsonUnregister`/`hasHiveEntries`
|
||||
(additive + dedup + immutability), each `make*Handler` factory against a mock bridge + mock adapter, and the
|
||||
openclaw in-process handler factory.
|
||||
|
||||
### 7.3 The two invariants every package MUST prove
|
||||
|
||||
1. **Hooks fail open (always exit 0 / never throw to host).** For the four stdin-JSON tools: an injected
|
||||
bridge error / malformed payload still results in `exit(0)` (CC `runHook` contract). For openclaw: an
|
||||
injected handler-body error is swallowed by the handler's own try/catch and the returned promise resolves
|
||||
(never rejects) — proving the host's per-handler try/catch is not relied on as the only safety net.
|
||||
2. **Byte-identical reversibility.** Uninstall restores the pre-install config to SHA-256-identical state when
|
||||
the config pre-existed (round-trip-verified, mirroring CC uninstall.ts), OR removes exactly the file(s) we
|
||||
created when it did not (no orphans, no leftover backup). Test by snapshotting the config bytes before
|
||||
install and asserting equality (or absence) after uninstall, for both the pre-existed and absent cases.
|
||||
|
||||
---
|
||||
|
||||
## 8. Build Order + Tier B Handling
|
||||
|
||||
**Locked build order:**
|
||||
|
||||
```
|
||||
hooks-core
|
||||
→ codex (Tier A; becomes the 2nd reference shape: CC-clone + create-if-missing)
|
||||
→ codex-desktop (Tier A; thin re-export over codex + the tool-detection.ts pointer fix)
|
||||
→ cursor (Tier A; JSON + field renames + degraded events)
|
||||
→ [source-verify spike: hermes + openclaw] ← DONE in this workflow (D2); results in §5.4/§5.5
|
||||
→ hermes (Tier B; YAML codec, no PreCompact, HOOKS_COHORT add)
|
||||
→ openclaw (Tier B; JSON5 + in-process TS handlers, in-process factory, debounce, provenance)
|
||||
```
|
||||
|
||||
**Tier B handling (D2):** the source-verify spike is complete (this workflow). Both repos resolved
|
||||
(`NousResearch/hermes-agent` MIT; `openclaw/openclaw`); `confidenceAfterVerify: high` for both. Hermes
|
||||
**confirmed** the design (build as designed, 3 hooks, YAML codec). OpenClaw **required two corrections**
|
||||
(in-process handler model instead of `runHook`; `before_agent_finalize` removed in favor of debounced
|
||||
`message:sent`) — both folded into §5.5; neither changes scope, only the host↔handler glue and one event name.
|
||||
Therefore no second spike is needed before implementation; the remaining Tier B unknowns are the small
|
||||
open questions in §9 (dedup mechanism, compiled-handler loadability), resolvable during implementation.
|
||||
|
||||
`hooks-core` is built and unit-tested first (it has no tool dependency). Each tool package is built →
|
||||
tested → and its `HOOKS_COHORT` entry added (so the launcher only offers a hook action once the bin is real),
|
||||
then the next package. Re-pin the hermes/openclaw repo SHAs before relying on the cited line numbers.
|
||||
|
||||
---
|
||||
|
||||
## 9. Risks + Open Questions
|
||||
|
||||
**Design impact from verification:** the Tier B verification (D2) produced **one design-impacting change** —
|
||||
OpenClaw's handler model is in-process TypeScript (not `runHook` stdin/exit-0) and `before_agent_finalize` was
|
||||
a wrong-system event. Both are resolved within the approved scope and architecture (the in-process wrapper +
|
||||
debounced `message:sent` substitution are folded into §5.5) — the *scope* (5 packages + 1 core, openclaw =
|
||||
bespoke JSON5 + in-process TS, Stop debounced) was already correct in the approved design, so this is a
|
||||
mechanism clarification, not a scope or architecture change. It does **not** require human re-approval, but it
|
||||
IS recorded here for visibility.
|
||||
|
||||
**Risks:**
|
||||
|
||||
1. **Tier B repos move fast** — hermes verified against `main` pushed 2026-05-31; re-pin both to commit SHAs
|
||||
before implementation relies on line numbers.
|
||||
2. **Create-if-missing is a behavioral fork from the frozen CC reference** (CC hard-throws if config absent).
|
||||
Easy to get the uninstall side subtly wrong (orphaned files). The §3.1 primitives + §7.2 invariant-2 tests
|
||||
are the guard.
|
||||
3. **Config round-trip fidelity** — hermes YAML and openclaw JSON5 do not survive naive parse→stringify.
|
||||
Mitigated by literal byte-identical backups for uninstall (not diff-merge fidelity) and minimal-touch edits.
|
||||
4. **Hermes headless consent** — without `HERMES_ACCEPT_HOOKS=1` the hooks silently never register under the
|
||||
launcher. The launcher MUST set it; otherwise capture is silently zero.
|
||||
5. **OpenClaw double-counting** — gateway-layer capture can duplicate backend (CC/codex) capture; provenance
|
||||
stamping + a dedup heuristic are required before shipping.
|
||||
|
||||
**Open questions (need a human decision before / during writing-plans):**
|
||||
|
||||
- **OQ-1 — `HOOKS_COHORT` membership:** the design explicitly names hermes for `HOOKS_COHORT`. Confirm that
|
||||
codex, codex-desktop, and cursor are ALSO added to `HOOKS_COHORT` as each ships a real bin (the cohort gate
|
||||
applies to every newly-real package, not just hermes). Default assumption: yes, add each as it ships.
|
||||
- **OQ-2 — `ShimSource` union extension:** confirm `'codex'` and `'hermes'` are valid `ShimSource` values in
|
||||
shim-core (`hook-event-types.ts`); if absent, the spec assumes a small additive extension there. Confirm
|
||||
that additive shim-core edit is acceptable (it ships in the OSS mirror).
|
||||
- **OQ-3 — YAML/JSON5 dep placement:** keep the `yaml` (hermes) and `json5` (openclaw) parser deps in the
|
||||
consumer packages (default) vs in `hooks-core`. Default keeps `hooks-core` codec-agnostic and avoids forcing
|
||||
a YAML dep on the JSON tools.
|
||||
- **OQ-4 — hermes PreCompact approximation:** PreCompact is confirmed absent. Do we (a) ship hermes with no
|
||||
compaction maintenance, or (b) opportunistically call `compact_memory` from the hermes Stop handler? Default:
|
||||
document the gap; do not invent — leave (b) as an optional follow-up.
|
||||
- **OQ-5 — openclaw handler shipping format:** ship a compiled `.js` `handler.ts`-equivalent (default, robust)
|
||||
vs a raw `.ts` relying on the gateway's TS loader. Default: compiled `.js`. Confirm the `HOOK.md` `default`
|
||||
export resolution works against a compiled file on a real openclaw install (the one remaining
|
||||
needs-a-live-install validation).
|
||||
- **OQ-6 — openclaw dedup mechanism:** exact provenance/dedup story (content-hash window? backend-detection?)
|
||||
needs a decision before openclaw ships. Not blocking codex/cursor/hermes.
|
||||
- **OQ-7 — OSS subtree-split allowlist:** confirm `scripts/oss-subtree-split.sh` includes the new
|
||||
`hive-mind-hooks-core` (and the four new hook packages, if the script enumerates rather than globs) in the
|
||||
export set.
|
||||
|
||||
No TBD/placeholder remains unresolved outside this open-questions list; the spec is internally consistent and
|
||||
scoped to a single implementation plan (hooks-core + 5 packages, build order in §8).
|
||||
@@ -0,0 +1,52 @@
|
||||
# Temporal Substrate Fix — Design Spec
|
||||
|
||||
**Date:** 2026-06-09 · **Status:** approved (brainstorming), pre-implementation
|
||||
**Origin:** Memori head-to-head (`benchmarks/results/memori-head-to-head-RESULT-2026-06-09.md`) — our substrate ties Memori overall (80.84 substrate-vs-substrate / 82.21 our-prompt vs 81.98) but **loses temporal −9pp** (73.5/73.8 vs 82.7). Code map: temporal info is captured in the DB but stripped before the LLM sees it.
|
||||
**Scope:** "Surface time" (additive only). NOT time-aware re-ranking, NOT timestamped-triple re-representation.
|
||||
**Goal:** lift LoCoMo temporal toward peer-best (~82–87%) without regressing single (89) / multi (77) / open (66); change the **real substrate** (`hive-mind-core` → waggle-os), measurable on the existing harness; keep OSS subtree-split clean.
|
||||
|
||||
## Root cause (from `Explore` map, file:line)
|
||||
- `memory_frames.created_at` IS populated from source timestamps (schema.ts:59; LoCoMo ingest 02b:93 passes the session date). Importance/semantic hits SELECT `created_at` (fetchImportantFrames). **The data exists.**
|
||||
- **The loss is in rendering + distillation + prompt:**
|
||||
- GAP 2 (load-bearing): benchmark `buildContext` (40:135-153) renders fact/snippet text only — **strips `created_at`**. Production `renderRecallResult` shows date-only for snippets, nothing for facts.
|
||||
- GAP 3: our answer prompt has **no temporal-arithmetic instruction**. (Memori's `ANSWER_PROMPT` does — but our `theirs` arm still scored 73.5 because the context we fed it had no timestamps for that instruction to use. **Proof the fix is in the context, not the prompt.**)
|
||||
- GAP 1: distilled facts written with `createdAt=null` (28:137, 31:124) AND `fetchDistilledFacts` doesn't even SELECT created_at. Distilled facts are cross-session syntheses → no single meaningful date → deferred to Phase 2.
|
||||
|
||||
## Approach (chosen): shared renderer in `hive-mind-core`, additive
|
||||
One source of truth the whole stack imports; no benchmark/production drift; OSS-clean.
|
||||
|
||||
## Phase 1 — Surface snippet time + prompt guidance (ZERO LLM re-cost)
|
||||
The high-leverage, cheap lever. No re-ingest, no re-distill — only rendering + prompt, then re-run answer+judge.
|
||||
|
||||
**Production (`packages/hive-mind-core/src/mind/`):**
|
||||
1. Add/extend a context renderer (near `recall-context.ts` renderRecallResult) so each retrieved snippet is prefixed `[YYYY-MM-DD]` (from `created_at`), and the memory block opens with one anchor line: `Reference date (most recent memory): YYYY-MM-DD`. Compact format (≤~8 tokens/item) to bound the token bump.
|
||||
2. Export a reusable `TEMPORAL_GUIDANCE` prompt fragment: *"Memories are timestamped [YYYY-MM-DD]. Resolve relative time ('last year', 'two months ago') to absolute dates using the memory's timestamp as the anchor. On conflicting facts, prefer the most recent."* Attach it to the injected-memory block in the agent's memory-recall path — **scoped to memory recall, NOT a global system-prompt change.**
|
||||
|
||||
**Benchmark (`hive-mind-test/scripts/locomo/`):**
|
||||
3. `40-cell-retrieval-gpt41mini.mjs` `buildContext`: render each importance+semantic snippet with its `[YYYY-MM-DD]` (hits already carry `created_at`; verify) + the reference-date anchor line. Mirror the production renderer's format.
|
||||
4. `ours` prompt arm: prepend `TEMPORAL_GUIDANCE`. `theirs` arm: keep Memori's verbatim ANSWER_PROMPT unchanged (it already has the instruction — now it finally has timestamps to act on; this is the cleanest before/after).
|
||||
5. (Readiness only) add `created_at` to the `fetchDistilledFacts` SELECT so Phase 2 can use it.
|
||||
|
||||
**Measure:** re-run both arms on the identical ruler (gpt-4.1-mini answerer+judge, Memori judge prompt, 1540 Qs) via the existing orchestrator. Compare temporal + the other three + tokens/query.
|
||||
|
||||
## Phase 2 — Dated distilled facts (OPTIONAL, only if Phase 1 underdelivers; has LLM re-cost)
|
||||
Re-distillation prompts the distiller to attach the relevant absolute date(s) into time-bearing fact text, and stamps each distilled frame with the latest contributing session date. Costs a re-distillation LLM pass + re-run. Decide after Phase 1 numbers.
|
||||
|
||||
## Success / regression gate
|
||||
- **PASS:** temporal ↑ materially (target ≥ ~82, Memori parity), AND single/multi/open each within **±1.5pp** of today (89.1 / 77.0 / 65.6), AND tokens/query bump disclosed (expected small, compact format).
|
||||
- Overall projected: temporal 73.8→82.7 ≈ **+1.9pp → ~84% overall** (our-prompt arm), clearing Memori beyond single-run noise.
|
||||
|
||||
## Repos & OSS hygiene
|
||||
- Canonical change in `waggle-os/packages/hive-mind-core` (production). Benchmark imports the built `D:/Projects/hive-mind` (OSS checkout) — rebuild its dist after mirroring, OR the renderer change is small enough to mirror directly; verify the benchmark picks up the new renderer before the paid run.
|
||||
- All new logic in `hive-mind-core` (not vault/evolution/compliance) → subtree-split filter unaffected.
|
||||
|
||||
## Risks
|
||||
- Token bump from per-item dates (worsens the efficiency axis we already lose) → keep format compact, measure avg/p50.
|
||||
- Benchmark uses a built `hive-mind` dist, not waggle-os source directly → must ensure the renderer change reaches the benchmark (rebuild/mirror) or the measurement won't reflect the fix.
|
||||
- `recall.hits` must expose `created_at` to `buildContext` — verify in implementation; if absent, add to the recall projection.
|
||||
|
||||
## Out of scope (YAGNI)
|
||||
Time-aware re-ranking (GAP 5), since/until wiring into default recall (GAP 6), full triple re-representation. Revisit only if Phase 1+2 miss the gate.
|
||||
|
||||
## Follow-on sub-project B (separate spec)
|
||||
SOTA campaign: run Zep/LangMem/Mem0 on our ruler + judge hardening (trio-strict) + public write-up. Depends on this fix landing a number.
|
||||
@@ -0,0 +1,136 @@
|
||||
# Waggle AI OS Positioning Audit Design
|
||||
|
||||
Date: 2026-06-28
|
||||
Status: Approved concept, pending implementation plan
|
||||
|
||||
## Purpose
|
||||
|
||||
Create a repeatable E2E audit that grades Waggle honestly as an AI OS. The test should show where Waggle is compelling, where it feels addictive or habit-forming, how it compares with realistic competitors, and which product improvements would most increase adoption.
|
||||
|
||||
The audit is a report generator, not a gatekeeper. It should fail only when the audit cannot run. A poor Waggle score is valid output and should produce clear improvement areas instead of failing CI.
|
||||
|
||||
## Product Question
|
||||
|
||||
Can Waggle be positioned as an AI OS rather than another AI chat app?
|
||||
|
||||
The audit answers this through five lenses:
|
||||
|
||||
1. Does a new user reach value quickly?
|
||||
2. Does memory and continuity create a reason to return tomorrow?
|
||||
3. Does Waggle cover enough of the user's real workflow to become the primary AI surface?
|
||||
4. Does Waggle compare favorably with the user's current default tool?
|
||||
5. Which missing or weak surfaces prevent stronger AI OS positioning?
|
||||
|
||||
## Five Personas
|
||||
|
||||
The audit will use five personas that span non-technical to power-user workflows. Each persona has a real current competitor and a specific "one tool" criterion.
|
||||
|
||||
| Persona | Current default | Primary job-to-be-done | One-tool criterion |
|
||||
|---|---|---|---|
|
||||
| Sofia, small business operator | ChatGPT, Gmail, Canva | Draft customer replies, campaign ideas, supplier follow-ups | Daily communications and decisions happen in Waggle |
|
||||
| Mara, marketing/writer | ChatGPT, Claude, Notion AI | Turn notes and research into branded copy | Voice, drafts, and campaign memory compound in Waggle |
|
||||
| Imran, consultant/strategist | Claude, ChatGPT, Gamma | Convert calls and notes into frameworks, briefs, and follow-ups | Client context and recurring strategy work live in Waggle |
|
||||
| Daniel, finance/ops analyst | Excel Copilot, ChatGPT, Looker | Explain variance, summarize metrics, prepare board commentary | Data commentary and recurring monthly memory live in Waggle |
|
||||
| Priya, AI power user | Claude Code/Codex, Hermes/OpenClaw, custom scripts | Coordinate AI workflows, skills, connectors, and memory | Waggle is the front door for non-coding agent work |
|
||||
|
||||
## Competitor Set
|
||||
|
||||
The audit will score against realistic alternatives rather than a generic "AI tool" baseline:
|
||||
|
||||
- ChatGPT: strong general chat, weak workspace/memory control.
|
||||
- Claude: strong writing and reasoning, weak OS/workflow surface.
|
||||
- Claude Code/Codex-style developer tools: strong coding agents, weak non-coding workspace OS.
|
||||
- Notion/Workspace AI: strong document/workspace adjacency, weaker agent runtime and local memory.
|
||||
- Hermes/OpenClaw-style agent frameworks: strong technical extensibility, weak non-technical UX and guided positioning.
|
||||
|
||||
The test will not call competitor services. It will use a documented benchmark matrix based on known product categories already reflected in the repo's competitive benchmark tests.
|
||||
|
||||
## Scoring Model
|
||||
|
||||
Each persona receives a 100-point score:
|
||||
|
||||
| Dimension | Points | Evidence |
|
||||
|---|---:|---|
|
||||
| Onboarding clarity | 15 | App loads, first-run path is understandable, no broken shell |
|
||||
| Time to first value | 15 | Core route/API response speed and reachable primary action |
|
||||
| Memory and continuity | 20 | Save/recall flow, workspace isolation, persistence signal |
|
||||
| Workflow coverage | 15 | Relevant personas, skills, connectors, workspace surfaces |
|
||||
| Competitive advantage | 15 | Capability gap versus persona's current default |
|
||||
| Addiction/return signal | 20 | External trigger, internal trigger fit, investment, stored value |
|
||||
|
||||
Overall grades:
|
||||
|
||||
- 90-100: Strong AI OS position, ready for broader acquisition testing.
|
||||
- 75-89: Strong niche fit, needs sharper first-session magic or workflow coverage.
|
||||
- 60-74: Promising, but users still have obvious reasons to return to competitors.
|
||||
- 40-59: Positioning is plausible but product experience is not yet persuasive.
|
||||
- 0-39: Users likely experience Waggle as another AI chat/tool wrapper.
|
||||
|
||||
## Audit Output
|
||||
|
||||
The Playwright spec will produce a structured Markdown report and a JSON summary under a test artifact directory. The report should include:
|
||||
|
||||
- Overall grade and short interpretation.
|
||||
- AI OS positioning verdict.
|
||||
- Addiction level: weak, emerging, strong, or very strong.
|
||||
- Persona-by-persona scores.
|
||||
- Competitor comparison by persona.
|
||||
- Top improvement areas, ranked by score impact.
|
||||
- Evidence notes for failed or weak dimensions.
|
||||
|
||||
The report is the deliverable. The test should log the report location and pass when report generation succeeds.
|
||||
|
||||
## E2E Shape
|
||||
|
||||
Target file:
|
||||
|
||||
`tests/e2e/ai-os-positioning-audit.spec.ts`
|
||||
|
||||
The spec will run against the existing Playwright server configuration and `WAGGLE_E2E_BASE_URL`.
|
||||
|
||||
The test will:
|
||||
|
||||
1. Load the app shell with onboarding skipped for stable route checks.
|
||||
2. Probe required product routes and APIs.
|
||||
3. Create isolated workspaces where allowed.
|
||||
4. Save and recall persona-specific memory anchors.
|
||||
5. Inspect personas, skills, connectors, marketplace, hooks, fleet, and health endpoints.
|
||||
6. Score each persona using deterministic evidence.
|
||||
7. Generate Markdown and JSON artifacts.
|
||||
|
||||
The spec will avoid real LLM dependency by default. If a live LLM is configured, future versions can add optional answer-quality grading, but the first implementation should remain deterministic.
|
||||
|
||||
## Improvement Backlog Rules
|
||||
|
||||
Improvement areas should be generated from score gaps, not hand-written optimism.
|
||||
|
||||
Examples:
|
||||
|
||||
- Low onboarding clarity: improve first-session explanation, reduce setup choices, make "AI OS" visible in first viewport.
|
||||
- Low memory continuity: make memory save/recall visible earlier, show citations or "why I know this."
|
||||
- Low workflow coverage: add persona-specific connectors, templates, or import paths.
|
||||
- Low addiction signal: add durable external triggers such as daily brief, return reminders, import nudges, or OS-level hotkey.
|
||||
- Low competitive advantage: make the differentiator explicit against ChatGPT/Claude for that persona.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- Do not run paid competitor APIs.
|
||||
- Do not require a real LLM key.
|
||||
- Do not fail CI only because Waggle's product score is low.
|
||||
- Do not replace moderated human testing; this audit complements the existing user-test protocol.
|
||||
- Do not make product changes while adding the audit.
|
||||
|
||||
## Verification
|
||||
|
||||
Implementation verification should include:
|
||||
|
||||
- TypeScript compiles for the new Playwright spec.
|
||||
- The audit spec runs and produces Markdown plus JSON artifacts.
|
||||
- Existing E2E harness still starts the local Waggle server.
|
||||
- The report contains all five personas, an overall grade, and at least one improvement area when any dimension is below full score.
|
||||
|
||||
## Open Assumptions
|
||||
|
||||
- Report artifacts can live in the existing Playwright/test artifact area rather than committed docs.
|
||||
- The audit will start in report mode and later can grow strict thresholds if the team wants launch gates.
|
||||
- Existing competitor benchmark claims in the repo are sufficient for category-level comparison without live competitor calls.
|
||||
101
docs/superpowers/specs/2026-06-30-goal-ancestry-design.md
Normal file
101
docs/superpowers/specs/2026-06-30-goal-ancestry-design.md
Normal file
@@ -0,0 +1,101 @@
|
||||
# Goal-Ancestry Context Chain — Design
|
||||
|
||||
**Date:** 2026-06-30 · **Author:** Claude Opus 4.8 (1M) · **Owner:** Marko (founder)
|
||||
**Arc:** AI-OS external-agent recon · STEAL NOW item **#6** from
|
||||
`docs/analysis/external-agent-launching-and-memory-comparison-2026-06-29.md`
|
||||
**Effort:** S · **Branch:** `feat/goal-ancestry-context`
|
||||
|
||||
---
|
||||
|
||||
## 1. Goal & Non-Goals
|
||||
|
||||
**Goal.** Give every agent run a durable sense of **why it exists** — a short "ancestry"
|
||||
breadcrumb (the purpose above the current turn) injected into the system prompt, complementing
|
||||
(not duplicating) hive-mind recall and the live AwarenessLayer task state. Paperclip supplies a
|
||||
mission→project→goal→task chain each run; this is Waggle's honest equivalent.
|
||||
|
||||
**Non-goals (YAGNI):**
|
||||
- No new persistence / data model. No workspace "charter/mission" field (reserved for later).
|
||||
- No UI. No feature flag (purely additive — renders nothing when there's no "why").
|
||||
- The `Orchestrator` does **not** reach into the DB for ancestry — it renders what it's handed.
|
||||
- The live current task stays in the existing self-awareness "Active Tasks" section (no dup).
|
||||
|
||||
## 2. Honest Mapping (founder-approved 2026-06-30)
|
||||
|
||||
Waggle has no mission→project→goal→task hierarchy. The durable "why" sources are:
|
||||
|
||||
| Level | Waggle source | Included? |
|
||||
|---|---|---|
|
||||
| `mission` | *(no workspace-charter field exists yet)* | reserved — omitted today |
|
||||
| `project` | active **workspace** name (+ template) — the durable container | **yes** |
|
||||
| `goal` | **`AgentDef.goal`** / persona goal — the agent's declared purpose | **yes** |
|
||||
| `task` | already rendered by the self-awareness "Active Tasks" section | omitted (no dup) |
|
||||
|
||||
All four levels exist in the type for future-proofing; only `project` + `goal` are populated now.
|
||||
|
||||
## 3. Architecture — pure renderer + caller populates
|
||||
|
||||
**Unit A — `GoalAncestry` type (`packages/shared/src/types.ts`).**
|
||||
```ts
|
||||
/** Durable "why" injected into the agent system prompt (AI-OS #6). All optional. */
|
||||
export interface GoalAncestry {
|
||||
mission?: string; // reserved — no workspace charter field yet
|
||||
project?: string; // workspace name (+ template)
|
||||
goal?: string; // agent's declared goal / persona purpose
|
||||
task?: string; // omitted today (lives in the awareness section)
|
||||
}
|
||||
```
|
||||
|
||||
**Unit B — Orchestrator renders it (`packages/agent/src/orchestrator.ts`).**
|
||||
- `OrchestratorConfig.goalAncestry?: GoalAncestry`; stored as `private goalAncestry: GoalAncestry | null`.
|
||||
- A pure helper `renderGoalAncestry(a: GoalAncestry | null): string` returns a `# Why You're Here`
|
||||
block listing only the present levels as `Label: value` lines, or `''` when nothing is present.
|
||||
- `buildSystemPrompt()` adds a **cached** section (keyed on `JSON.stringify(goalAncestry) || 'empty'`,
|
||||
same pattern as the identity section) inserted **after identity, before self-awareness** — purpose
|
||||
frames capability/awareness. The empty result is filtered out by the existing `.filter(Boolean)`.
|
||||
|
||||
Render format (only present levels):
|
||||
```
|
||||
# Why You're Here
|
||||
Project: Acme Redesign (engineering)
|
||||
Goal: Ship the launcher live-output pane and keep tests green
|
||||
```
|
||||
|
||||
**Unit C — Caller populates (`packages/server/src/local/index.ts`, the per-session orchestrator).**
|
||||
- Build a `GoalAncestry` from available context at construction: `project` ← active workspace
|
||||
name (+ template if present), `goal` ← the resolved persona/agent goal for the session.
|
||||
- `mission`/`task` left undefined. When neither `project` nor `goal` is known (e.g. bare personal
|
||||
chat), pass `undefined` → section renders nothing → byte-identical to today.
|
||||
|
||||
## 4. Data Flow
|
||||
```
|
||||
session construct (local/index.ts) ── workspace + persona/agent ──▶ GoalAncestry { project, goal }
|
||||
└▶ new Orchestrator({ ..., goalAncestry })
|
||||
└▶ buildSystemPrompt() ── cachedSection('goal_ancestry') ──▶ "# Why You're Here\nProject: …\nGoal: …"
|
||||
(empty ancestry ⇒ '' ⇒ filtered out ⇒ prompt unchanged)
|
||||
```
|
||||
|
||||
## 5. Error Handling / Edge Cases
|
||||
- All levels optional; missing → omitted line. All-empty → no section (no heading, no blank).
|
||||
- Long values: truncate each level to a sane cap (≤200 chars) so a verbose goal can't bloat every turn.
|
||||
- The renderer is total (never throws); a malformed ancestry object just yields the lines it can.
|
||||
|
||||
## 6. Testing (TDD)
|
||||
- `renderGoalAncestry`: full ancestry → heading + one line per present level, in mission→project→goal→task order;
|
||||
partial (only `goal`) → just that line; empty/`null` → `''`; over-long value → truncated.
|
||||
- `buildSystemPrompt`: with ancestry → section present and ordered after identity / before self-awareness;
|
||||
without → section absent (prompt unchanged); cache hit on unchanged ancestry (no recompute).
|
||||
- Caller wiring: a session with workspace + persona goal yields a populated `goalAncestry`; bare session yields none.
|
||||
|
||||
**Gates:** `tsc --noEmit` 0 (shared/agent/server); new units RED→GREEN; existing orchestrator suite green.
|
||||
|
||||
## 7. File Change List
|
||||
| File | Change |
|
||||
|---|---|
|
||||
| `packages/shared/src/types.ts` | add `GoalAncestry` interface |
|
||||
| `packages/agent/src/orchestrator.ts` | `OrchestratorConfig.goalAncestry`, field, `renderGoalAncestry`, cached section in `buildSystemPrompt` |
|
||||
| `packages/server/src/local/index.ts` | populate `goalAncestry` for the per-session orchestrator |
|
||||
| tests (2) | `orchestrator` goal-ancestry unit tests + (light) caller-wiring assertion |
|
||||
|
||||
## 8. Open Questions
|
||||
- **None blocking.** Mapping founder-approved. `mission` + a workspace-charter field is a separate future item.
|
||||
248
docs/superpowers/specs/2026-06-30-launcher-live-output-design.md
Normal file
248
docs/superpowers/specs/2026-06-30-launcher-live-output-design.md
Normal file
@@ -0,0 +1,248 @@
|
||||
# Launcher Live-Output Pane (piped, observed mode) — Design
|
||||
|
||||
**Date:** 2026-06-30 · **Author:** Claude Opus 4.8 (1M) · **Owner:** Marko (founder)
|
||||
**Arc:** AI-OS external-agent launcher · STEAL NOW item **#4** from
|
||||
`docs/analysis/external-agent-launching-and-memory-comparison-2026-06-29.md`
|
||||
**Branch:** `feat/launcher-self-enabling-pipeline` (continues the launcher trio shipped in `4ade22f7`)
|
||||
**Capture model (founder-approved 2026-06-30):** **piped stdio**, not node-pty.
|
||||
|
||||
---
|
||||
|
||||
## 1. Goal & Non-Goals
|
||||
|
||||
**Goal.** Close the launcher's "no eyes" gap. Today `launchTool()` spawns external agents
|
||||
`detached + stdio:'ignore' + unref()`, so Waggle sees nothing until a Stop-hook frame lands.
|
||||
Give the user an opt-in way to **watch a launched agent's live stdout/stderr** in the dock.
|
||||
|
||||
**Non-goals (YAGNI — explicitly out of scope for v1):**
|
||||
- No `node-pty` / true terminal emulation, no ANSI/TUI rendering fidelity, no `xterm.js`.
|
||||
- No **input send** back to the agent (that is the node-pty path; deferred).
|
||||
- No multi-process tabbed terminal — **one pane per running tool**, opened on demand.
|
||||
- No auto-ingest of captured output into memory (display-only → no injection surface).
|
||||
- No change to the default detached launch path or to the #2 persistence guarantee.
|
||||
|
||||
---
|
||||
|
||||
## 2. The Crux: Two Launch Modes
|
||||
|
||||
The feature rides on one inversion of the existing contract:
|
||||
|
||||
| | **Detached (default, unchanged)** | **Observed (new, opt-in)** |
|
||||
|---|---|---|
|
||||
| spawn | `detached:true`, `stdio:'ignore'`, `unref()` | `stdio:['ignore','pipe','pipe']`, **no `unref()`** |
|
||||
| survives sidecar restart | **Yes** (the whole point; #2 reconciles it) | **No** — sidecar holds the pipes; tethered |
|
||||
| live output | none | streamed to the dock |
|
||||
| tracker persistence | persisted to pidfile | **in-memory only** (can't survive → must not claim to) |
|
||||
|
||||
Making observation a **mode** rather than a replacement is what lets #4 coexist with the
|
||||
`4ade22f7` persistence (#2) instead of silently breaking its "survives restart" promise.
|
||||
|
||||
---
|
||||
|
||||
## 3. Backend Design
|
||||
|
||||
### 3.1 `spawnObserved` DI seam — `packages/agent/src/tool-launcher.ts`
|
||||
|
||||
New injectable dep mirroring `spawnDetached`, kept hermetic so tests never spawn real
|
||||
processes. It returns the pid **plus an abstract output handle** so the buffer can subscribe
|
||||
without leaking `ChildProcess` into the pure surface:
|
||||
|
||||
```ts
|
||||
/** Minimal, test-injectable view of a live observed process. */
|
||||
export interface ObservedHandle {
|
||||
/** Subscribe to decoded stdout+stderr text chunks. */
|
||||
onData(cb: (chunk: string) => void): void;
|
||||
/** Fired once when the process exits. code is null on signal-kill. */
|
||||
onExit(cb: (code: number | null) => void): void;
|
||||
}
|
||||
|
||||
spawnObserved?: (
|
||||
binary: string,
|
||||
args: string[],
|
||||
options: { cwd?: string; env?: NodeJS.ProcessEnv },
|
||||
) => { pid: number | null; error?: string; handle?: ObservedHandle };
|
||||
```
|
||||
|
||||
Production default: `spawn(binary, args, { cwd, env, stdio: ['ignore','pipe','pipe'] })`
|
||||
(no `detached`, no `unref`); wires `child.stdout`/`child.stderr` `'data'` → `onData` (utf8),
|
||||
and `child` `'exit'` → `onExit`.
|
||||
|
||||
`launchTool()` gains `observe?: boolean`. When `observe === true` it calls `spawnObserved`
|
||||
(falling back to the same `LAUNCH_COHORT`/`installedPath` guards) and returns the existing
|
||||
`LaunchResult` **plus an optional `output?: ObservedHandle`**. When false/absent the path is
|
||||
byte-for-byte today's `spawnDetached`. The signal-emit / sidecar-url env injection is shared
|
||||
across both modes (already in place).
|
||||
|
||||
### 3.2 Output buffer — `packages/agent/src/tool-output-buffer.ts` (new)
|
||||
|
||||
A `ToolOutputBuffer` class owning a **bounded ring buffer per pid**, modeled on the SignalBus
|
||||
500-cap philosophy:
|
||||
|
||||
- `attach(pid, handle: ObservedHandle)` — subscribe; push chunks (split to lines, **ANSI
|
||||
stripped on ingest** via an inline regex — no new dependency) into a ring capped at
|
||||
**`MAX_LINES = 2000`** and **`MAX_BYTES = 256 KB`** (whichever first; oldest evicted).
|
||||
- `getTail(pid): { lines: string[]; exited: boolean; exitCode: number | null }` — replay.
|
||||
- `subscribe(pid, listener): () => void` — live fan-out to SSE clients; returns an unsubscribe.
|
||||
- On `onExit`: stamp `exited/exitCode`, emit a terminal event to live listeners, retain the
|
||||
tail for late readers, and schedule eviction of the whole entry after a short grace (so a
|
||||
pane opened just after exit still shows the final output).
|
||||
|
||||
In-memory only — like the tracker's non-persisted state and the SignalBus, it is lost on
|
||||
sidecar restart (consistent with observed processes being tethered).
|
||||
|
||||
### 3.3 Tracker change — `packages/agent/src/tool-process-tracker.ts`
|
||||
|
||||
- `TrackedProcess` gains optional `observed?: boolean`.
|
||||
- `register(pid, toolId, workspaceId?, opts?: { observed?: boolean })`.
|
||||
- `persist()` writes `records.filter(p => !p.observed)` — **observed pids are never persisted**,
|
||||
so a sidecar restart can never resurrect a stale "Running" badge for a tethered (now-dead)
|
||||
process or a pid-reused stranger. Observed pids still appear in `list()` while the sidecar
|
||||
lives, so the badge + pane work for the whole session.
|
||||
|
||||
### 3.4 Routes — `packages/server/src/local/routes/tools.ts`
|
||||
|
||||
- **`POST /api/tools/launch`** — `launchBodySchema` gains `observe: z.boolean().optional()`.
|
||||
When `observe`, the handler: calls `launchTool({ ..., observe: true })`; on success
|
||||
`tracker.register(pid, id, workspaceId, { observed: true })` and
|
||||
`outputBuffer.attach(pid, result.output)`. Response unchanged (202 + `{ ok, pid }`).
|
||||
- **`GET /api/tools/stream?pid=` (new, SSE)** — validates `pid` is **tracked AND observed**
|
||||
(404 otherwise); then mirrors the proven `chat.ts:580` pattern: `reply.hijack()` →
|
||||
`raw.writeHead(200, text/event-stream + loopback CORS)` → replay `getTail` as
|
||||
`event: line` frames → `subscribe` for live `line` frames → on exit emit `event: exit`
|
||||
`data: { code }` and `raw.end()`. `reply.raw.on('close', unsubscribe)` cleans up on client
|
||||
disconnect. No new buffer/transport primitive — pure composition.
|
||||
- `outputBuffer` is decorated on the Fastify instance exactly like `toolProcessTracker`
|
||||
(lazy-init, `fastify-plugin`-propagated) so `/launch` and `/stream` share one instance.
|
||||
|
||||
### 3.5 Security & resource bounds
|
||||
|
||||
- Output is **display-only**, never auto-ingested → no LLM/injection surface; React escapes
|
||||
all text; ANSI is stripped so no terminal control sequences reach the DOM.
|
||||
- `/stream` is loopback-bound like every local route; `pid` must be one **we** spawned and
|
||||
marked observed (reuses the tracker's "only our pids" guard philosophy).
|
||||
- Ring-buffer caps (2000 lines / 256 KB / pid) bound memory; entries evicted after exit grace.
|
||||
|
||||
---
|
||||
|
||||
## 4. Frontend Design
|
||||
|
||||
Constraint (founder, 2026-06-30): **keep current UX, progressive disclosure — only basics on
|
||||
the dock menu, richer entry points on ⌘K.**
|
||||
|
||||
### 4.1 `apps/web/src/lib/adapter.ts`
|
||||
|
||||
- `launchTool(payload)` gains optional `observe?: boolean` in its body.
|
||||
- New `streamToolOutput(pid, { onLine, onExit, signal })` — opens `GET /api/tools/stream?pid=`
|
||||
via the same auth'd `fetch` + `ReadableStream` reader + `event:/data:` frame parse the chat
|
||||
SSE path already uses (no `EventSource`, which can't carry the device token). Returns a
|
||||
close handle; aborts via `AbortSignal`.
|
||||
|
||||
### 4.2 `LauncherApp.tsx` — dock unchanged, badge becomes the disclosure
|
||||
|
||||
- The per-tool **cards and their basic buttons (Launch/Stop/Install/Verify/Uninstall) are
|
||||
untouched.** The default dock **Launch** stays **detached** (no behavior change).
|
||||
- **Progressive disclosure:** the existing **Running** badge becomes clickable. Clicking it
|
||||
toggles an inline collapsible `<ToolOutputPane pid=… toolId=… />` beneath that card.
|
||||
- Observed launch → pane streams live output (mono scroll area, auto-scroll-to-bottom unless
|
||||
the user scrolled up, exit-code footer).
|
||||
- Detached launch (no buffer) → pane shows a one-line hint: *"This agent was launched in the
|
||||
background (no live output). Use ⌘K → ‘Watch a coding agent live’ to start one you can
|
||||
watch."* — honest, no fake stream.
|
||||
- **Watch mode** (entered via ⌘K deep-link, below): a `?watch=1` route param puts LauncherApp
|
||||
in a mode where the per-tool **Launch** action sends `observe:true` and auto-opens that
|
||||
tool's pane. No new always-visible buttons — it reuses the existing Launch control's intent.
|
||||
|
||||
### 4.3 ⌘K — `lib/command-catalog.ts` (the rich entry point)
|
||||
|
||||
Add **one** curated catalog item next to the existing `launch-agent` entry:
|
||||
|
||||
```ts
|
||||
{ id: "watch-agent", group: "do", name: "Watch a coding agent live",
|
||||
subtitle: "Claude Code · Cursor · Codex — stream its output", icon: Eye, to: "/launcher?watch=1" }
|
||||
```
|
||||
|
||||
This honors "basics on the dock, depth one keystroke away": the watch path is discoverable in
|
||||
⌘K and deep-links into LauncherApp's watch mode; the dock itself gains no new buttons. Exact
|
||||
param plumbing (`useSearchParams` in the launcher host) is a plan detail.
|
||||
|
||||
### 4.4 New component — `apps/web/src/components/os/apps/launcher/ToolOutputPane.tsx`
|
||||
|
||||
Self-contained: takes `{ pid, toolId }`, opens `adapter.streamToolOutput` on mount, renders a
|
||||
bounded virtualized-enough mono list (cap render to last N lines to match the server cap),
|
||||
shows a spinner until first line, an exit-code chip on close, and a copy-all affordance.
|
||||
Cleans up the stream on unmount. ~one focused file (<200 LOC), one clear purpose.
|
||||
|
||||
---
|
||||
|
||||
## 5. Data Flow
|
||||
|
||||
```
|
||||
⌘K "Watch a coding agent live" ──▶ /launcher?watch=1
|
||||
└▶ LauncherApp (watch mode): Launch ──▶ adapter.launchTool({ id, …, observe:true })
|
||||
└▶ POST /api/tools/launch {observe} ─▶ launchTool({observe:true})
|
||||
└▶ spawnObserved → {pid, handle}
|
||||
├▶ tracker.register(pid, id, ws, {observed:true}) (in-memory, not persisted)
|
||||
└▶ outputBuffer.attach(pid, handle) ── ring buffer (2000 ln / 256 KB, ANSI-stripped)
|
||||
Running badge click ──▶ <ToolOutputPane pid>
|
||||
└▶ adapter.streamToolOutput(pid) ─▶ GET /api/tools/stream?pid (SSE, reply.hijack)
|
||||
└▶ replay tail → live `line` frames → `exit` frame → close
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Error Handling
|
||||
|
||||
- `spawnObserved` failure → `launchTool` returns `ok:false` with the spawn error (today's path).
|
||||
- `/stream` with an unknown/non-observed/dead pid → **404** `{ error }`; pane shows the hint, not a spinner-forever.
|
||||
- Client disconnect / unmount → `reply.raw.on('close')` unsubscribes; `AbortSignal` tears down the fetch reader.
|
||||
- Process exits → terminal `exit` event with code; pane freezes the final tail + shows the chip.
|
||||
- Buffer overflow → oldest lines evicted silently (bounded by design); pane mirrors the server cap.
|
||||
- Sidecar restart mid-watch → stream errors out; pane shows *"output ended (sidecar restarted)"*; the observed pid is gone from `list()` so the badge clears on next 5s poll.
|
||||
|
||||
---
|
||||
|
||||
## 7. Testing (TDD — mirror `4ade22f7`'s gate discipline)
|
||||
|
||||
**Backend (`packages/agent`, `packages/server`):**
|
||||
- `spawnObserved` injected fake emits synthetic data/exit → buffer fills, caps at 2000 lines /
|
||||
256 KB, finalizes with exit code, evicts after grace.
|
||||
- `launchTool({observe:true})` returns `output` handle; `observe:false`/absent unchanged (regression-lock).
|
||||
- `tracker.register(..., {observed:true})` lists the pid but `persist()` excludes it; reconcile never sees it.
|
||||
- `/api/tools/launch {observe}` registers + attaches; `/api/tools/stream` replays tail, streams
|
||||
live frames, emits `exit`, 404s unknown/non-observed pid, unsubscribes on close.
|
||||
|
||||
**Frontend (`apps/web`):**
|
||||
- Running badge toggles the pane; pane renders streamed lines (mocked `streamToolOutput`),
|
||||
shows exit chip, shows the detached-launch hint when no buffer.
|
||||
- ANSI-bearing lines render stripped; ⌘K catalog exposes "Watch a coding agent live".
|
||||
|
||||
**Gates:** `tsc --noEmit` 0 across agent/server/web · all new units RED→GREEN · existing
|
||||
launcher suites stay green (tool-launcher / tool-process-tracker / tools-routes / LauncherApp).
|
||||
|
||||
---
|
||||
|
||||
## 8. File-by-File Change List
|
||||
|
||||
| File | Change |
|
||||
|---|---|
|
||||
| `packages/agent/src/tool-launcher.ts` | `ObservedHandle` type, `spawnObserved` dep + default, `launchTool` `observe` option + `output` in result |
|
||||
| `packages/agent/src/tool-output-buffer.ts` | **new** — `ToolOutputBuffer` ring buffer (attach/getTail/subscribe/exit + ANSI strip) |
|
||||
| `packages/agent/src/tool-process-tracker.ts` | `observed?` on `TrackedProcess`; `register` opt; `persist()` filters observed |
|
||||
| `packages/agent/src/index.ts` | export `ToolOutputBuffer`, `ObservedHandle` |
|
||||
| `packages/server/src/local/routes/tools.ts` | `observe` in launch schema/handler; decorate `toolOutputBuffer`; new `GET /api/tools/stream` (SSE) |
|
||||
| `apps/web/src/lib/adapter.ts` | `observe` in `launchTool`; new `streamToolOutput` |
|
||||
| `apps/web/src/components/os/apps/LauncherApp.tsx` | clickable Running badge → pane; `?watch=1` mode |
|
||||
| `apps/web/src/components/os/apps/launcher/ToolOutputPane.tsx` | **new** — live output pane |
|
||||
| `apps/web/src/lib/command-catalog.ts` | `watch-agent` ⌘K entry |
|
||||
| tests (4–6 files) | per §7 |
|
||||
|
||||
---
|
||||
|
||||
## 9. Open Questions / Decisions
|
||||
|
||||
- **None blocking.** All three forks resolved: piped (not pty), observed-mode (not replacement),
|
||||
⌘K-deep-link (not new dock buttons).
|
||||
- **Deferred to a later arc (noted, not built):** node-pty upgrade for true terminal + input
|
||||
send; auto-tail of detached launches via a log file; multi-process terminal tabs.
|
||||
- **Founder decisions from the analysis doc unaffected by this item** (budget caps #13,
|
||||
recall-gate cost #8, tuiui re-recon) — out of scope here.
|
||||
@@ -0,0 +1,126 @@
|
||||
# Pluggable Tool-Adapter Registry — Design
|
||||
|
||||
**Date:** 2026-06-30 · **Author:** Claude Opus 4.8 (1M) · **Owner:** Marko (founder)
|
||||
**Arc:** AI-OS external-agent recon · STEAL NOW item **#5** from
|
||||
`docs/analysis/external-agent-launching-and-memory-comparison-2026-06-29.md`
|
||||
**Effort:** M · **Branch:** `feat/tool-adapter-registry` · **v1 scope (founder-approved):** full registry + JSON loader.
|
||||
|
||||
---
|
||||
|
||||
## 1. Goal & Non-Goals
|
||||
|
||||
**Goal.** Turn the 7 hardcoded external tools — currently the same IDs duplicated across **8
|
||||
structures** in `shared` + `agent` — into one derived **adapter registry**, and let a self-hosted /
|
||||
KVARK operator add a **PATH-based CLI runtime by dropping a JSON file** (`~/.waggle/adapters/*.json`),
|
||||
with **no core edit, no recompile, and no executable code load** (honors CLAUDE.md §7: no eval / no
|
||||
dynamic require).
|
||||
|
||||
**Non-goals (YAGNI):**
|
||||
- No third-party **GUI-desktop** adapters (candidate-path detection stays built-in code — see §3 escape hatch). CLI agents are the launch target.
|
||||
- No `require()`/dynamic code load — third-party adapters are **declarative data only**.
|
||||
- No change to the launch/observe behavior shipped in #1–#4. No UI redesign (the dock derives its cohort lists from the same source).
|
||||
- No signed-plugin trust store (that was the rejected "M-plus" option).
|
||||
|
||||
## 2. The 8 hardcoded structures collapsing into 1 source of truth
|
||||
|
||||
| Today (authored separately) | After |
|
||||
|---|---|
|
||||
| `shared`: `SUPPORTED_TOOLS`, `LAUNCH_COHORT`, `TOOL_DISPLAY_NAMES` | derive from `BUILTIN_TOOL_MANIFESTS` |
|
||||
| `agent`: `HOOK_POINTER_BY_TOOL`, `detectorsById` + per-tool `detect*` fns | one manifest-driven detect loop |
|
||||
| `agent` `tool-launcher`: `HOOKS_COHORT`, `hookPackageFor` | derive from the registry (`hookCapable`) |
|
||||
| `agent` `launcher-prompt-args`: `promptArgsForTool` | built-ins unchanged; third-party via declarative `promptArgTemplate` |
|
||||
|
||||
## 3. Architecture — data (shared) vs behavior (agent)
|
||||
|
||||
### Unit A — `ToolManifest` + built-in manifests (`packages/shared/src/tool-detection.ts`)
|
||||
|
||||
```ts
|
||||
export type ToolDetectSpec =
|
||||
| { kind: 'path'; binaryName: string } // PATH lookup (CLI tools; the only third-party-allowed kind)
|
||||
| { kind: 'candidates' }; // GUI/desktop — paths resolved by an agent-side resolver (built-in only)
|
||||
|
||||
export interface ToolManifest {
|
||||
id: string;
|
||||
displayName: string;
|
||||
launchable: boolean;
|
||||
hookCapable: boolean; // ⟺ ships a real @waggle/hive-mind-hooks-<id> bin
|
||||
hookPointer: string; // relative pointer path for hook-status probe
|
||||
detect: ToolDetectSpec;
|
||||
/** Declarative inline-prompt arg template for THIRD-PARTY path adapters, e.g.
|
||||
* ['--print', '{prompt}']. Built-ins keep their logic in launcher-prompt-args.ts. */
|
||||
promptArgTemplate?: string[];
|
||||
/** true = first-party (the 7); false/absent = loaded third-party. */
|
||||
builtin?: boolean;
|
||||
}
|
||||
```
|
||||
|
||||
`SUPPORTED_TOOLS` stays the `as const` literal **type anchor** (preserves the `ToolId` union → zero
|
||||
blast radius on the ~7 places typed `Record<ToolId, …>`). `BUILTIN_TOOL_MANIFESTS: ToolManifest[]`
|
||||
holds the per-tool **data** (the 7). The other consts **derive** from it:
|
||||
`TOOL_DISPLAY_NAMES`, `LAUNCH_COHORT`, `HOOK_POINTER_BY_TOOL` (and a new `HOOKS_COHORT` = manifests
|
||||
where `hookCapable`). Pure data → consumed by both the web bundle and the sidecar.
|
||||
|
||||
### Unit B — registry + detection loop (`packages/agent`)
|
||||
|
||||
- **`tool-registry.ts` (new).** `getToolRegistry(deps?)` returns the merged adapter list: the 7
|
||||
built-in manifests (each `candidates` manifest paired with its existing agent-side resolver
|
||||
`(deps) => string[]`, keyed by id — the **escape hatch**) **plus** validated third-party manifests
|
||||
from the loader (Unit C). Built-in ids win on collision.
|
||||
- **`tool-detection.ts`.** `detectAll` iterates `getToolRegistry()` instead of the `detectorsById`
|
||||
map: `detect.kind:'path'` → `detectByPath(id, binaryName)`; `detect.kind:'candidates'` → the
|
||||
adapter's resolver → `detectByCandidates`. `probeHooks` reads `manifest.hookPointer`. The bespoke
|
||||
`detectClaudeCode`/`detectCursor`/… wrappers and `detectorsById` are deleted; `detectByPath` /
|
||||
`detectByCandidates` / the 3 candidate-path helpers are **kept** (they're the reusable engine).
|
||||
- **`tool-launcher.ts`.** `HOOKS_COHORT` and the launch-cohort guard derive from the registry
|
||||
(`hookCapable` / `launchable`); `hookPackageFor(id)` = `manifest.hookPackage ?? @waggle/hive-mind-hooks-<id>`.
|
||||
|
||||
### Unit C — declarative loader (`packages/agent/src/tool-manifest-loader.ts`, new)
|
||||
|
||||
Reads `~/.waggle/adapters/*.json`, **zod-validates** each, and returns `ToolManifest[]`:
|
||||
- Allowed `detect.kind` for third-party: **`'path'` only** (candidates would require code).
|
||||
- **Safe-string refinement** on `id` / `binaryName` / `hookPointer` / template entries: no shell
|
||||
metacharacters (`; | & $ \` ( )`), no path traversal (`..`), no absolute path separators in
|
||||
`binaryName`. Reject (skip + log) any manifest that fails — never throw into detection.
|
||||
- Injected `readDir` / `readFile` deps so the loader is hermetic in tests; missing dir → `[]`.
|
||||
- `builtin: false` stamped on every loaded manifest.
|
||||
|
||||
## 4. Data Flow
|
||||
```
|
||||
BUILTIN_TOOL_MANIFESTS (shared, data) ──┐
|
||||
├─▶ getToolRegistry() ──▶ detectAll loops adapters
|
||||
~/.waggle/adapters/*.json ─ loader ─────┘ (built-in resolvers for `candidates`;
|
||||
(zod + safe-string, kind:'path' only) detectByPath for `path`)
|
||||
SUPPORTED_TOOLS/ToolId (anchor, unchanged)
|
||||
LAUNCH_COHORT / HOOKS_COHORT / TOOL_DISPLAY_NAMES / HOOK_POINTER_BY_TOOL ── derive from manifests
|
||||
```
|
||||
|
||||
## 5. Error Handling / Security
|
||||
- Loader never throws into detection: a malformed/unsafe manifest is skipped + logged; detection proceeds with built-ins.
|
||||
- `detect.kind:'candidates'` from a third-party manifest is rejected (code-only strategy).
|
||||
- Safe-string refinement blocks shell-metachar / traversal injection in adapter fields (the boundary defense for external descriptors).
|
||||
- Built-in ids always win over a third-party manifest claiming the same id (no built-in hijack).
|
||||
- No `require()`, no eval, no dynamic import of adapter code — data only.
|
||||
|
||||
## 6. Testing (TDD)
|
||||
- **shared:** `BUILTIN_TOOL_MANIFESTS` has 7 entries; `LAUNCH_COHORT`/`HOOKS_COHORT`/`TOOL_DISPLAY_NAMES`/`HOOK_POINTER_BY_TOOL` derive correctly and match today's values (regression-lock the current 7-tool reality).
|
||||
- **loader:** valid path-manifest → parsed + `builtin:false`; `kind:'candidates'` rejected; shell-metachar / `..` rejected; missing dir → `[]`; injected fake fs.
|
||||
- **registry:** built-ins present; a loaded third-party `path` adapter appears and is detectable; built-in id wins a collision.
|
||||
- **detection:** `detectAll` over the registry yields the same results as today for the 7 (injected spawns — existing `tool-detection.test.ts` stays green); a third-party path adapter detects via `detectByPath`.
|
||||
- **launcher:** `HOOKS_COHORT` derivation matches today's 6; `hookPackageFor` honors a manifest override.
|
||||
|
||||
**Gates:** `tsc` 0 (shared/agent/server/web); existing `tool-detection`/`tool-launcher`/`tools-routes*` suites stay green; new units RED→GREEN.
|
||||
|
||||
## 7. File Change List
|
||||
| File | Change |
|
||||
|---|---|
|
||||
| `packages/shared/src/tool-detection.ts` | `ToolManifest`/`ToolDetectSpec` types, `BUILTIN_TOOL_MANIFESTS`, derive the 5 consts |
|
||||
| `packages/agent/src/tool-registry.ts` | **new** — `getToolRegistry()` (built-ins + loaded), candidate-resolver map |
|
||||
| `packages/agent/src/tool-manifest-loader.ts` | **new** — zod + safe-string loader for `~/.waggle/adapters/*.json` |
|
||||
| `packages/agent/src/tool-detection.ts` | drive `detectAll` from the registry; delete per-tool wrappers + `detectorsById` |
|
||||
| `packages/agent/src/tool-launcher.ts` | derive `HOOKS_COHORT` / cohort guard / `hookPackageFor` from the registry |
|
||||
| `packages/agent/src/index.ts` | export registry + loader + manifest types |
|
||||
| `apps/web/src/components/os/apps/LauncherApp.tsx` | derive its local `LAUNCH_COHORT`/`HOOKS_COHORT` from shared manifests (kill the local copies) |
|
||||
| tests (5–6) | per §6 |
|
||||
|
||||
## 8. Open Questions
|
||||
- **None blocking.** Scope (full registry + loader), the candidate-path escape hatch, and the data-only/no-`require()` security model are founder-approved. Third-party GUI-desktop adapters + a signed-plugin trust store are explicit future items, not gaps.
|
||||
201
docs/superpowers/specs/2026-07-02-sticky-erasure-design.md
Normal file
201
docs/superpowers/specs/2026-07-02-sticky-erasure-design.md
Normal file
@@ -0,0 +1,201 @@
|
||||
# Sticky Erasure — GDPR Art.17 suppression that survives re-import
|
||||
|
||||
**Date:** 2026-07-02 · **Feature:** #7 Art.17 tail · **Status:** design approved, implementation pending
|
||||
**Author:** brainstormed w/ founder (Marko), grounded by `sticky-erasure-recon` workflow (6 agents, full write-path map).
|
||||
|
||||
---
|
||||
|
||||
## Problem
|
||||
|
||||
The #7 Art.17 arc (closed 2026-07-02 S1) rotates `archive_uid` to an opaque random id on erase to
|
||||
close a re-identification vector. That rotation has a **documented side-effect** (`raw-archive.ts:194-200`):
|
||||
rotating the uid frees `RawArchive.append()`'s `content→uid` dedup key, so **re-importing an
|
||||
already-erased source re-materializes it**. The searchable summary, verbatim raw-turns, and KG
|
||||
already came back on re-harvest before the rotation too (pre-existing) — so erasure is not "sticky":
|
||||
a user who exercises their right-to-erasure on a ChatGPT thread gets it back the next time that
|
||||
thread is re-exported/re-synced.
|
||||
|
||||
The fix named in-code is an **erased-subject suppression list** consulted at every write seam. A
|
||||
content-keyed tombstone is explicitly **rejected** — storing `sha256(erased-PII)` would reintroduce
|
||||
the very content-derived re-id vector the rotation removed.
|
||||
|
||||
## Decisions (founder-ratified 2026-07-02)
|
||||
|
||||
| # | Decision | Choice |
|
||||
|---|----------|--------|
|
||||
| Granularity | key on | **Per-source `(source, source_ref)`** — the same tuple `eraseBySourceRef` uses; the only durable identity that survives erasure. NOT per-content-hash (re-id vector), NOT per-subject-string (no schema handle, over-erase). |
|
||||
| Durability | permanence | **Sticky + explicit re-consent** — permanent by default; a deliberate "Allow re-import again" UI action deletes the suppression row. |
|
||||
| Scope | reach | **Group-A write paths only**; connector + `ingest_source` documented as a known limitation (they persist no durable subject key today — already un-erasable by subject, orthogonal pre-existing gap). |
|
||||
| Backfill | past erasures | **Backfill** `erased_subjects` from retained `raw_archive` erased rows so historical erasures are sticky too. |
|
||||
| Tombstone | content-hash | **OFF** (consistent with the ratified anti-re-id rotation). |
|
||||
| Capture point | where recorded | **Inside `MindErasure`** — one point feeds both the route and the MCP `erase_memory` tool (the tool emits no audit event; a ledger fed from audit events would silently miss all MCP erasures). |
|
||||
| Mind scope | isolation | **Per-mind** (per the mind-isolation pin `feedback_mind_isolation_no_cross_mind_mixing`). |
|
||||
| Read-failure | on `isSuppressed` error | **Fail-closed on the item** (skip re-materialization). A failing local-SQLite read implies a broken DB where the follow-on INSERT fails anyway; Art.17 wins on the ambiguous item. |
|
||||
|
||||
## Architecture
|
||||
|
||||
Suppression must live **one level up from `FrameStore.createIFrame`** — that universal chokepoint
|
||||
sees only `(gopId, content)` (content-hash), which is exactly the rejected key. `(source, source_ref)`
|
||||
is only in scope at the harvest boundaries. So the check hooks at those boundaries; the capture hooks
|
||||
at the erase boundary.
|
||||
|
||||
### 1. Schema — `erased_subjects` (new table)
|
||||
|
||||
Home: `packages/hive-mind-core/src/mind/schema.ts` (`SCHEMA_SQL`, appended after the `raw_archive`
|
||||
block so it ships on first-init) + an idempotent guarded block in `db.ts` `runMigrations()` (same
|
||||
pattern as the `raw_archive` / `ai_interactions` migrations, no CHECK-list drift).
|
||||
|
||||
```sql
|
||||
CREATE TABLE IF NOT EXISTS erased_subjects (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
source TEXT NOT NULL,
|
||||
source_ref TEXT NOT NULL,
|
||||
erased_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now')),
|
||||
reason TEXT,
|
||||
UNIQUE(source, source_ref)
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_erased_subjects_lookup ON erased_subjects(source, source_ref);
|
||||
```
|
||||
|
||||
**No `content`, no `content_sha256`** — generic substrate only. This is the deliberate *opposite* of
|
||||
`install_audit` (proprietary-interleaved, hand-stripped on OSS export): `erased_subjects` forward-ports
|
||||
to `marolinik/hive-mind` verbatim. No immutability triggers (rows are deletable — that is the re-consent path).
|
||||
|
||||
**Backfill** (one-time, idempotent, in the same migration block, after table create):
|
||||
```sql
|
||||
INSERT OR IGNORE INTO erased_subjects (source, source_ref, erased_at, reason)
|
||||
SELECT source, source_ref, erased_at, erased_reason
|
||||
FROM raw_archive WHERE erased_at IS NOT NULL;
|
||||
```
|
||||
Limitation: past erasures of archive-less summaries (subject-mode on a legacy/append-failed frame with
|
||||
no `raw_archive` row) leave no skeleton to backfill from — accepted (they had no durable archive anchor).
|
||||
|
||||
### 2. `SuppressionStore` (new `mind/suppression.ts`)
|
||||
|
||||
Stateless wrapper over the mind db (like other `mind/` stores), exported from `@waggle/core` and
|
||||
`@waggle/hive-mind-core`:
|
||||
|
||||
```ts
|
||||
class SuppressionStore {
|
||||
constructor(db: Database) // or the project's DB wrapper, matching RawArchive's ctor
|
||||
isSuppressed(source, sourceRef): boolean // indexed SELECT EXISTS; FAIL-CLOSED (error → true)
|
||||
record(source, sourceRef, reason?): void // INSERT OR IGNORE (idempotent)
|
||||
unsuppress(source, sourceRef): boolean // DELETE; returns whether a row was removed
|
||||
list(): { source, sourceRef, erasedAt, reason }[]
|
||||
}
|
||||
```
|
||||
|
||||
`isSuppressed` swallows read errors and returns `true` (fail-closed on the item) with a loud
|
||||
`logger.error` — a genuine read failure means the DB is broken and the subsequent write fails too.
|
||||
|
||||
### 3. Capture — inside `MindErasure` (`mind/erasure.ts`)
|
||||
|
||||
- `eraseBySourceRef(source, sourceRef, reason)` → `suppression.record(source, sourceRef, reason)`
|
||||
(subject mode — the canonical key arrives verbatim).
|
||||
- `eraseFrameComplete(frameId)` → for **each resolved** `(source, source_ref)` subject (it already
|
||||
resolves them via `reconstructSource` + the content-prefix/`metadata.sourceId` fallback) →
|
||||
`suppression.record(...)`. Subject-less frames (connector/ingest single frames) resolve no durable
|
||||
subject → nothing recorded (out of scope, documented).
|
||||
|
||||
Because both the `/api/memory/erase` route (frame + subject mode) and the MCP `erase_memory` tool call
|
||||
these primitives, both surfaces feed the ledger with no drift. Recording happens **inside the erase
|
||||
transaction** so a rolled-back erase does not leave a stale suppression row.
|
||||
|
||||
### 4. Consumption guards (fail-closed)
|
||||
|
||||
| Seam | Guard | Covers |
|
||||
|------|-------|--------|
|
||||
| 3 harvest loops — `server/routes/harvest.ts`, `hive-mind-mcp-server/tools/harvest.ts`, `memory-mcp/tools/harvest.ts` | `if (isSuppressed(item.source, item.id)) { skippedSuppressed++; continue; }` at loop top, via **one shared helper** in hive-mind-core (kills twin-drift) | The whole fan-out per item: summary `createIFrame` + `setMetadata` + `RawArchive.append` + `writeRawTurnFrames` + KG/vector cognify + wiki |
|
||||
| `server/local/harvest-autosync-frame.ts` `writeAutoSyncSummaryFrame` | guard before `createIFrame` | 30-min in-process auto-sync **and** daily cron `harvest_sync` (single shared writer) |
|
||||
| `hive-mind-core/mind/raw-archive.ts` `RawArchive.append` | skip INSERT + return `{ created:false }` if suppressed | Defense-in-depth for any future non-loop caller of the documented-gap primitive |
|
||||
|
||||
The 3 loops call `RawArchive.append` and `writeRawTurnFrames` only from inside the loop, so the loop
|
||||
`continue` already covers those vectors; the `append` guard is a durable backstop, not the primary gate.
|
||||
The shared helper signature: `shouldSuppressHarvestItem(suppression, source, sourceRef): boolean` (or a
|
||||
thin method on `SuppressionStore`) so the 3 call sites are `continue`-only, logic shared.
|
||||
|
||||
`skippedSuppressed` is surfaced in each import's result summary so a suppressed re-import is visible
|
||||
(not a silent drop) — per the "no silent caps" discipline.
|
||||
|
||||
### 5. Re-consent surface
|
||||
|
||||
- `SuppressionStore.unsuppress` + `list` exposed via `@waggle/core`.
|
||||
- Server routes (`server/src/local/routes/memory-center.ts`, next to the erase endpoint):
|
||||
- `GET /api/memory/suppression` → `list()`
|
||||
- `POST /api/memory/suppression/allow` `{ source, sourceRef }` → `unsuppress()`
|
||||
- Memory Center UI (`apps/web/.../memory/MemoryCenterTab.tsx` or the trust/erase area): a "Suppressed
|
||||
sources" list, each row with an **"Allow re-import again"** button (confirm → POST → row disappears).
|
||||
Minimal; co-located with the existing Erase affordance.
|
||||
|
||||
### 6. Out of scope (documented limitation)
|
||||
|
||||
`ingest_source` (doc/URL/PDF/text) and the PRO connector cron persist **no durable `(source, source_ref)`**
|
||||
— `connectorDataToItems` discards the item id, `ingest_source` stamps no `metadata.sourceId` and creates
|
||||
**unlinked** KG entities. They are already un-reachable by subject-mode erasure today (pre-existing,
|
||||
independent of this feature). Making them sticky requires threading the connector item id + stamping
|
||||
`sourceId` + routing `ingest_source` through `importEntitiesForFrame` — a separate engineering item.
|
||||
Noted in `raw-archive.ts` / the erase docstrings.
|
||||
|
||||
### 6b. Known limitations (post-review, deferred — adversarial review 2026-07-02, 21 raised / 14 confirmed)
|
||||
|
||||
All confirmed HIGH + most MEDIUM/LOW findings were **fixed in-arc**. Three are deliberately deferred:
|
||||
|
||||
- **claude-code `decision-of` derived subject (MEDIUM, notable).** `extractDecisions` emits a SEPARATE
|
||||
item per parent whose id = `stableHarvestId('claude-code','decision-of',parentId)` and whose content
|
||||
is the parent's decision-pattern lines. Erasing/suppressing the PARENT subject does not reach this
|
||||
derived subject (distinct key), so the derived frame (quoting the parent's decision text) survives
|
||||
erasure AND re-materializes on re-import/auto-sync. Partly pre-existing (pre-arc it re-materialized
|
||||
under a random key too). Proper fix couples erasure to the derivation (follow `metadata.extractedFrom`
|
||||
in the sweep + record the derived key) — non-trivial, claude-code-specific; deferred to a follow-up.
|
||||
- **`isSuppressed` fail-closed labeling (LOW, cosmetic).** If ONLY `erased_subjects` is unreadable while
|
||||
other tables are healthy (single-table corruption / manual DROP within one process lifetime), every
|
||||
item is skipped and the harvest response mislabels the whole-corpus silent drop as "N erased subjects
|
||||
suppressed (GDPR Art.17)". Fail-closed is the correct Art.17 posture; only the count/message conflates
|
||||
a read-error with a real suppression. Uncommon trigger; deferred (surface a separate error counter).
|
||||
- **Workspace-scope suppression is inert (LOW, cosmetic).** A `mind=workspace` erase records a suppression
|
||||
row in the workspace mind, but harvest only ever writes/reads the personal mind, so the row can never
|
||||
suppress anything and the workspace re-consent panel's promise is vacuously true. Consistent with the
|
||||
ratified per-mind model; the UI/API framing over both minds is the only misleading part. Deferred.
|
||||
|
||||
### 7. OSS forward-port (P2, monorepo-first per §7.5)
|
||||
|
||||
`suppression.ts` + the `schema.ts`/`db.ts` additions are **generic substrate** (no governance/trust
|
||||
fields) → forward-port to `marolinik/hive-mind` `packages/core` verbatim via the curated forward-port
|
||||
(co-located tests, import rewrites). Done after the monorepo commits land + any blocking OSS PR, matching
|
||||
the prior arc's cadence. The `schema.ts`/`db.ts` diffs still get the manual proprietary-review pass.
|
||||
|
||||
## Testing (TDD)
|
||||
|
||||
Unit (`hive-mind-core/tests/mind/suppression.test.ts`):
|
||||
- `record` idempotent; `isSuppressed` true/false; `unsuppress` removes + returns bool; `list` shape.
|
||||
- `isSuppressed` **fail-closed**: a forced read error returns `true`.
|
||||
|
||||
Migration/backfill (`hive-mind-core/tests/mind/schema` or db migration test):
|
||||
- Table created on fresh init; migration idempotent on re-run.
|
||||
- Backfill populates `erased_subjects` from pre-existing `raw_archive` erased rows.
|
||||
|
||||
Capture (`hive-mind-core/tests/mind/erasure.test.ts`):
|
||||
- `eraseBySourceRef` records the pair; `eraseFrameComplete` records each resolved subject; rolled-back
|
||||
erase leaves **no** suppression row.
|
||||
|
||||
Integration — **the headline test**:
|
||||
- Harvest a source → erase it (subject or frame mode) → **re-import the same source** → assert the
|
||||
summary, raw-turns, `raw_archive` row, and KG entities do **NOT** re-materialize, and `skippedSuppressed>0`.
|
||||
- `unsuppress` → re-import the same source → assert it **does** re-materialize.
|
||||
- `RawArchive.append` on a suppressed subject → `{ created:false }`, no row.
|
||||
- `writeAutoSyncSummaryFrame` on a suppressed subject → no frame.
|
||||
|
||||
Gates: `npx tsc --noEmit` on {shared, hive-mind-core, server, memory-mcp, apps/web}; touched-area
|
||||
Vitest green; then a multi-lens adversarial review workflow (matching the arc's cadence) with every
|
||||
confirmed finding fixed/documented in-arc.
|
||||
|
||||
## Phasing
|
||||
|
||||
1. **Substrate** — schema + migration + backfill + `SuppressionStore` + capture in `MindErasure`. (hive-mind-core)
|
||||
2. **Guards** — shared helper + 3 harvest loops + `writeAutoSyncSummaryFrame` + `RawArchive.append` backstop.
|
||||
3. **Re-consent** — routes + Memory Center UI.
|
||||
4. **Adversarial review** — multi-lens workflow; fix confirmed findings.
|
||||
5. **OSS forward-port** — P2, after monorepo lands (per §7.5).
|
||||
|
||||
Commit per phase; push held until founder asks (main is shared with concurrent sessions — fetch-checked
|
||||
fast-forward at push time, as in the prior arc).
|
||||
30
docs/superpowers/specs/2026-07-03-www-fact-sheet.md
Normal file
30
docs/superpowers/specs/2026-07-03-www-fact-sheet.md
Normal file
@@ -0,0 +1,30 @@
|
||||
# www rebuild — verified fact sheet (2026-07-03)
|
||||
|
||||
Every marketing claim on apps/www traces to one of these. Verified directly against the repo / local OSS clone this session unless noted.
|
||||
|
||||
| Claim on site | Source | Status |
|
||||
|---|---|---|
|
||||
| "The AI workspace that remembers" / persistent memory substrate | `packages/hive-mind-core/src/mind/` (FrameStore, HybridSearch, KnowledgeGraph, IdentityLayer, AwarenessLayer); `docs/memory-architecture.md` | ✅ |
|
||||
| Local-first, SQLite on device | `CLAUDE.md` §1 (better-sqlite3 + sqlite-vec); mind substrate | ✅ |
|
||||
| Harvest imports: ChatGPT, Claude, Gemini, Perplexity, PDF, Markdown, URL | `CLAUDE.md` §2 harvest adapter list (`packages/hive-mind-core/src/harvest/`) | ✅ |
|
||||
| LoCoMo 86.49% (N=1,540), +4.54pp vs Memori 81.95, z=4.64; Mem0 same-protocol 73.96; single-hop 92.27 | `benchmarks/results/locomo-sota-2026-06/`; `docs/methodology.md`; MEMORY.md benchmark-discipline pin (87.66 withdrawn — never use) | ✅ |
|
||||
| Reproduce: `cd hive-mind/benchmarks/locomo && node artifacts/w4-n1540/recount.mjs` → `overall 1332/1540 = 86.49%` + "RECOUNT OK" | Read verbatim from `D:/Projects/hive-mind/benchmarks/locomo/README.md` + `artifacts/w4-n1540/recount.mjs` tail | ✅ verified on local clone |
|
||||
| hive-mind is Apache-2.0 | `D:/Projects/hive-mind/LICENSE` (read: Apache License Version 2.0) | ✅ |
|
||||
| npm: `@hive-mind/core` (+ wiki-compiler, mcp-server, cli, claude-code-hooks, enrichment, wiki-web) | OSS `README.md` npm badges | ✅ |
|
||||
| Models: Claude, GPT, Gemini, Grok, DeepSeek, Perplexity, OpenRouter | `litellm-config.yaml` model_name entries (grep this session) | ✅ — NO Qwen entry; "local model with your own keys" kept as configuration option, not bundled inference |
|
||||
| Tiers: TRIAL 15d all features; FREE forever 5 workspaces + agents + built-in skills; PRO $19 unlimited/marketplace/all connectors; TEAMS $49-seat shared/WaggleDance/governance; ENTERPRISE→KVARK | `packages/shared/src/tiers.ts` (read this session: TRIAL_DURATION_DAYS=15, FREE workspaceLimit 5, spawnAgents true, customSkills false; PRO connectorLimit -1, workspaceLimit -1, customSkills true; TEAMS per CLAUDE §1 table) | ⚠️ SUPERSEDED 2026-07-05 |
|
||||
| _(post Solo-vs-Team collapse)_ Two-tier: TRIAL = 15-day Team preview → Solo; FREE(Solo) forever = unlimited workspaces+connectors, marketplace/custom skills, cloud embeddings, PDF/JSON export, basic audit; TEAMS $49-seat shared/WaggleDance/governance; ENTERPRISE→KVARK. **PRO removed.** | `packages/shared/src/tiers.ts` (TIERS=['TRIAL','FREE','TEAMS','ENTERPRISE'], TIER_LABELS FREE→'Solo'); `CLAUDE.md` §1 | ✅ |
|
||||
| "Memory + Harvest free forever" | `tiers.ts` header comment (strategy line, verbatim) | ✅ |
|
||||
| Annual prices $190 / $490-seat | Stripe M7 verification (MEMORY.md; pro_annual/teams_annual lookup keys exist live+test) | ✅ |
|
||||
| 22 personas (8 universal + 14 specialists) | `CLAUDE.md` §5 / `persona-data.ts` | ✅ |
|
||||
| Loops report-only + approval queue (held actions) | MEMORY 0629 S2 (Loops v0 L1 report-only) + S3 (`pending_actions` + `/api/approval`) — shipped to main | ✅ |
|
||||
| Memory Center: view original source, erasure survives re-import, provenance | MEMORY 0630 S4 (view-source), 0702 S2 (sticky erasure `erased_subjects`), #7 arc closed | ✅ |
|
||||
| Injection scanning on external input | `packages/agent/src/injection-scanner.ts` | ✅ |
|
||||
| Keys in local vault | `packages/core/src/vault.ts` | ✅ |
|
||||
| Skills integrity audit ("verified" badge) | MEMORY 0629 S1 (PRO skill-audit loop + badge) | ✅ |
|
||||
| MCP server catalog | `packages/shared/src/mcp-catalog.ts` — surfaced WITHOUT a count on site | ✅ |
|
||||
| Windows & macOS desktop (Tauri 2.0) | `CLAUDE.md` §1 | ✅ |
|
||||
| KVARK sovereign copy | `CLAUDE.md` §9 canonical copy | ✅ |
|
||||
|
||||
## Do-not-claim (removed from old site or never added)
|
||||
SOC 2 · multi-device/priority sync · 48h email SLA · dedicated account manager · 14-day trial (it's 15) · "advanced graph queries" as paid gate · user counts/testimonials · 87.66% · hero-visual fake stats (12,847 edges / 42ms P99 / 17 providers) · SSO/RBAC as shipped Teams feature (RBAC Phase 5 DEFERRED per founder pin) · Qwen as a configured provider.
|
||||
@@ -0,0 +1,87 @@
|
||||
# Waggle Marketing Site (apps/www) — World-Class Rebuild — Design Spec
|
||||
|
||||
**Date:** 2026-07-03
|
||||
**Scope:** `apps/www` presentation layer only. Auth (Clerk), billing (Stripe checkout + webhook), legal pages, and the account page keep their current wiring; they get visual coherence passes only. No changes outside `apps/www` except reading docs.
|
||||
**Mode:** Autonomous (goal-hook). Decisions below are final unless implementation reveals a blocker.
|
||||
|
||||
---
|
||||
|
||||
## 1. Product truth (what the site is allowed to say)
|
||||
|
||||
Only claims verifiable in this repository. Canonical sources:
|
||||
|
||||
- **What it is:** Waggle is a workspace-native personal AI workspace with persistent memory. Tauri 2.0 desktop app (Windows/macOS), local Node sidecar, SQLite memory substrate on-device. (`CLAUDE.md` §1, `packages/hive-mind-core/src/mind/`)
|
||||
- **Memory:** FrameStore + HybridSearch (vector+keyword) + KnowledgeGraph + IdentityLayer + AwarenessLayer. Harvest imports ChatGPT/Claude/Claude Code/Gemini/Perplexity/PDF/Markdown/URL histories on-device. (`packages/hive-mind-core/src/{mind,harvest}/`)
|
||||
- **Benchmark:** LoCoMo 86.49% (N=1,540, GPT-4.1-mini answerer+judge — the prior leader's own protocol), +4.54pp over prior best 81.95, z=4.64. Reproducible offline (`benchmarks/results/locomo-sota-2026-06/`, `node recount.mjs`). **Never 87.66** (withdrawn).
|
||||
- **Open source:** memory substrate published as `hive-mind` (github.com/marolinik/hive-mind), Apache-2.0, npm packages.
|
||||
- **Model-agnostic:** LiteLLM routing; Claude, GPT, Gemini, Qwen/local models (`litellm-config.yaml`).
|
||||
- **Tiers:** TRIAL $0/15 days (all features) → FREE forever (5 workspaces, agents, built-in skills) → PRO $19/mo → TEAMS $49/seat/mo → ENTERPRISE (KVARK, consultative). Memory + Harvest free forever. (`packages/shared/src/tiers.ts`) _SUPERSEDED 2026-07-05: PRO removed (Solo-vs-Team collapse). Pricing is now **two-tier** — Solo (free forever, unlimited workspaces+connectors, marketplace/custom skills) → Team $49/seat + Enterprise. Any future www regen must NOT reintroduce a PRO tier._
|
||||
- **Sovereignty:** data local by default; Memory Center provenance ("view original source") + Art.17 erasure that survives re-import; audit trail; injection scanning on external input.
|
||||
- **Breadth:** 22 personas (8 universal modes + 14 specialists), 15 workspace templates, skills marketplace, connectors, MCP catalog, Loops (report-only + approval queue), launcher/hooks for external AI dev tools (7-tool cohort), WaggleDance team signals.
|
||||
|
||||
**Do-not-claim list (current site violates some):** SOC 2, priority sync across devices, 48h email SLA, dedicated account manager, "14-day trial" (it's 15), user counts, "advanced graph queries" as a paid gate, uptime, customers/testimonials. The full verified fact sheet from repo exploration is appended in §9 before implementation of copy.
|
||||
|
||||
## 2. Audience & jobs
|
||||
|
||||
1. **Knowledge-work professional** (consultant, analyst, founder, PM): wants an AI that stops forgetting; cares about their data staying theirs. Primary CTA: Download / Start free.
|
||||
2. **Technically sophisticated evaluator** (developer, CISO-adjacent, OSS-curious): wants proof, architecture honesty, reproducibility, license. Secondary paths: Methodology, GitHub, EU-AI-Act page.
|
||||
3. **Team lead in regulated industry:** shared memory without cloud exposure; Teams tier; KVARK escalation path.
|
||||
|
||||
## 3. Messaging architecture
|
||||
|
||||
**Category line:** *The AI workspace that remembers.*
|
||||
|
||||
**Narrative spine (landing page order):**
|
||||
|
||||
1. **Hero** — name the wound + the promise. Headline direction: "AI that starts from zero is a tool. AI that remembers you is a colleague." → committed form decided in copy pass; one headline, no A/B variants rendered (keep resolver infra dormant). Sub: memory persists across models and sessions, on your machine. CTAs: Download (OS-aware) + "Read the benchmark" (proof-forward secondary). Microline: Local-first · Model-agnostic · Apache-2.0 substrate.
|
||||
2. **Problem → Turn** ("Every other AI starts from zero") — 3 tight beats of the reset tax: re-explaining context, re-pasting docs, losing decisions. Then the turn: your context is an asset; it should compound.
|
||||
3. **How it works** — 3 steps grounded in real product: (1) Bring your history (Harvest imports from ChatGPT/Claude/Gemini + files), (2) Work in workspaces (personas, skills, any model underneath), (3) It compounds (memory graph grows; provenance + erasure controls). Each step gets a small product-true visual.
|
||||
4. **Memory, shown** — the signature visual: memory substrate diagram (frames → hybrid search → knowledge graph → any model). This is the "what's actually different" section for the technical reader; terminology from the real architecture.
|
||||
5. **Proof band** — LoCoMo 86.49% with the honest protocol sentence + link to /docs/methodology + GitHub. Numbers restrained, no chart-junk: one comparison bar (86.49 vs 81.95 vs 73.96) with sources.
|
||||
6. **Feature grid** — 6 cards, each a real subsystem: Personas (22), Harvest, Multi-model routing, Loops & approvals, Skills & connectors + MCP, Memory Center (provenance/erasure). Terse, concrete, no adjectives.
|
||||
7. **Sovereignty band** — local-first SQLite, what leaves the machine (only model calls you configure), EU AI Act posture (Art. 17 erasure, audit), open substrate. Trust through specificity.
|
||||
8. **Personas strip** — the bee-mascot brand moment (assets exist), reframed with correct count (22 personas).
|
||||
9. **Open source section** — hive-mind: Apache-2.0, npm, reproduce-the-benchmark instructions in a code block (`git clone … node recount.mjs`). Developer-credibility anchor.
|
||||
10. **Pricing** — Free / Pro $19 / Teams $49-seat + KVARK enterprise line. Bullets rewritten from `tiers.ts` capabilities only. Trial framing: "15-day full trial, then free forever tier" per tiers.ts. _SUPERSEDED 2026-07-05: **two-tier** now — Solo (free) / Team $49-seat + KVARK enterprise line; no PRO column._
|
||||
11. **Final CTA** — echo hero promise, Download + GitHub.
|
||||
12. **Footer** — product/research/company/legal columns (keep, tidy).
|
||||
|
||||
**Voice:** plain, confident, specific; short sentences; zero hype adjectives ("revolutionary", "supercharge" banned). Claims carry their evidence inline or link to it. British-neutral English, sentence case everywhere (Linear/Anthropic idiom).
|
||||
|
||||
## 4. Visual language
|
||||
|
||||
- **Keep warm-Hive identity** (tokens already in `globals.css`, product-matching): hive graphite scale + honey accent + Hanken Grotesk/JetBrains Mono. This is a *refinement*, not a rebrand.
|
||||
- **Elevation moves:** consistent 8-pt spacing rhythm; type scale via CSS custom properties (`--text-display` clamp(40,6vw,72) down to `--text-xs`); max-width 1120–1200 container; generous section padding (96–160px); honey used only for accents/CTAs/moments of proof (≤10% of any viewport); mono font for "machine truth" (paths, numbers, protocol lines) — a signature device.
|
||||
- **Texture:** existing honeycomb SVG pattern at ≤5% opacity in hero + final CTA only. Subtle radial honey glow behind hero visual. No parallax.
|
||||
- **Motion:** IntersectionObserver reveal (translateY 12px + fade, 500ms, stagger 60ms) via a tiny client `<Reveal>` component; `prefers-reduced-motion` disables all. Hero visual gets one slow ambient animation (pulse along graph edges). Nothing autoplays aggressively.
|
||||
- **Illustration:** bee mascots (public/brand) confined to the Personas strip; hero uses an abstract product-true "memory graph/terminal" composition (SVG, hand-built, no screenshots since none exist marketing-grade — verify in audit; if real app screenshots exist, prefer one in a framed window).
|
||||
|
||||
## 5. Frontend architecture
|
||||
|
||||
- **Stack unchanged:** Next.js 15 app router, next-intl (copy stays in `messages/en.json`, fully rewritten), Clerk, Stripe. No Tailwind (repo decision) — but **replace inline CSSProperties objects with CSS Modules** per component + shared primitives in `globals.css` (buttons, container, section, eyebrow, card). Media queries live in CSS, killing the `<style>{responsiveCss}</style>` + `!important` hacks.
|
||||
- **Components:** rebuild `_components/` as: `Navbar`, `Hero`, `ProblemTurn`, `HowItWorks`, `MemoryDiagram`, `ProofBand`, `FeatureGrid`, `SovereigntyBand`, `PersonasStrip`, `OpenSource`, `Pricing`, `FinalCTA`, `Footer`, plus primitives `Reveal` (client), `Button`, `SectionHeading`. Server components by default; client only for Navbar scroll state, pricing toggle, Reveal, hero ambient animation.
|
||||
- **Keep intact:** `api/stripe/*`, `account`, `sign-in`, `sign-up`, `(legal)/*`, `docs/methodology` (restyle header/nav link only), `middleware.ts`, `sitemap.ts` (extend), hero A/B resolver infra (dormant), `DownloadCTA` OS detection + event taxonomy (`_lib/`).
|
||||
- **Tests:** update `__tests__` to new structure; keep vitest green.
|
||||
|
||||
## 6. SEO / a11y / perf
|
||||
|
||||
- Metadata: keep dual head strategy; new OG image (1200×630, brand-built, replaces logo.jpeg); JSON-LD `SoftwareApplication` + `Organization`; sitemap covers all public routes; descriptive titles per page.
|
||||
- A11y: one `h1`; landmark structure (`header/main/section[aria-labelledby]/footer`); focus-visible styles; 4.5:1 contrast for body text (hive-300 on hive-950 passes; verify hive-400 usages); skip-to-content link; reduced-motion.
|
||||
- Perf: no new deps; system-loaded Google fonts already via `next/font`; hero visual pure SVG; images via `next/image` where raster; static generation for all marketing routes.
|
||||
|
||||
## 7. Verification criteria (done =)
|
||||
|
||||
1. `npm run build` (apps/www) clean; `npm run test` (apps/www vitest) green; `next lint` clean.
|
||||
2. Every factual claim on the page traceable to §1/§9 sources; do-not-claim list absent.
|
||||
3. Self-review pass (fresh-eyes + adversarial: copy honesty, visual rhythm, mobile at 375px, keyboard nav) with fixes applied.
|
||||
4. No regressions to checkout/auth flows (routes untouched; smoke via build + route presence).
|
||||
|
||||
## 8. Rejected alternatives
|
||||
|
||||
- **Full rebrand / new visual identity** — rejected: product ships warm-Hive; site-product coherence beats novelty.
|
||||
- **Multi-page marketing site (separate /features, /memory, /pricing pages)** — rejected for now: content volume doesn't justify it; single narrative page + methodology doc is the strongest shape at this stage. Anchors + navbar cover navigation.
|
||||
- **Tailwind migration** — rejected: repo explicitly chose vanilla CSS for this app; CSS Modules give the same maintainability without a new build dependency.
|
||||
|
||||
## 9. Appendix — verified fact sheet
|
||||
|
||||
*(Filled from the product-truth exploration before copy implementation; see `docs/superpowers/specs/2026-07-03-www-fact-sheet.md`.)*
|
||||
27
docs/superpowers/specs/2026-07-03-www-rebuild-report.md
Normal file
27
docs/superpowers/specs/2026-07-03-www-rebuild-report.md
Normal file
@@ -0,0 +1,27 @@
|
||||
# apps/www rebuild — completion report (2026-07-03)
|
||||
|
||||
Companion to `2026-07-03-www-marketing-site-design.md` (spec) and `2026-07-03-www-fact-sheet.md` (verified claims). All changes uncommitted, on `main` working tree, awaiting founder go-ahead.
|
||||
|
||||
## Shipped
|
||||
|
||||
**New landing narrative** (page.tsx): Hero → Problem → How it works → Memory substrate → LoCoMo proof → Feature grid → Sovereignty → Personas (kept BrandPersonasCard) → Open source → Pricing → Final CTA. Every string rewritten in `messages/en.json`; every claim traces to the fact sheet.
|
||||
|
||||
**New components** (CSS Modules, server-first): Hero + HeroVisual (zero-JS animated memory-window SVG; replaced fake-stats client version), ProblemTurn, MemoryDiagram, ProofBand (honest 0-100 bar chart: 86.49 / 81.95 / 73.96 + protocol footnote), FeatureGrid (6 real subsystems, custom line icons), SovereigntyBand, OpenSource (verified reproduce-terminal: `node artifacts/w4-n1540/recount.mjs`), rebuilt Navbar/Footer/Pricing/FinalCTA/HowItWorks. New primitives: Reveal (IntersectionObserver, `html.js`-gated for no-JS, reduced-motion safe), BrandMark (SVG replaces JPEG logo), shared `.btn/.eyebrow/.section-*` in globals.css.
|
||||
|
||||
**Deleted**: WowBeat, ComparisonBeat, Pillars, TrustBand, ProofPointsBand, hero A/B variant infra (hero-variants.ts, hero-headline-resolver.ts), 105MB unreferenced assets (icon-*.jpeg, bee-*-light.png, stale `dist/`). Public: 145→40MB.
|
||||
|
||||
**Claims removed** (were unverifiable): SOC 2, priority sync, 48h SLA, dedicated AM, 14-day trial (→15), SSO/RBAC (deferred per founder), hero fake stats (12,847 edges / 42ms / 17 providers).
|
||||
|
||||
**SEO/a11y/perf**: new 1200×630 OG card (public/brand/og.png, sharp-rendered), app/icon.svg favicon, JSON-LD (SoftwareApplication+Organization), app/robots.ts, skip-link, aria-labelled sections, contrast bumps (text-dim→text-muted at small sizes), `/` now **static** (was dynamic — searchParams resolver removed), checkout popup-blocker fix (window.open→location.assign), inline role=alert instead of alert(), mobile grid-blowout fix (minmax(0,1fr)).
|
||||
|
||||
## Gates
|
||||
`npx tsc --noEmit` 0 · `next build` clean (17 routes, / static, 191kB first-load) · vitest 10/10 · `next lint` 0 errors. Visual QA via Playwright at 1440px + 375px (hero, proof, features, pricing, terminal verified; horizontal overflow found and fixed).
|
||||
|
||||
## Open / needs founder
|
||||
1. **Not committed/pushed** — say the word and I'll commit (suggest: single `feat(www): rebuild marketing site` or per-phase).
|
||||
2. **Download funnel**: all CTAs → github.com/marolinik/waggle-os/releases/latest — verify repo is public with a release, else funnel 404s.
|
||||
3. **No real product screenshots / social proof** — needs real app captures + quotes; hero uses product-true SVG meanwhile.
|
||||
4. **Legal pages** still carry "Day-0 placeholder pending legal counsel" copy (untouched).
|
||||
5. **Analytics** still stubbed (event-taxonomy no-ops in prod; Privacy policy mentions PostHog).
|
||||
6. Stale historical docs untouched per surgical rule: SESIJA-D/E manifests, LIGHTHOUSE.md (pre-rebuild numbers — re-run post-deploy).
|
||||
7. `landing.metadata.*` in en.json kept for future locale wiring (layout uses constants).
|
||||
Reference in New Issue
Block a user