This commit is contained in:
Oleg Maslov
73
docs/analysis/agent-teams-ai-vs-waggle-2026-07-15.md
Normal file
73
docs/analysis/agent-teams-ai-vs-waggle-2026-07-15.md
Normal file
@@ -0,0 +1,73 @@
|
||||
# Agent Teams AI vs Waggle OS — Competitive Teardown & Steal List
|
||||
|
||||
**Date:** 2026-07-15
|
||||
**Target:** https://github.com/777genius/agent-teams-ai (v2.1.2, HEAD `1e018a1f`, pushed 2026-07-15)
|
||||
**Method:** shallow clone + 3 parallel deep-read agents (architecture/Kanban, runtime adapters, differentiators). Cross-corroborated; one hallucinated claim (a `docs/product-analysis/WAGGLE-OS-PRODUCT-INTELLIGENCE.md`) verified NOT to exist and discarded.
|
||||
|
||||
> **⚠ LICENSE: AGPL-3.0.** Ideas and mechanism designs only. **Zero code copy** into Waggle (proprietary). Their copyleft is deliberate — likely SaaS-later moat.
|
||||
|
||||
---
|
||||
|
||||
## 1. What Agent Teams AI is
|
||||
|
||||
Electron 40 + React 19 + Zustand desktop app (pnpm, ~2,300 TS files) that orchestrates **teams of external coding-agent runtimes** through a Kanban board. "Manage agents like a CTO manages engineers." Free, local-first, zero telemetry (analytics functions are literally no-op stubs), single $0 pricing tier, Discord community, agentteams.live landing (Nuxt, ~30 locales).
|
||||
|
||||
**Runtime reality vs marketing:** the "9 supported tools" (Claude Code, Codex, OpenCode, Cursor, SuperGrok, Copilot, Z.AI, MiniMax, Kiro) are NOT 9 adapters. Two bundled sidecar binaries do everything:
|
||||
- `claude-multimodel` (from their closed `777genius/agent_teams_orchestrator` repo) — one multi-provider runtime with `anthropic | codex | gemini | opencode` providers inside.
|
||||
- `terminal-platform` daemon — PTY/terminal workspace.
|
||||
|
||||
Cursor/Grok/Z.AI/MiniMax/Kiro are just **model routes through OpenCode/OpenRouter** plus provider-auth connections. The "free model no auth" hook = OpenCode's built-in `opencode/big-pickle` route (`accessKind: 'builtin_free'`).
|
||||
|
||||
**Notable:** the actual teammate launcher + change-ledger **writer** live in the external closed-source orchestrator CLI, not in the AGPL repo. The open repo is the shell: board state engine (`agent-teams-controller/`), readers, review UI, MCP server (FastMCP, tool groups team/task/lead/kanban/review/message/process/runtime/workSync/crossTeam).
|
||||
|
||||
## 2. Category comparison
|
||||
|
||||
| Dimension | Agent Teams AI | Waggle OS |
|
||||
|---|---|---|
|
||||
| Core object | Kanban task executed by external coding CLIs | Workspace agent with persistent memory |
|
||||
| Memory | None (rides Claude's own JSONL; no substrate) | FrameStore + HybridSearch + KG + Identity/Awareness — **the moat they don't have** |
|
||||
| Agent runtime | External CLIs (2 sidecar binaries) | Own agent-loop + LiteLLM routing + personas |
|
||||
| Multi-agent | Lead-orchestrator + teammates, worktree isolation, runtime lanes | WaggleDance signals, subagent-orchestrator, coordinator persona |
|
||||
| Code change control | **Hunk-level review + content-addressed change ledger** — their crown jewel | Review-before-apply proposals (skills only, steal #3 arc) |
|
||||
| External tool launch | Detect/install/launch 2 runtimes; provider auth bridge | Launcher: 7-tool cohort detect/launch + 6 hook packages (AI-OS arc) |
|
||||
| Harvest/import | None | 10+ adapters, dedup, sticky erasure |
|
||||
| Monetization | $0 only, AGPL copyleft, no Stripe anywhere | 4-tier, Stripe live, KVARK funnel |
|
||||
| Compliance | Nothing (no redaction, no audit trail in OSS repo) | install_audit, execution traces, compliance/, GDPR erasure, EU AI Act story |
|
||||
|
||||
**Verdict: adjacent, not head-on.** They orchestrate *coding* agents on repos; Waggle is a general workspace agent platform with memory. Overlap zone = our AI-OS/Launcher arc + subagent orchestration. Their memory-less design means every team relaunch starts cold — our substrate is exactly the gap they can't close without building one.
|
||||
|
||||
## 3. Steal list (mechanisms, not code — AGPL)
|
||||
|
||||
### Tier 1 — high value, direct fit
|
||||
1. **Post-compact context re-injection** (`TeamProvisioningService.injectPostCompactReminder` + `handleCompactBoundary`). On `compact_boundary` stream event, re-inject standing rules + fresh task snapshot into the agent — strict one-shot flag, deferral guards (only when agent idle, no relay in-flight), re-arm on re-compact, and instruction "do NOT start new work this turn, reply one status line." We persist compaction summaries as memory frames (#12, just shipped); the *active re-injection with idle guards* is the missing half for our agent-loop long-runs.
|
||||
2. **Rate-limit auto-resume** (`AutoResumeService.ts`). Parse reset time out of provider error, `setTimeout` nudge ~30s after reset; guards: 12h ceiling, staleness, **run-id capture** (never nudge a session that advanced), re-check alive at fire. Pure-function plan (`scheduled | manual{reason}`) = testable. Fits agent-loop + ai_task scheduler (#17).
|
||||
3. **Scheduler hardening trio** for our cron-store/ai_task: (a) **warm-up timer** — pre-provision runtime N min before cron tick; (b) **auto-pause after N consecutive failures** (default 3); (c) **interrupted-run recovery** — on boot, mark prior-process `running/pending` runs `failed_interrupted`. Plus cwd-exclusion lock so two schedules never share a directory.
|
||||
4. **Tool-approval coordinator patterns** (`RuntimeToolApprovalCoordinator.ts`) to harden our `confirmation.ts`/`permissions.ts`: per-team timeout **actions** (allow|deny|wait), in-flight response claiming (no double-answer), **stale-runId rejection**, live `reEvaluate()` of pending approvals when settings change.
|
||||
|
||||
### Tier 2 — strong, larger arcs
|
||||
5. **Task-change ledger (content-addressed)** — append-only JSONL events per task + sha256 blob store + precomputed summary bundles + freshness stamp (size+mtime+sha256 of 4KB journal tail) for validated-vs-degraded fast path. For Waggle this is a **compliance asset**: agent file-mutation audit trail with provenance/confidence tiers slots straight into our EU-AI-Act/install_audit story, and enables agent-change review UI later.
|
||||
6. **Hunk-level review of agent edits** (their crown jewel): accept/reject per hunk, decisions keyed by stable original indices + context hashes (replay-safe over recompute), reject applied via snippet reverse-replacement with diff3 fallback, stale-check before apply, 10-deep undo. Big arc; extends our review-before-apply from skills to file edits. Only worth it when Waggle agents do heavy multi-file work.
|
||||
7. **Stall monitor + turn-settled control plane.** Level-triggered scanner classifying "is agent actually progressing," alert dedupe journal; provider-neutral Stop-hook "turn settled" spool→drain→reconciler. We already ship Stop hooks (`maybeEmitDiscovery`, WAGGLE_SIGNAL_EMIT) — extending to turn-settled telemetry gives Mission Control real liveness.
|
||||
8. **Provenance-tiered cost attribution** — source-confidence enum (`sdk_exact → gateway_exact → log_parsed → tokenizer_estimated → cost_estimated`) + **"API-equivalent cost"** shown even on subscription/free runtimes (great free-tier framing: "Waggle saved you $X"). Budget evaluator engineering: single-flight drain queue, config fingerprint skip, per-period dedupe keys (threshold notifies once/month). Extends our cost-tracker.
|
||||
|
||||
### Tier 3 — cheap wins / process
|
||||
9. **Critical-coverage test config** — separate vitest project gating ONLY security-boundary files (IPC guards, path decode) with hard thresholds, instead of blanket 80%. Cheap, high-leverage; we could scope one to injection-scanner, vault, channels SEC paths, erasure.
|
||||
10. **Interactive shell-env resolver** — spawn user's login shell → `env -0` to capture real PATH (single-flight, 12s timeout, SIGTERM→SIGKILL, cooldown, best-effort background variant). Directly relevant to our `tool-detection.ts`/Launcher misses when Tauri inherits bare env.
|
||||
11. **Board-as-DAG research note** (`adaptive-task-graphs-research-note.md`) — tasks=nodes, blockedBy=edges, ready work = graph frontier; **selective verification scaled to graph impact**; straggler release as first-class action; coordination-metrics panel (idle rounds, straggler tail, wasted tokens). Feed into WaggleDance/subagent-orchestrator design when we do multi-agent task graphs.
|
||||
12. **Agent-graph live viz** (`packages/agent-graph`, d3-force + canvas, port/adapter isolated) — animated org-chart of running agents with message particles. Mission Control candy; concept only (AGPL).
|
||||
13. **docs/research/ corpus** (~65 files) — ready-made competitive/architecture lit review: ACP deep-dive, CLI-adapter exhaustive search, inter-agent communication standards, orchestrator competitor patterns. Worth one reading pass for the WaggleDance roadmap.
|
||||
|
||||
### Explicitly NOT steal
|
||||
- Their runtime-adapter layer — we already own our agent runtime; wrapping external CLIs as workers is their category, not ours.
|
||||
- Kanban board as primary UX — wrong center of gravity for Waggle (memory/chat-first). Task DAG concepts (item 11) transfer without the board.
|
||||
- Their gap we already beat: no memory, no harvest, no secret redaction on agent output, no telemetry/product signal, no monetization rail.
|
||||
|
||||
## 4. Recommended sequencing (founder call)
|
||||
|
||||
- **Quick arc (days):** #2 rate-limit auto-resume, #3 scheduler trio, #4 approval hardening, #9 critical-coverage config, #10 shell-env resolver. All bolt onto shipped Tier-3 steal infra (#17 ai_task, channels, confirmation).
|
||||
- **Medium arc:** #1 post-compact re-injection (pairs with shipped #12), #7 turn-settled liveness, #8 API-equivalent cost framing.
|
||||
- **Strategic (separate proposal):** #5 change ledger as compliance/audit feature — strongest differentiated fit (EU AI Act) — then #6 hunk review on top if/when agents do heavy file work.
|
||||
|
||||
## 5. Session-log note
|
||||
|
||||
Their repo is itself built by agent teams (guardrail file forbids testing team-launch on named real projects; `.controller-compact-prompt` broker re-prime prompt checked in). Test fixtures break Windows checkout (`Filename too long` under `test/fixtures/team/task-change-ledger/`) — their content-addressed fixture paths exceed MAX_PATH; if we build a ledger, keep blob dirs shallow.
|
||||
138
docs/analysis/cowagent-vs-waggle-2026-07-09.md
Normal file
138
docs/analysis/cowagent-vs-waggle-2026-07-09.md
Normal file
@@ -0,0 +1,138 @@
|
||||
# CowAgent vs Waggle OS — Competitive Teardown & Steal List
|
||||
|
||||
**Date:** 2026-07-09 · **Method:** 4 parallel Opus deep-read agents over a fresh clone of
|
||||
[`zhayujie/CowAgent`](https://github.com/zhayujie/CowAgent) (commit 2026-07-08) + comparison against this repo.
|
||||
All CowAgent claims below are code-verified with file:line by the analysts.
|
||||
|
||||
---
|
||||
|
||||
## 0. What CowAgent is
|
||||
|
||||
**CowAgent = `chatgpt-on-wechat` rebranded in place.** Same repo (created Aug 2022), so its "super AI
|
||||
assistant" pivot launched with **45.9k stars / 10.3k forks** already attached; last push was yesterday,
|
||||
release cadence ~2-3 weeks (v2.0.0 Feb → v2.1.3 Jul 2026). Python monolith (~73k LOC / 292 files), MIT.
|
||||
Commercial parent: **LinkAI** (link-ai.tech) — the open-core-funnel structure is *identical* to
|
||||
Waggle→KVARK: OSS demand-gen → hosted cloud + enterprise (workspaces/RBAC/audit all cloud-only, none in OSS).
|
||||
|
||||
Feature surface is a near-mirror of Waggle: 3-tier memory + nightly distillation, hybrid keyword+vector
|
||||
retrieval, markdown knowledge wiki + graph, self-evolution, skill hub with one-click install, MCP,
|
||||
multi-model routing, Electron desktop + web console + CLI, 13 IM/chat channels.
|
||||
|
||||
---
|
||||
|
||||
## 1. Head-to-head verdict
|
||||
|
||||
| Axis | Winner | Evidence |
|
||||
|---|---|---|
|
||||
| **Memory substrate** | **Waggle, decisively** | CowAgent: brute-force O(N) vector scan (no ANN/sqlite-vec), no reranker, naive 0.7/0.3 linear fusion on mismatched score scales, LLM-prompt-only dedup/contradiction handling, nightly **lossy whole-file rewrite** of MEMORY.md with zero provenance, no GDPR erasure, no identity/awareness layers, no entity extraction (its "knowledge graph" is just markdown links), **zero benchmarks**. Waggle: sqlite-vec + cross-encoder reranker, FrameStore provenance, sticky erasure, LoCoMo 86.49% SOTA. |
|
||||
| **Security** | **Waggle, decisively** | CowAgent has **no prompt-injection defense at all**, no trust model, no permission tiers, no cost tracker; SSRF guard is opt-in and OFF by default; bash tool has only a minimal catastrophic-command blocklist; web console is single-user shared-password. Waggle: injection-scanner (mandated), trust-model, permissions, Tauri IPC allowlist, vault, cost-tracker. |
|
||||
| **Multi-agent / personas** | **Waggle** | CowAgent is explicitly single-agent, one AGENT.md, no persona layer, no orchestration. Waggle: 22 personas, subagent-orchestrator, coordinator, workflow-composer, WaggleDance. |
|
||||
| **Engineering rigor** | **Waggle** | CowAgent grade **C+/B−**: only 9 of 217 tests gated in CI, no lint/typecheck/coverage gates, ~41% typing, 5,028-line god file (`web_channel.py`), unpinned heavy deps. Waggle: ~8k gated Vitest tests, Playwright E2E, strict tsc, CI gates, production signoff. |
|
||||
| **Team / governance / billing** | **Waggle** | CowAgent OSS has none (all punted to LinkAI cloud). Waggle has tiers+Stripe+governance in-product. |
|
||||
| **Distribution & reach** | **CowAgent, decisively** | 13 channels incl. the entire WeChat/WeCom/QQ ecosystem + Telegram/Slack/Discord (Waggle: **0** IM channels). One-line `curl \| bash` installer with China-network resilience (Gitee/pip-mirror fallbacks) and zero-key boot. 45.9k-star inherited brand, trending, 4-language docs (246 .mdx — best-in-class). |
|
||||
| **Online self-evolution** | **CowAgent** | Runtime, conversation-driven evolution loop (see steal #1). Waggle's evolution is offline/eval-gated only. |
|
||||
| **Docs** | **CowAgent** | 246 .mdx, trilingual, per-channel guides. Genuinely excellent. |
|
||||
|
||||
**Net:** CowAgent out-distributes us (channels, installer, brand gravity, docs) but is a shallower,
|
||||
single-user, security-weak product with a hobbyist-grade memory engine and no proof. Waggle's moat
|
||||
(benchmarked substrate, security, teams, rigor) is real. The asymmetric move: **graft their funnel
|
||||
mechanics onto our core** — their moat (WeChat ecosystem + 45k stars) is the only thing we can't copy.
|
||||
|
||||
---
|
||||
|
||||
## 2. Steal list (consolidated, ranked by value/effort)
|
||||
|
||||
### Tier 1 — high value, low-to-medium effort
|
||||
|
||||
1. **"Dream Diary" — user-facing nightly consolidation narrative.** Their Deep Dream distillation
|
||||
(`agent/memory/summarizer.py:414`, prompt :55-141, diary write :585) emits a second `[DREAM]` section: a
|
||||
short narrative of what was merged/conflicted/cleaned, saved to `memory/dreams/YYYY-MM-DD.md` and surfaced
|
||||
in a Self-Evolution UI tab. Waggle already *does* the substance (reconcile, contradiction-detector,
|
||||
dedup) but shows the user nothing. One extra LLM output section + one UI surface = observability,
|
||||
delight, and a retention mechanic. **Cheapest high-impact steal.**
|
||||
2. **Anti-nag file-change gate + "fix the source, not the symptom."** Their evolution reviewer may only
|
||||
notify the user if a watched file *actually changed* (mtime/size snapshot diff, `evolution/executor.py:459`);
|
||||
the prompt forbids logging a symptom to memory when the root cause is an editable skill
|
||||
(`evolution/prompts.py:63-68`). Portable discipline for our evolution + memory writes.
|
||||
3. **Online idle-triggered self-evolution.** Daemon scans sessions every 60s; fires on idle ≥ N sec AND
|
||||
(enough turns OR context >80% of budget) (`evolution/trigger.py:38-53`). Spawns an isolated reviewer
|
||||
agent with a restricted toolset and workspace-confinement guards (`executor.py:117-228, 409-444`),
|
||||
default-`[SILENT]`, with backup_id + `evolution_undo`. It patches skills, **completes promised-but-unfinished
|
||||
deliverables**, and rarely writes memory. Waggle has all the pieces (subagent-orchestrator,
|
||||
evolution-orchestrator, cron-store) but no runtime conversation-driven loop. **Highest strategic value.**
|
||||
4. **IM channels as distribution surface.** Their `channel_factory.py` + `ChatChannel` base +
|
||||
per-platform `*_message.py` normalization is a clean ~2-file-per-platform adapter pattern. Waggle has
|
||||
zero IM reach; "your Waggle workspace agent, live in Slack/Telegram/Discord" is a reach multiplier and
|
||||
fits the Teams tier perfectly (Slack first — it's the Teams buyer's habitat). Port the pattern over the
|
||||
Fastify sidecar; skip the China stack.
|
||||
5. **One-line installer + interactive setup wizard.** `run.sh` (1,362 lines): dep detection → clone with
|
||||
mirror fallback → venv/pip with proxy handling → interactive model+channel wizard writing config →
|
||||
start → CLI handoff. Zero-key boot (config works before any API key; keys added in UI). Waggle has no
|
||||
`curl | bash` self-host story for the sidecar.
|
||||
|
||||
### Tier 2 — solid, medium effort
|
||||
|
||||
6. **Embedding-based on-demand MCP tool retrieval.** Above a threshold (20), tool descriptions are
|
||||
embedded and only top-k relevant tools are injected per turn, union-only within a run so schemas never
|
||||
vanish mid-run (`tool_manager.py:606-676`). Direct upgrade path for our MCP + tool-filter as catalogs grow.
|
||||
7. **MCP hot-reload.** `(mtime, sha256)` signature diff on mcp.json → add/remove/restart only changed
|
||||
servers, no process restart (`tool_manager.py:378-439`). Plus background async MCP boot so the agent
|
||||
serves traffic while `npx`/`uvx` servers start.
|
||||
8. **Hard-capped always-injected core digest.** `MEMORY.md` ≤50 items / 200 lines / 25KB, LLM-maintained
|
||||
dense, always in prompt, with "spillover → memory_search" pointer (`workspace.py:110,186`). A clean
|
||||
token-budget pattern to layer on top of recallMemory/IdentityLayer.
|
||||
9. **Tiered consecutive-failure loop breaker.** 5 identical-arg calls → stop; 3 identical-arg failures →
|
||||
stop; 6 same-tool diff-arg failures → stop; 8 same-tool failures → hard abort with user-facing give-up
|
||||
copy (`agent_stream.py:269-330`). More granular than our boolean loop-guard.
|
||||
10. **Per-capability model routing UI.** Chat/vision/image-gen/ASR/TTS/embedding each routed to a
|
||||
different vendor with one click in the web console. We have LiteLLM underneath; we lack the picker UX.
|
||||
11. **Multi-source skill install grammar + SKILL.md interop.** One resolver accepts Hub name,
|
||||
`owner/repo`, git URL/SSH, local path, direct SKILL.md URL, zip/tar URL, `clawhub:`/`github:` prefixes —
|
||||
with SHA-256 checksums and zip-slip guards (`cli/commands/skill.py`). They use Anthropic's SKILL.md
|
||||
frontmatter convention, making skills cross-tool with Claude Code/OpenClaw — a marketplace-liquidity
|
||||
play our marketplace should join.
|
||||
|
||||
### Tier 3 — nice-to-have / situational
|
||||
|
||||
12. **`context_summary_callback` dual-use** — one summarization LLM call both persists trimmed turns to
|
||||
daily memory and re-injects the summary into live context (`summarizer.py:352`). Saves a call in compaction.
|
||||
13. **Scheduler/cron-pair stripping before long-term memory flush** (`summarizer.py:770`) — keeps
|
||||
automated noise out of long-term memory; directly relevant to WaggleDance signals.
|
||||
14. **Retrieval-time temporal decay** — exp half-life 30d multiplier at fusion time (`manager.py:472`).
|
||||
~~Complementary to our write-time dating; trivial add.~~ **ALREADY SHIPPED (verified 2026-07-11
|
||||
Tier 3 recon): `packages/hive-mind-core/src/mind/scoring.ts:52-63` has exact 30d-half-life
|
||||
exponential decay, write-time anchored, default ON via 'balanced' profile. Do not re-recon.**
|
||||
15. **Skill auto-enable by requirement satisfaction** — skills gate on `requires.env/bins` presence and
|
||||
surface "setup needed" hints (`agent/skills/config.py`). Nice marketplace UX.
|
||||
16. **Trigram FTS5 cascade for CJK keyword search** (`storage.py:952`) — only if we target non-Latin markets.
|
||||
17. **Scheduler as agent tool with `ai_task` mode** — cron/interval/once tasks that re-invoke the agent
|
||||
and push results to the originating channel (`scheduler_tool.py`). We have cron-store; theirs is a
|
||||
cleaner agent-facing proactivity surface.
|
||||
|
||||
### Explicitly NOT worth stealing
|
||||
- Their retrieval engine (we're strictly better), their knowledge graph (link-parsing only), their
|
||||
security model (worse on every axis), Electron+PyInstaller packaging (Tauri is superior), voice-provider
|
||||
breadth (18 ASR/TTS vendors — off-positioning for us).
|
||||
|
||||
---
|
||||
|
||||
## 3. Strategic read
|
||||
|
||||
1. **They validated our exact business model** — MIT OSS assistant → cloud/enterprise funnel (LinkAI ≈ KVARK).
|
||||
They're running it with a 45.9k-star head start and daily commits. This raises urgency on our OSS
|
||||
launch (hive-mind sits at 0 stars) — the SOTA-gated launch strategy now has a fast-moving reference competitor.
|
||||
2. **Their moat is distribution, not tech.** WeChat-ecosystem channels + inherited brand + one-line
|
||||
install. Nothing in their core survives contact with our substrate on quality, but none of our quality
|
||||
is *visible* the way "works in your WeChat/Slack in 2 minutes" is.
|
||||
3. **Differentiation story writes itself:** benchmarked memory (86.49 LoCoMo vs their zero evidence),
|
||||
security (injection scanning vs none), teams/governance in-product (vs cloud-only), test rigor
|
||||
(8k gated tests vs 9). Useful ammunition for waggle-os.ai comparison copy.
|
||||
4. **Their one genuine capability lead** — runtime self-evolution that finishes unfinished tasks and
|
||||
patches its own skills from live conversations — is buildable on infrastructure we already have, and
|
||||
would neutralize their best demo.
|
||||
|
||||
## 4. Source reports
|
||||
|
||||
Full per-domain analyst reports (memory/knowledge, agent core, distribution, code quality) were produced
|
||||
2026-07-09; key findings are consolidated above. Clone analyzed at commit `2026-07-08 fix(desktop):
|
||||
support web_password auth`.
|
||||
@@ -0,0 +1,214 @@
|
||||
# External-Agent Launching & Memory — Comparison + Build Decision
|
||||
|
||||
**Date:** 2026-06-29 · **Author:** synthesis lead (Claude Opus 4.8 1M) · **Audience:** Marko (founder build decision)
|
||||
**Repos compared:** `paperclipai/paperclip` · `jaylfc/taOS` · `jaylfc/taosmd` · `jaylfc/tuiui` (unverified)
|
||||
**Waggle baselines audited:** agent launcher (AI-OS arc) + memory substrate (hive-mind-core)
|
||||
|
||||
> Provenance note: external-repo descriptions are sourced from recon agents. `jaylfc/tuiui` returned **no data** (likely 404 / private / misnamed) — its section is marked provisional. All Waggle file paths in this doc were existence-verified on `docs/w4-sota-doc-sync` (2026-06-29). Behavioral claims about Waggle internals are from the launcher/memory recon, cross-checked against CLAUDE.md §10.
|
||||
|
||||
---
|
||||
|
||||
## 1. TL;DR / Verdict
|
||||
|
||||
**Can we improve Waggle's external-agent launching? Yes — materially, and cheaply.** The launcher today is an honest detect→launch→hook→signal→UI pipeline, but it is *fire-and-forget with no eyes*: it spawns tools `stdio:'ignore'`, sees nothing until a Stop-hook frame lands, and the dock can't even self-enable the signal bus it built. Three of the four external projects independently converged on the orchestration primitives we're missing.
|
||||
|
||||
**Single highest-leverage move:** make `launchTool()` **self-enabling and resumable** — inject `WAGGLE_SIGNAL_EMIT` + `WAGGLE_SIDECAR_URL` + a `runId`/`taskId` into the launch env (the seam is `tool-launcher.ts:226-229`, today it injects *only* `WAGGLE_WORKSPACE_ID`), and persist the process tracker so launched agents survive a sidecar restart. This turns the existing-but-dark pipeline on. Everything else (heartbeat scheduler, worktree isolation, group-chat) is a follow-on.
|
||||
|
||||
**Steal from paperclip, adopt jaylfc, both, or neither?**
|
||||
- **paperclip → STEAL (patterns, not code):** its heartbeat scheduler, pluggable-adapter contract, git-worktree isolation, and per-agent budget caps are the cleanest map onto our Loops/launcher/CostTracker work. MIT-licensed, so code is *legally* portable — but it's PostgreSQL-centric and a different product thesis, so port ideas.
|
||||
- **jaylfc/taOS → PARTIAL:** steal the universal-message-envelope + thin-adapter group-chat seam for WaggleDance; ignore the Python/LXC runtime. **Non-OSS license — patterns only, never code.**
|
||||
- **jaylfc/taosmd → PARTIAL (two ideas):** the source-span **provable-memory recall gate** and **temporal validity windows** on the KG. **Commons-Clause — re-implement, never copy.**
|
||||
- **jaylfc/tuiui → PROVISIONAL/IGNORE:** unverified; no findings returned.
|
||||
|
||||
**Founder decisions flagged:** (a) do launched external agents count against per-agent budget caps (CostTracker), and at which tier? (b) is the provable-memory recall gate worth the per-ingest LLM verify cost on the free-forever memory moat? Both deferred to §8.
|
||||
|
||||
---
|
||||
|
||||
## 2. What Waggle Already Has (honest baseline)
|
||||
|
||||
### 2a. Launcher subsystem (AI-OS arc — real, but partial)
|
||||
|
||||
| Layer | File | State |
|
||||
|---|---|---|
|
||||
| Tool catalog / types | `packages/shared/src/tool-detection.ts` | `SUPPORTED_TOOLS` (7), `LAUNCH_COHORT` (7), display names. Solid. |
|
||||
| Detection engine | `packages/agent/src/tool-detection.ts` | PATH probe for CLIs, candidate-path for desktop apps, hook-pointer probe with **backup-exists verification** (catches partial rollback). DI'd, hermetic. Solid. |
|
||||
| Launch + hooks | `packages/agent/src/tool-launcher.ts` | `launchTool()` detached spawn, `runHookCommand()` shells `npx @waggle/hive-mind-hooks-<id>`. `HOOKS_COHORT` = 6 tools. |
|
||||
| Process tracker | `packages/agent/src/tool-process-tracker.ts` | In-memory `Map<pid,record>`, liveness via `kill(pid,0)`, refuses to kill un-spawned pids. **Not persisted.** |
|
||||
| Sidecar routes | `packages/server/src/local/routes/tools.ts` | `/detect`, `/launch` (202), `/processes`, `/kill`, `/hooks`. zod-validated. |
|
||||
| Signal bus | `packages/server/src/local/signal-bus.ts` | 500-cap in-memory ring buffer. Ephemeral. |
|
||||
| v2 bus surface | `routes/waggle-dance.ts` | normalizes → `WaggleMessage`, dispatches; installs 1C bridge once. |
|
||||
| 1C bridge | `waggle-dance-bridge.ts` | maps 10 protocol subtypes → 5 legacy UX categories; **zero frontend change** to surface activity. Clever. |
|
||||
| Shim emitter | `packages/hive-mind-shim-core/src/signal-emitter.ts` | `maybeEmitDiscovery()` fail-open POST; fires only on stop/pre-compact at high\|critical. |
|
||||
| Hook bodies | `packages/hive-mind-hooks-core/src/handlers-core.ts` | tool-agnostic SessionStart/UserPrompt/Stop/PreCompact via `EventAdapter`. |
|
||||
| Dock UI | `apps/web/src/components/os/apps/LauncherApp.tsx` | detect list, Launch/Stop/Install/Verify/Uninstall, 5s `/processes` poll, optional prompt textarea. |
|
||||
|
||||
**Stub/gap reality (corrects CLAUDE.md §10, which says "6 hooks are Wave 2/3 stubs"):**
|
||||
- Recon found **6 real / 1 stub**: only `hive-mind-hooks-claude-desktop` is still `export {}`. claude-code/codex/codex-desktop/cursor/hermes/openclaw all ship real `bin` installers. **CLAUDE.md §10 OW-3 is stale — verify before quoting it.**
|
||||
- **But the UI lags the backend:** `LauncherApp.tsx:57` hardcodes `HOOKS_COHORT=['claude-code']`, so users *can't* install the 5 other working hook packages from the dock.
|
||||
|
||||
**The four launcher gaps that matter:**
|
||||
1. **No eyes.** Spawn is `stdio:'ignore'` detached (`tool-launcher.ts:106`). Waggle never sees stdout/stderr — only the post-turn Stop-hook frame. No streaming, no attach, no PTY.
|
||||
2. **Capture is Stop-hook-only.** `maybeEmitDiscovery` fires only on stop/pre-compact at high\|critical. Mid-task visibility is nil; SessionStart/UserPrompt persist frames but never broadcast.
|
||||
3. **Launch is not self-enabling.** Env injects **only** `WAGGLE_WORKSPACE_ID` (`tool-launcher.ts:226`). It does *not* set `WAGGLE_SIGNAL_EMIT`/`WAGGLE_SIDECAR_URL`, so a dock-launched tool saves memory but stays **silent on the bus** unless the user globally exported the flag. The headline flow doesn't fire itself.
|
||||
4. **No isolation, no persistence, no orchestration.** Child inherits full `process.env` + cwd; no worktree/sandbox; tracker lost on restart; `launchTool` is one-shot fire-and-forget (no queue, retry, fan-out, completion callback).
|
||||
|
||||
### 2b. Memory substrate (hive-mind-core — the moat, and it's strong)
|
||||
|
||||
- **LoCoMo 87.66% same-judge SOTA** (+5.71pp over Memori, p<10⁻⁵) is delivered by a genuine hybrid stack, not one trick: FTS5/BM25 + sqlite-vec dense + chunk-level vectors + RRF (k=60) + ONNX cross-encoder rerank + KG contextual scoring, all in `mind/search.ts:118`.
|
||||
- Carries **both** representations: distilled/structured lanes (KG entities/relations, profile/fact/event) **and** a verbatim per-turn lane (`harvest/raw-turns.ts`) credited with the single-hop win.
|
||||
- **Offline-first by default:** in-process ONNX embedder (~23MB all-MiniLM, `inprocess-embedder.ts`) + reranker (~22MB ms-marco-MiniLM, `inprocess-reranker.ts`) on CPU, zero API keys; provider chain degrades gracefully.
|
||||
- **Write-time temporal dating** (`frames.ts` createdAt override + `scoring.ts` decay + `recall-context.ts` [YYYY-MM-DD] anchoring) is why temporal leads (+32.7pp vs Mem0).
|
||||
- **Bitemporal KG already exists:** `knowledge.ts` carries `valid_from`/`valid_to` soft-delete + dedup/merge + entity→frame bridge.
|
||||
|
||||
**Honest memory gaps (relevant to the comparison):**
|
||||
- **Not a zero-loss verbatim archive by default.** Primary ingest is the 4-pass LLM distillation (`harvest/pipeline.ts`); it keeps *summaries*, and `classifyFailureFallback='skip'` can drop whole batches on an LLM hiccup. Even the raw-turn lane caps at 2000 turns/conv, caps body length, skips system messages, drops injection-flagged turns, and content-hash-dedups — so it is **not** an append-only literal log.
|
||||
- **No source-span provenance gate.** Frames don't link to an immutable archive span; there's no "demote unsupported claims" verifier. We can't currently *measure* an extraction-hallucination rate.
|
||||
- **Full ingestion is not purely offline** — distillation needs an LLM. Only retrieval/rerank/embed are local.
|
||||
|
||||
---
|
||||
|
||||
## 3. paperclip (`paperclipai/paperclip`)
|
||||
|
||||
**What it is (verified by recon):** MIT-licensed Node.js + React **control plane** that orchestrates *teams* of external coding agents into a "company" (org charts, budgets, goals, governance, audit). ~70k stars, launched Mar 2026, pseudonymous solo maintainer (@dotta). Explicit boundary: *"Paperclip orchestrates. Agents run wherever they run and phone home."* It is **not** an execution plane and has **no memory layer** — that's the gap vs Waggle.
|
||||
|
||||
**Launching/orchestration model:** a DB-backed (PostgreSQL) **Heartbeat Execution** engine — a wakeup queue that per-tick does budget check → workspace resolution → secret injection → skill loading → adapter invocation. Four execution patterns: local CLI/session adapters (start/**resume** Claude Code, Codex, Gemini, etc.), shell-command execution, fire-and-forget HTTP/webhook, and **dynamically-loaded plugin adapters** (`~/.paperclip/adapter-plugins.json`, zero hardcoded imports, `createServerAdapter()`). Execution isolation via **git worktrees + operator branches**. Atomic task checkout (single-assignee) + per-agent monthly budget hard-stops.
|
||||
|
||||
**Call: STEAL (patterns; code is MIT so legally portable, but PG-centric → port ideas).**
|
||||
|
||||
| What to steal | Why | Where it lands in Waggle | Effort |
|
||||
|---|---|---|---|
|
||||
| **Heartbeat scheduler** (DB-backed wake queue: budget→workspace→secret→skill→invoke) | Cleaner orchestration spine than our chat/cron split; generalizes the new `job_type:'loop'` executor toward waking *external* tools, not just internal report-only loops | `packages/server` Loops/cron layer + `packages/agent` loop executor | **L** |
|
||||
| **Pluggable adapter contract** (`createServerAdapter()` + dynamic load) | We hardcode 7 tools in `tool-launcher.ts`/`tool-detection.ts`; an adapter registry lets self-hosted installs add runtimes without core edits | `tool-launcher.ts` + turn each `hive-mind-hooks-*` into a registered adapter | **M** |
|
||||
| **Session resume across heartbeats** | Paperclip reattaches Claude Code/Codex sessions to prior task context; our "Running" badge is one-shot | `tool-process-tracker.ts` + `/api/tools/launch` (add resume-by-session-id) | **M** |
|
||||
| **Git-worktree execution isolation** | We have *no* isolated exec workspace; concurrent launches collide in one workspace | alongside `LauncherApp` + `/api/tools/launch` + `packages/core` FileStore | **M** |
|
||||
| **Per-agent budget caps + atomic task checkout** | Maps directly onto `CostTracker` (`packages/agent/src/cost-tracker.ts`); reinforces the L2 approval-queue governance already shipped | `cost-tracker.ts` + Loops/approval-queue | **M** |
|
||||
| **Goal-ancestry context chain** (mission→project→goal→task injected each run) | Cheap, high-value; always supplies the "why," complements hive-mind recall | orchestrator `buildSystemPrompt()` | **S** |
|
||||
|
||||
**Risks:** control-plane/"company of agents" thesis ≠ our workspace-native memory-first positioning — adopt mechanisms, not narrative. Solo pseudonymous maintainer (bus factor). Young/fast-moving — AGENTS.md references a fork shipping only `hermes_local`/`hermes_gateway`, so the polished multi-adapter marketing may outrun code maturity (verify adapter implementations before porting). PostgreSQL heartbeat queue must be re-implemented on SQLite — not a lift-and-shift.
|
||||
|
||||
---
|
||||
|
||||
## 4. jaylfc/taOS
|
||||
|
||||
**What it is:** self-hosted Python/FastAPI agent OS that deploys long-lived agents into LXC/Docker containers and auto-clusters across consumer hardware. Headline: a **multi-framework group chat** where agents on ~15 different Python frameworks collaborate in one channel while *the platform* (not the framework) owns memory, files, credentials, identity — *"containers hold code, hosts hold state."* That principle directly parallels our memory-moat thesis. Source-available (Sustainable Use License — **not OSS**), beta, ~519 stars, solo maintainer.
|
||||
|
||||
**Launching model — important framing correction:** taOS does **NOT** launch external CLI coding agents (no Claude Code/Codex process orchestration). It deploys *in-process Python agent frameworks* into containers. So it is **not** a direct competitor to Waggle's launcher — it's an adjacent design point. The valuable part is the **collaboration seam**: (1) a shared SSE bridge (`/api/.../sessions/{slug}/events` + `/reply`) where heterogeneous agents join via ~25–100-LoC adapters translating a **universal message envelope** to each framework's native API; (2) an A2A message bus with realtime wake (`a2a-watch`) for point-to-point messaging. (True cross-framework delegation hand-off is explicitly deferred/unimplemented.)
|
||||
|
||||
**Stack fit:** **poor** for the runtime, **good** for the patterns. Python/FastAPI + LXC/systemd + sysfs hardware probing are Linux-server assumptions that don't port to our Windows/macOS Tauri 2.0 + Node sidecar. Adopt the *architecture*, not the code.
|
||||
|
||||
**Call: PARTIAL (patterns only — non-OSS license blocks code reuse for a commercial product).**
|
||||
|
||||
| What to steal | Where it lands | Effort |
|
||||
|---|---|---|
|
||||
| **Universal message envelope + thin per-adapter registry** (~25–100 LoC each) — lets Claude Code / Codex / Cursor sessions post into ONE shared Waggle channel instead of separate silos | `packages/waggle-dance` (normalized cross-agent message schema) | **M** |
|
||||
| **SSE-bridge group-chat seam** — our SignalBus + bridge is *already this shape* (`signal-bus.ts` + `waggle-dance-bridge.ts`); extend it to carry routed **chat turns**, not just discovery/skill_share | `signal-bus.ts` + `waggle-dance.ts` | **M** |
|
||||
| **"Containers hold code, hosts hold state" as an explicit launcher contract** — bind `WAGGLE_WORKSPACE_ID` memory + workspace files on the host so a launched agent's state survives swapping the underlying CLI | launcher env-injection + hook-capture (already in `LauncherApp`/shim-core) | **S** |
|
||||
| **Backend-driven capability discovery** (poll live backends for model/worker readiness, gate UI) vs filesystem discovery | model-route / spawn-agent path (helps open work #1 third-tier fallback) | **M** |
|
||||
| **A2A direct-messaging bus w/ realtime wake** — point-to-point agent coordination without round-tripping the UI channel | WaggleDance v2 | **L (defer)** |
|
||||
|
||||
**Convergent-validation signal (not a steal):** taOS independently picked LiteLLM + SQLite/FTS5 + ONNX hybrid search + temporal KG + LongMemEval/LoCoMo benchmarking — the *same* substrate choices as hive-mind-core. Their **97.0% claim is Recall@5 on LongMemEval-S (retrieval-only); end-to-end judge is 43–51%.** This is **not comparable** to our 87.66% LoCoMo end-to-end same-judge SOTA — different benchmark, different metric. Do not let a casual reader equate them.
|
||||
|
||||
---
|
||||
|
||||
## 5. jaylfc/taosmd vs Waggle memory (head-to-head)
|
||||
|
||||
taOSmd is taOS's memory layer, separately published. Thesis: **provable, auditable memory** — a zero-loss append-only verbatim archive is the source of truth; every extracted fact is tagged with its archive span; a background verifier demotes unsupported claims (the **recall gate**). Five substrates (temporal KG, vector, zero-loss archive, session catalog, crystal store) over SQLite + ONNX CPU embeddings + local Qwen3-4B. **License: MIT + Commons Clause** (cannot sell as a hosted service → re-implement ideas, never copy code). ~62 stars, single author, README self-corrected an inflated 74.6%→43–51% end-to-end after a bug fix.
|
||||
|
||||
| Capability | Waggle (hive-mind-core) | taOSmd | Who leads |
|
||||
|---|---|---|---|
|
||||
| End-to-end accuracy | **LoCoMo 87.66% same-judge SOTA** | LoCoMo 0.748 lenient / 0.659 strict *retrieval*; **e2e judge 43–51%** | **Waggle** (and not comparable on the headline) |
|
||||
| Retrieval stack | FTS5+sqlite-vec+chunk+RRF+CE rerank+KG | hybrid + RRF/mem0_additive/**MaxSim late-interaction** + bge-v2-m3 rerank | ~Tie; taOSmd has MaxSim we lack |
|
||||
| Verbatim archive | raw-turn lane, but **lossy** (2000-turn cap, body cap, dedup, skips system msgs) | **append-only JSONL, never overwritten, source of truth** | **taOSmd** |
|
||||
| Source-span provenance / hallucination gate | **none** (can't measure extraction-hallucination) | **claims tagged to spans + verifier + `prefer_verified` demotion**; measures 18.8% unsupported | **taOSmd** |
|
||||
| Temporal | write-time dating + decay (+32.7pp vs Mem0) | validity windows + point-in-time queries | ~Tie; taOSmd's *explicit validity windows* are sharper |
|
||||
| Bitemporal KG | `valid_from`/`valid_to` exists in `knowledge.ts` | validity-windowed triples + supersession | ~Tie |
|
||||
| Fully offline ingestion | **No** — distillation needs LLM (retrieval is offline) | **Yes** — local Qwen3-4B + ONNX, zero API keys | **taOSmd** |
|
||||
| Maturity / trust | production SOTA, regression-locked | beta, single author, self-corrected benchmark | **Waggle** |
|
||||
| Security default | injection scan at every boundary, parameterized queries | HTTP server ships **no auth** on :7900 | **Waggle** |
|
||||
|
||||
**Concrete steal list (ideas, not code — Commons Clause):**
|
||||
|
||||
1. **Provable-memory recall gate** *(highest-value memory idea)* — tag each frame/claim with its originating harvest span id; run a background verifier (reuse `contradiction-detector.ts` plumbing); let `HybridSearch` (`search.ts`) down-rank unverified claims via a `prefer_verified` flag. Attacks an extraction-hallucination class we currently can't even measure, and feeds the EU-AI-Act audit-trail goal. Lands in `packages/hive-mind-core/src/mind/`. **Effort M.**
|
||||
2. **Zero-loss verbatim archive as a first-class immutable tier** — elevate raw ingested text to an append-only, never-overwritten store every frame links back to (precondition for #1 and for audit). Lands in `harvest/raw-turns.ts` + schema. **Effort M.** *(Note: this complements, does not replace, distillation — see §9.)*
|
||||
3. **MaxSim late-interaction as a selectable fusion mode** — low-risk retrieval lever to A/B on the LoCoMo harness against the current reranker. `search.ts` `SearchOptions`. **Effort S.**
|
||||
4. **Explicit temporal validity windows on KG relations** — we already have `valid_from`/`valid_to`; add point-in-time query + supersession surfacing to harden the temporal lead and enable "what was true as of date X" for Identity/Awareness. `knowledge.ts`. **Effort M.**
|
||||
|
||||
**Do not adopt:** the five-substrate complexity wholesale, the no-auth HTTP server, the 384→1024 zero-pad waste (we already do this — separate cleanup), or their self-reported numbers as validated.
|
||||
|
||||
---
|
||||
|
||||
## 6. jaylfc/tuiui — PROVISIONAL (unverified)
|
||||
|
||||
**Recon returned `null` for this repo.** It could not be fetched — likely 404, private, renamed, or a misremembered name. **No conclusions can be drawn.** The implied premise (a TUI / terminal-multiplexer UI, by the `tui` + `ui` name) maps to a genuine Waggle gap: §2a gap #1 — the launcher has **no terminal/PTY/live-output surface** for launched agents. *If* such a project exists, the concept worth borrowing for `LauncherApp.tsx` is a **PTY-backed live-output pane** (node-pty piped through the sidecar, streamed to a dock terminal view) so users can watch/attach to a launched agent instead of waiting for a Stop-hook frame. **Action: re-run recon with a verified URL before treating any of this as prior art.** Until then, treat the PTY idea as sourced from §2a's own gap analysis, not from tuiui.
|
||||
|
||||
---
|
||||
|
||||
## 7. Gap Analysis
|
||||
|
||||
| Capability | Waggle today | paperclip | taOS | Best-in-class | Priority |
|
||||
|---|---|---|---|---|---|
|
||||
| Detect installed external tools | **Strong** (7 tools, hook-status w/ backup verify) | adapter-declared | n/a (no CLI launch) | **Waggle** | — |
|
||||
| Launch external CLI agent | Yes, detached fire-and-forget | Yes, via adapters + heartbeat | No | paperclip | — |
|
||||
| **Self-enabling launch (signals on by default)** | **No** (only `WAGGLE_WORKSPACE_ID`) | Yes | n/a | paperclip | **P0** |
|
||||
| **Live output / PTY / attach** | **None** (`stdio:'ignore'`) | partial (tracks runs) | SSE channel | tuiui? (unverified) | **P1** |
|
||||
| Session resume / reattach | No (one-shot badge) | **Yes** (across heartbeats) | host-state persists | paperclip | **P1** |
|
||||
| Process persistence across restart | **No** (in-memory) | Yes (DB-backed) | Yes (host state) | paperclip | **P1** |
|
||||
| Execution isolation (worktree/sandbox) | **None** (inherits env+cwd) | **Yes** (worktrees+branches) | container-per-agent | paperclip / taOS | **P1** |
|
||||
| Orchestration (queue/retry/fan-out/budget) | **None** (202 & forget) | **Heartbeat + budget caps** | A2A bus | paperclip | **P2** |
|
||||
| Multi-agent group chat | discovery signals only | org-chart routing | **universal-envelope SSE** | taOS | **P2** |
|
||||
| Memory: end-to-end accuracy | **87.66% SOTA** | **none** | retrieval-only/43–51% e2e | **Waggle** | — |
|
||||
| Memory: zero-loss verbatim archive | lossy | none | **append-only** | taOSmd | **P2** |
|
||||
| Memory: provenance / hallucination gate | **none** | none | **recall gate** | taOSmd | **P2** |
|
||||
| Memory: fully-offline ingestion | retrieval only | none | **yes** | taOSmd | **P3** |
|
||||
|
||||
---
|
||||
|
||||
## 8. Recommendation & Phased Plan
|
||||
|
||||
Respecting Waggle constraints: TS monorepo + Tauri 2.0, sovereignty/offline-first, injection-scanning + vault-only secrets + no-eval, and the memory+harvest-free-forever moat.
|
||||
|
||||
### STEAL NOW (this arc / next)
|
||||
|
||||
| # | Item | What & why | Where (files) | Effort | Risk |
|
||||
|---|---|---|---|---|---|
|
||||
| **1** | **Self-enabling, identified launch env** | Inject `WAGGLE_SIGNAL_EMIT`, `WAGGLE_SIDECAR_URL`, `runId`, `taskId` alongside `WAGGLE_WORKSPACE_ID` so a dock launch actually lights the bus it built. **Highest leverage — turns the dark pipeline on.** | `tool-launcher.ts:226-229` | **S** | Low. Keep emit opt-out per-tier. |
|
||||
| **2** | **Persist the process tracker** | Pidfile-backed store + boot reconciliation so Running badges/kill/attribution survive sidecar restart. | `tool-process-tracker.ts` (`register()` seam) | **S/M** | Low. |
|
||||
| **3** | **Fix UI/backend cohort drift** | Drive `HOOKS_COHORT` from `tool-launcher.ts` (6 real) instead of hardcoded `['claude-code']`; expose codex/cursor/hermes/openclaw install in the dock. | `LauncherApp.tsx:57` | **S** | Low. Smoke each installer. |
|
||||
| **4** | **PTY live-output pane** | node-pty in the sidecar, piped stream to a dock terminal view (swap `stdio:'ignore'` for piped via the `spawnDetached` DI seam). Closes the "no eyes" gap; the §6 tuiui premise. | `tool-launcher.ts:96-109` + new `/api/tools/stream` + `LauncherApp` | **M** | Med — cross-platform PTY on Windows; injection-scan any echoed prompt. |
|
||||
| **5** | **Pluggable adapter contract** | `createServerAdapter()`-style registry + dynamic load so self-hosted installs add runtimes without core edits (paperclip's cleanest idea). Refactors the 7 hardcoded tools into adapters. | `tool-launcher.ts` + `shared/tool-detection.ts` + `hive-mind-hooks-*` | **M** | Med — keep the DI test harness green. |
|
||||
| **6** | **Goal-ancestry context chain** | Inject mission→project→goal→task "why" each run; cheap orchestrator win complementing recall. | orchestrator `buildSystemPrompt()` | **S** | Low. |
|
||||
|
||||
### STEAL SOON (memory moat — needs founder sign-off on cost)
|
||||
|
||||
| # | Item | What & why | Where | Effort | Risk |
|
||||
|---|---|---|---|---|---|
|
||||
| **7** | **Zero-loss verbatim archive tier** | Append-only, never-overwritten raw store every frame links back to (provenance anchor + EU-AI-Act audit). **Additive — does not replace distillation.** | `harvest/raw-turns.ts` + `mind/schema.ts` | **M** | Med — storage growth; needs retention policy. |
|
||||
| **8** | **Provable-memory recall gate** | Tag frames→spans, background verifier (reuse `contradiction-detector.ts`), `prefer_verified` down-rank in `HybridSearch`. First time we can *measure* extraction-hallucination. | `mind/search.ts` + `mind/scoring.ts` + `contradiction-detector.ts` | **M** | **Founder call:** per-ingest LLM verify cost vs free-forever moat. Gate behind a flag; verify async/batched. |
|
||||
| **9** | **MaxSim late-interaction fusion (A/B)** | Selectable fusion mode; low-risk retrieval lever to test on LoCoMo harness — **must not regress 87.66%.** | `mind/search.ts` `SearchOptions` | **S** | Low — behind flag, A/B only. |
|
||||
| **10** | **Explicit KG validity-window queries** | Point-in-time + supersession on existing `valid_from`/`valid_to`; hardens temporal lead. | `mind/knowledge.ts` | **M** | Low. |
|
||||
|
||||
### DEFER
|
||||
|
||||
| # | Item | Why defer |
|
||||
|---|---|---|
|
||||
| 11 | **Git-worktree execution isolation** | High value (concurrent-launch collisions) but **L** effort; do after PTY + persistence land and multi-launch is real. Founder call on whether desktop users need per-task worktrees yet. |
|
||||
| 12 | **Heartbeat scheduler** | Generalize `job_type:'loop'` toward waking *external* tools — but it's an **L** rework of the just-shipped Loops layer; let Loops v0/L2 get usage feedback first. |
|
||||
| 13 | **Per-agent budget caps + atomic checkout** | Maps to CostTracker; do alongside heartbeat. **Founder call:** do external-agent launches count against budget, at which tier? |
|
||||
| 14 | **Universal-envelope group chat / A2A bus** | WaggleDance v2 territory; build after single-agent launch is observable and resumable. |
|
||||
| 15 | **Fully-offline distillation (local Qwen)** | Nice for SBC/KVARK sovereign story, but distillation-quality risk; retrieval is already offline. |
|
||||
|
||||
**Founder decisions needed:** (A) recall-gate LLM verify cost vs free-forever moat (#8); (B) whether launched external agents consume per-agent budget, and tier gating (#13); (C) re-run recon on a verified `tuiui` URL before citing it (#4/§6).
|
||||
|
||||
---
|
||||
|
||||
## 9. What NOT To Do (anti-recommendations)
|
||||
|
||||
1. **Do NOT rewrite the SOTA memory substrate to chase taosmd's framing.** We hold 87.66% LoCoMo end-to-end same-judge SOTA; taosmd's headline is Recall@5 retrieval (97%) with **43–51% end-to-end**. The verbatim-archive + recall-gate ideas are *additive provenance tiers*, **not** a replacement for our distillation+hybrid pipeline. Any change to `search.ts`/`scoring.ts` must A/B against the LoCoMo harness and not regress 87.66%.
|
||||
2. **Do NOT take a Python runtime (taOS/taosmd) into the Tauri binary.** LXC/systemd/sysfs/FastAPI are Linux-server assumptions incompatible with the Windows/macOS Node-sidecar desktop. Port architecture, not runtime.
|
||||
3. **Do NOT copy code from taOS or taosmd.** taOS = Sustainable Use License (non-OSS); taosmd = MIT + **Commons Clause** (no selling as a service). Waggle is commercial (KVARK demand-gen). Re-implement ideas cleanly; cite as prior art at most.
|
||||
4. **Do NOT adopt paperclip's PostgreSQL heartbeat queue as-is.** Re-implement the *pattern* on SQLite/better-sqlite3; a PG dependency breaks the single-file sovereign deploy.
|
||||
5. **Do NOT import taosmd's no-auth HTTP server pattern.** Our sidecar already guards origins; keep injection-scan-at-every-boundary and vault-only secrets.
|
||||
6. **Do NOT adopt paperclip's "zero-human company of agents" narrative.** It clashes with workspace-native, memory-first, human-in-the-loop positioning (and the just-shipped L2 approval queue). Mechanisms yes, thesis no.
|
||||
7. **Do NOT spawn launched agents with full ambient credentials indefinitely.** Today the child inherits all of `process.env`. When adding orchestration (#11–13), scrub/scope env and inject vault secrets per-execution (paperclip's encrypted-at-rest, not-in-prompt model is the bar).
|
||||
8. **Do NOT cite CLAUDE.md §10 OW-3's "6 stub hooks" as current.** Recon shows 6 real / 1 stub; update the doc when the cohort-drift fix (#3) lands.
|
||||
9. **Do NOT treat tuiui findings as real** until a verified URL is re-recon'd. The PTY recommendation stands on §2a's own gap analysis regardless.
|
||||
147
docs/analysis/local-agent-studio-adoption-2026-06-28.md
Normal file
147
docs/analysis/local-agent-studio-adoption-2026-06-28.md
Normal file
@@ -0,0 +1,147 @@
|
||||
All three checks confirmed and one flips a verdict: `prefers-reduced-motion` already exists in `apps/web/src/index.css` (so that ADOPT becomes a SKIP), the `searxng` MCP entry is real at `mcp-catalog.ts:104`, and the `confirmation.ts` taxonomy symbols are exactly as cited. Here is the hardened final report.
|
||||
|
||||
# Local-Agent-Studio → Waggle OS: Prioritized Adoption Report (FINAL)
|
||||
|
||||
## 1. Framing & honest verdict
|
||||
|
||||
Local-Agent-Studio (LAS) is a lean, single-user, **local-first Electron app**: an Ollama-routed tool loop bolted to ComfyUI image generation, multi-provider web search, a subprocess/Docker sandbox, and a clean React chat surface. It is a **media + sandbox toy** — no persistent memory, no knowledge graph, no tiers, no multi-agent orchestration, no governance. Waggle is categorically more mature on everything that constitutes its moat: the `mind/` substrate (FrameStore/HybridSearch/KG), the `confirmation.ts` risk taxonomy, the 200+ MCP catalog, the persona/evolution subsystems, and the 5-tier funnel. **Do not adopt LAS's architecture. Adopt a short list of its product *decisions*, port almost none of its *code*.**
|
||||
|
||||
Two corrections to the draft's optimism, both load-bearing:
|
||||
|
||||
1. **Waggle desktop ≠ KVARK.** KVARK is a *separate* sovereign on-prem product (www.kvark.ai); the Tauri desktop binary is the **demand-gen funnel** that qualifies leads into it. So "this unlocks air-gapped KVARK" is the wrong claim for any feature shipped in the desktop app. The right claim is "this is a *sovereignty proof-point* that opens the KVARK conversation." Sovereign-search and local-inference config qualify under that framing; **Docker command isolation does not** (it is server-side hardening KVARK itself would own, and it carries a Docker Desktop runtime dependency absent on virtually every consumer Tauri install — its value evaporates for the funnel product). Docker is therefore **demoted out of the top-5**.
|
||||
|
||||
2. **The single most strategically-aligned LAS idea is vision *input*, not anything in the draft's top-5.** It feeds Harvest/memory — the free-forever moat — and it is portable across all 13 providers (not Ollama-locked; the Anthropic SDK already exposes image blocks and LiteLLM passes multimodal through). It is also the largest slice, so it is named here as the **flagship strategic bet, scheduled as a deliberate vertical**, not smuggled into "quick wins."
|
||||
|
||||
Net: keep the cheap UX/cost guardrails and the one genuine sovereignty differentiator; sharpen reasoning into a *tier-gated* feature; treat vision as the moat play; drop Docker, ComfyUI, and the already-present reduced-motion CSS.
|
||||
|
||||
## 2. Adoption matrix
|
||||
|
||||
| Capability | Waggle status | Verdict | Impact | Effort | Strategic fit |
|
||||
|---|---|---|---|---|---|
|
||||
| Editable user message + context rewind | missing | **ADOPT** | H | S–M | core UX / retention → moar memory |
|
||||
| Per-turn web-search budget (max N/turn) | missing (daily only) | **ADOPT** | H | S | cost control (incl. built-in proxy) |
|
||||
| SearXNG / self-hosted sovereign search provider | partial (MCP entry only, not native) | **ADAPT** | H | M | **KVARK qualification** (on-prem search) |
|
||||
| Reasoning **control** (`--think` + override + wire Claude thinking, tier-gated budget) | partial (model-locked, no UI, Claude unwired) | **ADAPT** | M | M | agent quality + PRO trigger |
|
||||
| Reasoning/thinking-trace panel (native `<details>`) | missing | **ADOPT** | L–M | S | premium polish (pairs above) |
|
||||
| Provider health probes + UI remote-endpoint config (Ollama/vLLM base URL) | partial (env-var only, no pre-route probe) | **ADAPT** | M | S–M | sovereign-inference proof-point |
|
||||
| Multimodal **vision input** (attach → base64 → model) | missing | **ADAPT (flagship)** | H | L | **memory moat** (Harvest ingests images) |
|
||||
| Local PC date/time + timezone injection | partial (memory-anchor only) | **ADOPT** | L–M | S | works air-gapped; label vs anchor |
|
||||
| Agent task queue (queue prompt while busy) | partial (bus is inter-agent) | **ADOPT** | L–M | S | UX polish |
|
||||
| Multi-format DB export tool (JSON/CSV/SQLite) | missing (as a tool) | **ADAPT** | L–M | S | data-engineer persona / tier |
|
||||
| Per-category permission toggles (files/search/terminal/db/mcp) | missing (trust-level + risk-class) | **ADAPT (caution)** | M | M | governance / TEAMS trigger — *but permission-model sprawl risk* |
|
||||
| Docker isolation mode for command exec | partial (subprocess denylist) | **DEFER** | L (for funnel) | M–L | off-funnel; KVARK-side, heavy dep |
|
||||
| Reduced-motion a11y CSS | **HAS** (`apps/web/src/index.css`) | **SKIP** | — | — | already shipped |
|
||||
| Image **generation** (ComfyUI graph submit/poll/presets) | missing (DALL-E/Replicate MCP exist) | **SKIP** | — | — | off-core; consumer-creative |
|
||||
| Agentic tool loop / router / observation chaining | has (better) | **SKIP** | — | — | redundant |
|
||||
| Streaming token events (requestId/SSE) | has | **SKIP** | — | — | redundant |
|
||||
| Custom markdown parser / streaming render | has | **SKIP** | — | — | redundant |
|
||||
| First-launch setup wizard | has (6-step OnboardingWizard) | **SKIP** | — | — | Waggle better |
|
||||
| Workspace file CRUD + path-traversal guard | has (`resolveSafe`) | **SKIP** | — | — | redundant |
|
||||
| JSON-RPC MCP client / discovery / invocation | has (200+ catalog) | **SKIP** | — | — | redundant |
|
||||
| Settings deep-merge / update.json checker | has / Tauri updater | **SKIP** | — | — | redundant + Electron-shaped |
|
||||
| Message compaction (keep last 14) | has (own ctx mgmt) | **SKIP** | — | — | redundant |
|
||||
| Runpod remote-GPU marketplace config | n/a | **SKIP** | — | — | off-strategy (cloud GPU) |
|
||||
|
||||
## 3. Specs for ADOPT / ADAPT items
|
||||
|
||||
### A. SearXNG sovereign search provider (ADAPT)
|
||||
**Build:** Promote SearXNG from "installable MCP" to a **first-class native search provider** so an on-prem/air-gapped deployment has real web search with zero cloud egress — and so Waggle can *demo* sovereign search as a KVARK qualification proof-point.
|
||||
- Add `searxng_search` alongside the existing tools in `packages/agent/src/search-tools.ts` (which today defines `perplexity_search`/`tavily_search`/`brave_search`). Port LAS's **`normalizeResult()` schema-adapter** — it maps heterogeneous `{snippet|content, href|url, name|title}` into Waggle's result shape — that's the only genuinely reusable LAS search code, and it's pure JS (Electron-free, fully portable).
|
||||
- Register it in `SEARCH_PROVIDERS` in `packages/server/src/local/routes/providers.ts` with a configurable base URL (`SEARXNG_HOST`) and **highest priority when set** (sovereign-first), falling back to the cloud four.
|
||||
- Add base-URL config to the Settings 'Search Providers' panel (`apps/web` SettingsApp).
|
||||
- **Verified:** the catalog already carries a `searxng` MCP entry (`packages/shared/src/mcp-catalog.ts:104`). Keep it; the native provider is the deterministic, agent-default path the MCP can't guarantee.
|
||||
- **Security:** route SearXNG results through the same `scanForInjection()` path as other web results — self-hosted ≠ trusted content.
|
||||
**Reuses:** provider-priority routing, `DailyRateLimiter`, the search tool contract. **Tier:** all tiers; the *sovereign* angle is the **ENTERPRISE/KVARK** sales line — framed honestly as a proof-point, since the desktop app is the funnel, not KVARK itself.
|
||||
|
||||
### B. Per-turn web-search budget (ADOPT)
|
||||
**Build:** A hard **max-searches-per-agent-turn** cap (LAS uses 3). Waggle's daily limiters (`DailyRateLimiter`) and the `web_search` 10/min `RateLimiter` in `system-tools.ts` (lines 40-41) do **not** stop a single malformed loop from firing search N times in one turn — and on FREE/TRIAL those calls can hit the **built-in anthropic proxy / Waggle-funded** path, so this is a Waggle cost exposure, not only the user's premium quota.
|
||||
- Add a per-turn counter scoped to the agent loop in `packages/agent/src/agent-loop.ts` / `retrieval-agent-loop.ts`, incremented by any `*_search` tool, that **short-circuits with an observation** ("search budget exhausted this turn") rather than throwing.
|
||||
- Express the cap as a constant (default 3–5) and let `loop-guard.ts` own it if a budget primitive already lives there — mirror `iteration-budget.ts`, don't invent a parallel mechanism.
|
||||
**Reuses:** the loop's observation-injection path. **Tier:** all; matters most for cost-controlled TEAMS/ENTERPRISE and for protecting Waggle-funded proxy spend.
|
||||
|
||||
### C. Editable user message + context rewind (ADOPT)
|
||||
**Build:** Let a user edit any prior user message; truncate everything after it; rerun from there. LAS does this with a **single array slice** (`messages.slice(0, index)` + clear queue) — the frontend technique is ~20 lines; the backend truncate is the real (small) work.
|
||||
- Frontend: add an Edit action in `apps/web/src/components/os/apps/ChatApp.tsx` (next to the existing copy/pin actions, ~1115-1209); on save, slice local message state and re-send.
|
||||
- Backend: Waggle chat is append-only `.jsonl` (`packages/server/src/local/routes/chat-persistence.ts`) with only `POST /api/chat` and `DELETE /api/chat/history` (`chat.ts:451`/`1916`). Add a **truncate-from-index** operation (a `fromIndex` on the send path is more surgical than a new endpoint) that rewrites the session `.jsonl` to the kept prefix before streaming the new turn. Add `editedAt` to `ChatMessage` in `apps/web/src/lib/types.ts` (~389-399).
|
||||
- **Security:** truncate must be path-scoped to the caller's own session file via the existing persistence helpers — no raw filename from the client.
|
||||
**Reuses:** SSE send path + persistence. **Tier:** all. Retention-grade UX → more sessions → more memory accumulated → moat.
|
||||
|
||||
### D. Reasoning controls + trace panel (ADAPT control / ADOPT trace — paired)
|
||||
**Build (control):** A `--think off|low|medium|high` message flag plus a per-request override in the POST body, resolving **message-flag > settings > model default**.
|
||||
- Parse the flag where the prompt is assembled and thread a `thinking` value through `packages/agent/src/retrieval-agent-loop.ts` (it already carries thinking at 70-76/440-444/622-628) into the model call.
|
||||
- **Wire Claude extended-thinking**, referenced in `prompt-shapes/claude.ts` but never sent: add the `thinking` block param in `packages/server/src/local/routes/anthropic-proxy.ts` (~172-189). **Caveat (not a one-liner):** extended thinking also requires handling the thinking-delta stream and the API's temperature/param constraints — budget for that, don't assume it's a single field. Qwen already flips `enable_thinking` via `litellm-config.yaml` (238-245); this makes it user-controllable and extends it to Claude.
|
||||
- **Cost gate (critical):** thinking tokens are billed. On the **built-in anthropic proxy** (FREE/TRIAL, Waggle-funded), expose only `off`/`low`; medium/high and explicit budgets are a **PRO+** capability via `TierCapabilities`. Otherwise a FREE user sets `--think high` and Waggle eats the bill.
|
||||
**Build (trace):** A collapsible reasoning panel using native `<details>` (LAS's exact pattern, no JS state). Add a `reasoning`/`thinking_trace` member to the `ContentBlock` union in `apps/web/src/lib/types.ts` (484-489) and render it in `chat-blocks/BlockRenderer.tsx` (today only groups tool steps into ActivityStream, 21-51). The benchmark harness already extracts `reasoning_content`; reuse that separation server-side.
|
||||
**Reuses:** prompt-shape selector + token stream. **Tier:** control surface = all (off/low); deeper budgets + trace = **PRO** polish point.
|
||||
|
||||
### E. Provider health probes + remote-endpoint config (ADAPT — pair)
|
||||
**Build:** A `GET /api/providers/health` returning LAS's uniform `{id, kind, status, latencyMs}` schema (probe Ollama `/api/tags`, vLLM, search backends; "configured" for credential-only ones), plus a **Settings UI to set remote Ollama/vLLM base URLs** instead of env-only (`OLLAMA_HOST`/`VLLM_HOST`).
|
||||
- Extend `checkOllama`/`checkVllm` in `packages/server/src/local/routes/local-inference.ts` (172-198) into a parallel `measured()` probe set; render a status panel in SettingsApp. Use the result to **pre-flight before routing** in `model-availability.ts` (`resolveUsableModel`) so a dead endpoint fails fast instead of timing out mid-turn.
|
||||
**Reuses:** existing discovery code. **Tier:** all. **Runpod-specific config is explicitly dropped** (cloud-GPU marketplace, off-strategy); the strategic value here is *local/sovereign* inference config — a KVARK proof-point, same framing as A.
|
||||
|
||||
### F. Local date/time + timezone injection (ADOPT)
|
||||
**Build:** Inject `new Date()` + IANA timezone into the system prompt, gated by a setting, and **skip web search when the user just asks "what's the date"** (LAS's `isLocalDateQuestion()`).
|
||||
- Waggle today injects only memory-derived anchor dates (`TEMPORAL_GUIDANCE` / `renderReferenceDateLine` in `hive-mind-core/src/mind/recall-context.ts`, wired at `orchestrator.ts:~808`). Add a real-clock line **next to** it in `buildSystemPrompt()`.
|
||||
- **Design tension to respect:** Waggle's temporal model is deliberately *memory-anchored* (relative dates resolve against memory timestamps, not wall-clock). Label the new line unambiguously as "current real-world date/time" and keep the memory **anchor** date separate, or the model will conflate "today" with the date of a recalled old frame.
|
||||
**Reuses:** existing temporal block. **Tier:** all; the gating switch supports air-gapped mode.
|
||||
|
||||
### G. Per-category permission toggles (ADAPT — with caution)
|
||||
**Build:** A Settings UI presenting files/search/terminal/database/mcp as **allow/ask/deny** toggles — the mental model users expect — *projected onto* Waggle's existing engine, not replacing it.
|
||||
- Map each category to the tools Waggle already classifies; persist into `PermissionsData` in `packages/server/src/local/routes/settings.ts` (232-307, already holds `defaultAutonomy`/`externalGates`/`workspaceOverrides`) and consult it inside `confirmation.ts` (`classifyGatedToolRisk`/`needsConfirmation`) as an **additional** gate.
|
||||
- **Sprawl warning (§3.2):** Waggle already has **three** permission axes — autonomy level (normal/trusted/yolo), risk class (critical/elevated/medium/low), and the `ALWAYS_CONFIRM` set. A category axis is a **fourth**. Ship it only with an explicit, documented **precedence rule** (recommended: deny/ask categories *tighten* but never *loosen* — they can force a confirm but can never autopass something the existing axes would gate). **Do not weaken `CRITICAL_NEVER_AUTOPASS`** (`confirmation.ts:220/237`, verified) — `isCriticalNeverAutopass` must still fire regardless of any category set to "allow."
|
||||
**Reuses:** `confirmation.ts` taxonomy + `settings.ts` overrides. **Tier:** governance surface → **TEAMS** trigger. Note LAS code reuse here is ~zero; this is a Waggle UI re-skin of Waggle's own model.
|
||||
|
||||
### H. Multimodal vision input (ADAPT — FLAGSHIP strategic bet)
|
||||
**Build:** Let users attach images that reach vision models — the **single most moat-aligned** LAS idea because it feeds **Harvest/memory** (ingest screenshots, diagrams, whiteboards) and lets agents *read* images, not merely because it's a chat nicety.
|
||||
- Port LAS's `attachments.cjs` MIME-detect + `imageBase64List()` (pure utility, framework-agnostic, portable).
|
||||
- Vertical wiring (this is why it's L effort — every layer is necessary, none is skippable):
|
||||
- `ContentBlock` union → add an image block (`apps/web/src/lib/types.ts` 484-489);
|
||||
- composer drag/drop/paste in `ChatApp.tsx` (crib the existing FilesApp `FileUploadZone`);
|
||||
- widen `AgentMessage.content` beyond `string | null` (`agent-loop.ts` 17-22);
|
||||
- construct provider-appropriate image blocks where `chat.ts` today does `{ role:'user', content: message }` (`chat.ts:689`) and in `anthropic-proxy.ts`.
|
||||
- **Portability is good, not a trap:** Anthropic, GPT, Gemini, and Qwen-VL all accept image input; the Anthropic SDK already exposes `ImageBlockParam`; LiteLLM forwards multimodal. This is **not** Ollama-locked — the value survives the Electron→Tauri / local→multi-provider move intact.
|
||||
- **Security/cost:** cap attachment size/count, strip EXIF on ingest, and tier-gate high-volume image turns on the built-in proxy (vision tokens are expensive).
|
||||
**Tier:** input = all; the *Harvest-into-memory* path is the **moat**. **Sequence:** a planned vertical *after* the §5 quick wins — it touches the most layers and deserves its own arc, not a slot in a guardrail sprint.
|
||||
|
||||
### I. Smaller ADOPTs
|
||||
- **Agent task queue:** queue user prompts while busy, dequeue on the `busy→idle` transition (LAS's `useEffect([busy])`), with a count badge. Mostly `ChatApp.tsx` state; portable. **Tier:** all.
|
||||
- **DB export tool:** a `create_dataset` tool writing JSON+CSV+(optional)SQLite via `node:sqlite` (still experimental on Node 20/22 — wrap in the graceful fallback LAS uses), built on existing `resolveSafe` file infra in `system-tools.ts`. **Security:** it has a write side-effect, so it **must** be added to `ALWAYS_CONFIRM` (`confirmation.ts:16`, verified) — same gate as `write_file`/`edit_file`. **Tier:** pairs with the **data-engineer** persona.
|
||||
|
||||
## 4. Explicit SKIPs / DEFERs
|
||||
|
||||
- **Docker isolation mode → DEFER (demoted from the draft's top-5).** Three independent reasons: (1) **Portability collapse** — it needs Docker Desktop installed; the typical Tauri desktop user (FREE/PRO/TEAMS) has no Docker runtime, so the feature silently no-ops or hard-fails. (2) **Wrong product** — KVARK is a *separate* sovereign server platform; container isolation is hardening *it* would own, not a demand-gen-funnel feature. (3) **§3.2 simplicity** — it's not a "clean dispatcher port"; it's a second execution subsystem (container lifecycle, image pull, volume mounts, cross-platform Docker detection, absent-Docker error paths) for marginal security gain over the *already shipping* `bash` denylist + `createSanitizedEnv()` + `confirmation.ts` gating. Keep subprocess+denylist+confirmation as the desktop answer; revisit Docker only as a KVARK-side, opt-in mode when a Docker host is guaranteed.
|
||||
- **Reduced-motion CSS → SKIP (verdict flipped).** **Already present** in `apps/web/src/index.css` (verified). Nothing to port.
|
||||
- **ComfyUI image generation** — ComfyUI-specific graph engine, near-zero fit with an enterprise-memory funnel, and it is *generation* (does not feed the moat) vs. vision *input* (does). If image gen is ever wanted, expose a thin `generate_image` tool over the **existing DALL-E/Replicate/OpenAI MCP entries** (`mcp-catalog.ts:159-161`), not a ported ComfyUI graph engine.
|
||||
- **Agentic tool loop / router / observation chaining / multi-stage routing** — `agent-loop.ts` + `orchestrator.ts` + `tool-filter.ts` are strictly more capable; porting LAS's Ollama router is a regression.
|
||||
- **Streaming token events, custom markdown parser, message compaction** — Waggle has SSE streaming, block rendering, and its own context management.
|
||||
- **First-launch setup wizard** — Waggle's 6-step persona-aware `OnboardingWizard` is better.
|
||||
- **Workspace file CRUD + path-traversal guard** — `resolveSafe()` is equivalent.
|
||||
- **JSON-RPC MCP client / discovery / invocation** — 200+ catalog + routes already ship; LAS's client is a subset.
|
||||
- **Settings deep-merge, `version.json` update checker, Electron packaging** — Waggle has settings infra and a **Tauri** updater; importing Electron patterns is architecturally wrong.
|
||||
- **Runpod remote-GPU config** — cloud-GPU marketplace, contradicts sovereign/local-first positioning. Keep only the generic UI remote-endpoint + health-probe idea (item E).
|
||||
|
||||
## 5. Ranked "do these" (top 5 quick wins)
|
||||
|
||||
1. **Editable message + context rewind** (C) — universal, every-session friction fix; one screen of frontend + a `fromIndex` truncate on the existing send path. Zero strategic downside, retention upside.
|
||||
2. **Per-turn search budget** (B) — cheapest guardrail in the report; a loop-scoped counter that prevents runaway quota/proxy burns (incl. Waggle-funded FREE/TRIAL calls). Reuses `loop-guard`/`iteration-budget` philosophy.
|
||||
3. **SearXNG sovereign search provider** (A) — the one genuine *sovereignty differentiator*; M effort, reuses provider routing, port only `normalizeResult()`. Framed honestly as a KVARK qualification proof-point, not "KVARK itself."
|
||||
4. **Reasoning control wiring, tier-gated** (D-control) — finishes the half-built thinking pipeline (wire Claude thinking, expose `off/low/medium/high`), improving agent task quality (→ better memory) while **capping cost on the built-in proxy by tier**. The trace panel rides along as cheap polish.
|
||||
5. **Provider health probes + UI remote-endpoint config** (E) — fail-fast model UX plus UI-driven local/vLLM endpoint config; the local-inference half of the sovereignty story. Runpod dropped.
|
||||
|
||||
**Flagship strategic bet — schedule as its own vertical, not a quick win:** **Multimodal vision input** (H). It is the *highest moat-value* LAS idea (Harvest ingests images) and fully portable across Waggle's providers, but it touches the most layers; give it a dedicated arc after the guardrail sprint.
|
||||
|
||||
*Honorable mentions, all small:* local clock injection (F), agent task queue (I), DB export tool (I — gate via `ALWAYS_CONFIRM`).
|
||||
|
||||
---
|
||||
|
||||
## Critique deltas (what changed from the draft, and why)
|
||||
|
||||
- **Docker isolation (E): top-5 → DEFER.** Three failures the draft missed: (1) portability — Docker Desktop is absent on the typical Tauri desktop install, so the value evaporates for the funnel product (the LAS→Waggle environment delta the brief asked to test); (2) strategic mis-attribution — it's KVARK-server hardening, and Waggle desktop ≠ KVARK; (3) §3.2 — it's a full second execution subsystem, not a "clean dispatcher port," for marginal gain over the shipping denylist+sanitization+confirmation stack.
|
||||
- **Reduced-motion CSS: ADOPT → SKIP.** Verified **already present** in `apps/web/src/index.css`. The draft's own "verify first" hedge was correct; I verified, and it flips.
|
||||
- **Vision input (H): "honorable mention/bigger bet" → named FLAGSHIP.** It is the only item that feeds the memory moat (the founder's #1 strategy), and I corrected the implicit portability worry: it is multi-provider, not Ollama-locked (Anthropic SDK image blocks + LiteLLM passthrough), so the value survives the port. Kept honest on L effort by scheduling it as a vertical, not a quick win.
|
||||
- **Reasoning (D): added a tier-gated cost guard.** The draft exposed `--think high` with no cost ceiling; on the Waggle-funded built-in proxy that's a FREE-tier billing hole. Now `off/low` for built-in proxy, deeper budgets PRO+. Also flagged that wiring Claude extended-thinking is more than one param (thinking-delta stream + API constraints).
|
||||
- **Per-category toggles (G): ADAPT → ADAPT (caution) with a mandatory precedence rule.** The draft layered a 4th permission axis onto Waggle's existing three without addressing contradictory-state risk; I require a "tighten-only, never loosen, never override `CRITICAL_NEVER_AUTOPASS`" rule (symbols verified at `confirmation.ts:16/220/237`).
|
||||
- **SearXNG (A) + health probes (E): strategic claim softened from "unlocks KVARK" to "KVARK qualification proof-point,"** because the desktop binary is the funnel, not the sovereign product. Confirmed the `searxng` catalog entry exists (`mcp-catalog.ts:104`) so the native-vs-MCP framing stands.
|
||||
- **Per-turn budget (B): widened the cost rationale** to include Waggle-funded built-in-proxy spend, not just the user's premium quota — strengthens the strategic case and bumps it up the ranking.
|
||||
- **Top-5 reordered** to weight leverage-per-effort and verified strategic fit: C, B, A, D, E — replacing Docker with health-probes and pulling the two cheapest universal wins (C, B) to the front.
|
||||
- **DB export + create_dataset:** made the `ALWAYS_CONFIRM` gating explicit (write side-effect) and flagged `node:sqlite` as still experimental — both were under-specified in the draft.
|
||||
55
docs/analysis/locomo-87.66-vs-85.26-integrity-2026-06-30.md
Normal file
55
docs/analysis/locomo-87.66-vs-85.26-integrity-2026-06-30.md
Normal file
@@ -0,0 +1,55 @@
|
||||
# LoCoMo headline integrity — 87.66% is not reproducible; canonical = 86.49% (settled 2026-07-01)
|
||||
|
||||
**TL;DR.** The published LoCoMo SOTA headline **87.66% (1350/1540)** does not reproduce on a fresh
|
||||
judge pass on **any** substrate. Fresh 7-lane W4 + fresh gpt-4.1-mini judge = **85.19%** on its own
|
||||
archived 2026-06-11 substrate and **86.49%** on the current substrate. The inflation is at the
|
||||
**judgment layer** (the harness's documented stale-verdict-replay bug), not substrate drift. Founder
|
||||
adopted **86.49%** as the canonical, reproducible, still-SOTA number (+4.54pp over Memori 81.95,
|
||||
z=4.64, p<10⁻⁵). Verify: `benchmarks/results/locomo-sota-2026-06/recount.mjs`.
|
||||
|
||||
## The investigation chain (each step verified)
|
||||
|
||||
1. **Evidence not in the repos.** The 87.66 report + raw answers/judgments lived only in the
|
||||
throwaway `hive-mind-test` repo + git-ignored local disk (`.gitignore **/benchmarks/results/*`);
|
||||
OSS showed the old 73.1%. (See `locomo-sota-evidence-drift-2026-06-30.md`.)
|
||||
|
||||
2. **Committed judgments recount low.** `memori-gpt41mini-ours-judgments.jsonl` @ `05f2146` (the
|
||||
report's cited input) recounts to **1313/1540 = 85.26%**, not 1350/87.66. Method validated: it
|
||||
reproduces the Mem0 (73.96%), Config-C (76.62%) and theirs-arm (82.14%) numbers exactly.
|
||||
|
||||
3. **The committed answers were the WRONG config** (founder's "7-lane W4?" catch). That answers file
|
||||
has `raw_detail=0, importance=0, ~3098 tok` — a **reduced 2-lane** run (distilled+semantic), not
|
||||
7-lane W4. Re-judged fresh = **85.39%**. The 87.66 needed the full 7-lane stack (raw-detail≈16,
|
||||
~3700 tok).
|
||||
|
||||
4. **Regenerated the true 7-lane W4** (PROFILES+DATEWIN+EPISODIC+RAWDETAIL, uncapped) on the current
|
||||
substrate → **86.49% (1332/1540)**. Better than 2-lane (+1.1pp), validates the Pareto — but still
|
||||
1.17pp below 87.66.
|
||||
|
||||
5. **Substrate confound ruled out.** The 87.66 (2026-06-11) ran on minds archived as
|
||||
`minds-pre-wave3c` (2026-06-11 00:38); the current minds were rebuilt larger on 2026-06-29. Fresh
|
||||
7-lane W4 on the **archived original substrate** = **85.19%** — *lower* than current. So the gap
|
||||
is NOT substrate drift (newer substrate scores higher); 87.66 doesn't reproduce even on its own
|
||||
substrate.
|
||||
|
||||
6. **Cause = stale-verdict replay.** The harness note (2026-06-15,
|
||||
`RESULT-backlog-closeout`): *"judge resumes by question_id and replayed stale verdicts."* The
|
||||
original 1350-correct judgment pass included replayed/inflated verdicts; it is lost and no fresh
|
||||
judge (85.19 / 86.49) reproduces it. The `41-judge` resume-by-linecount is the mechanism —
|
||||
reusing an OUT_FILE skips fresh judging.
|
||||
|
||||
## Definitive numbers (all fresh gpt-4.1-mini judge, Memori verbatim prompt, N=1540)
|
||||
| Configuration | Overall | vs Memori |
|
||||
|---|--:|--:|
|
||||
| Published claim (2026-06-11) | 87.66% (1350) | +5.71pp — *unreproducible* |
|
||||
| Archived 2026-06-11 substrate, 7-lane W4 | 85.19% (1312) | +3.24pp |
|
||||
| **Current substrate, 7-lane W4 (CANONICAL)** | **86.49% (1332)** | **+4.54pp, z=4.64** |
|
||||
| 2-lane committed file, fresh judge | 85.39% | +3.44pp |
|
||||
|
||||
## Resolution (2026-07-01)
|
||||
- Adopt **86.49%** as canonical. Pinned in `benchmarks/results/locomo-sota-2026-06/`
|
||||
(report + answers + judgments + `recount.mjs`), git-tracked.
|
||||
- Correct 87.66→86.49 across all surfaces (docs, public `apps/www`, `BenchmarkApp`, arXiv draft,
|
||||
OSS, memory index); recompute stats (+5.71→+4.54pp, z=4.42→4.64).
|
||||
- Prevent recurrence: the harness must use a fresh `OUT_TAG` per run; benchmark SOTA evidence must
|
||||
pin substrate+answers+judgments together in-repo. `hive-mind-test` is throwaway.
|
||||
82
docs/analysis/locomo-sota-evidence-drift-2026-06-30.md
Normal file
82
docs/analysis/locomo-sota-evidence-drift-2026-06-30.md
Normal file
@@ -0,0 +1,82 @@
|
||||
# LoCoMo SOTA evidence drift — root cause + consolidation (2026-06-30)
|
||||
|
||||
> **UPDATE 2026-07-01:** consolidating the evidence surfaced a deeper defect — the **87.66% number
|
||||
> itself does not reproduce** (fresh 7-lane W4 = 86.49% current / 85.19% archived substrate; stale-
|
||||
> verdict-replay inflation). Canonical headline is now **86.49%**. This doc's gitignore/side-repo
|
||||
> root cause still stands; the number correction is in
|
||||
> [`locomo-87.66-vs-85.26-integrity-2026-06-30.md`](./locomo-87.66-vs-85.26-integrity-2026-06-30.md).
|
||||
|
||||
**Symptom (founder):** "old benchmark results show, and the SOTA claim is not within waggle-os
|
||||
or hive-mind." The 87.66% LoCoMo memory-SOTA is real and verified, but looking at the canonical
|
||||
repos you see *old* numbers — the reproducible evidence isn't there.
|
||||
|
||||
**Verdict:** Confirmed. This is a recurrence of the §7.5 substrate-drift pattern, but in the
|
||||
**evidence/results** dimension, not the code dimension — with an aggravating `.gitignore` rule
|
||||
that was *silently swallowing* the result report.
|
||||
|
||||
---
|
||||
|
||||
## How it was diagnosed (4-repo evidence matrix)
|
||||
|
||||
| Repo / location | LoCoMo number visible | 87.66 evidence present? |
|
||||
|---|---|---|
|
||||
| waggle-os committed `benchmarks/results/` | April-2026 GEPA + v4–v6 manifests + `agentic-locomo-2026-04-25` | **No** |
|
||||
| waggle-os `docs/` (methodology §0, arXiv `.tex`/`.docx`, WAGGLE-CORNERSTONE) | 87.66 **prose** | Claim only — no reproducible data |
|
||||
| waggle-os on-disk `benchmarks/results/memori-phase22-RESULT.md` | **82.21%** (Phase-2.2 precursor) | No — and **git-ignored**, never committed |
|
||||
| OSS `marolinik/hive-mind` `benchmarks/locomo/RESULTS.md` + README badge | **73.1%** (N=320, Opus self-judge) | **No** |
|
||||
| `hive-mind-test` @ `05f2146` (private side repo) | **87.66%** (N=1540, same-judge) | **Yes — committed** |
|
||||
|
||||
**Substrate code (the engine) is NOT the problem this time.** All four SOTA-critical elements —
|
||||
`inprocess-reranker.ts`, `search.ts` reranker wiring, `resolve-relative-date.ts` /
|
||||
`parse-date-window.ts` (write-time temporal dating), `raw-detail-lane.ts` — are present in
|
||||
waggle-os `main` (`packages/hive-mind-core/src/mind/`) and in the OSS mirror. The reranker was
|
||||
reverse-ported from OSS to monorepo in `f47ee8f` (2026-06-11). (Minor: the OSS reranker sits at
|
||||
an older commit `974ad7b` but is content-equivalent.)
|
||||
|
||||
## Root cause (three compounding failures)
|
||||
|
||||
1. **`.gitignore` swallow.** `.gitignore` line `**/benchmarks/results/*` ignores everything
|
||||
directly under `benchmarks/results/`. The June W3.3 result report was generated there and
|
||||
silently never committed. Older results (`gepa-faza1/…`, `agentic-locomo-2026-04-25`,
|
||||
`manifest-v4/v5`) survive only because they were force-added / committed *before* the rule —
|
||||
so the directory shows a stale snapshot.
|
||||
2. **Evidence produced in a throwaway side repo.** The actual 87.66 run (report + 1,540×2
|
||||
answers + judgments) was produced and committed in `hive-mind-test`, which is a private
|
||||
benchmark working repo — not the product monorepo and not the public OSS repo. It was never
|
||||
forward-ported. This is the §7.5 "benchmark work in a side checkout is throwaway unless
|
||||
reverse-ported" failure mode.
|
||||
3. **OSS public repo never refreshed.** `marolinik/hive-mind` still advertises the earlier
|
||||
73.1% (N=320) result in `RESULTS.md` + README badge; the 87.66 number was never published
|
||||
there even though the winning-stack *code* was (PR #14).
|
||||
|
||||
Net effect: the SOTA *claim* (prose) shipped to the monorepo docs, but its *reproducible
|
||||
evidence* lived only on local disk (git-ignored) + a side repo. A fresh clone of either
|
||||
canonical repo shows old numbers — exactly the founder's report.
|
||||
|
||||
> Red herring: the founder pointed at `D:/Projects/waggle-os-w4` (branch `feature/w4-port`).
|
||||
> That worktree is **226 commits behind `main`** and only 3 doc/lint commits ahead, all of whose
|
||||
> content was already re-ported to `main` (paper via `4193e68a`). It holds nothing `main` lacks.
|
||||
|
||||
## The fix (this change)
|
||||
|
||||
**A. Commit the canonical evidence into the product monorepo (done here).**
|
||||
`benchmarks/results/locomo-sota-2026-06/` now holds the two canonical reports (verbatim, with
|
||||
provenance headers), an `INDEX.md` (the previously-untracked SOTA single-source-of-truth), and a
|
||||
`README.md` reproduction recipe. The raw ~3.8 MB answers/judgments are intentionally **not**
|
||||
duplicated into the lean product repo — they're pointered to `hive-mind-test` + OSS.
|
||||
|
||||
**B. Stop the silent swallow (structural drift-closure).** A `.gitignore` negation exception
|
||||
re-includes `benchmarks/results/locomo-sota-2026-06/**` so this evidence stays committed and
|
||||
future canonical SOTA evidence has a non-ignored home.
|
||||
|
||||
**C. OSS public update — prepared, founder-gated.** Updating `marolinik/hive-mind`
|
||||
`benchmarks/locomo/RESULTS.md` + README badge from 73.1% → 87.66% is **outward-facing** (it
|
||||
pre-announces the SOTA ahead of arXiv submission, whose citation pass + endorsement are still
|
||||
open). Left as a go/no-go for the founder rather than pushed unilaterally.
|
||||
|
||||
## Prevent recurrence (recommended follow-ups)
|
||||
- Add a one-line contract to `CLAUDE.md` §7.5 / `packages/hive-mind-core/CONTRIBUTING.md`:
|
||||
*"Benchmark SOTA evidence MUST land in `benchmarks/results/locomo-sota-*/` (gitignore-excepted)
|
||||
in the monorepo. `hive-mind-test` is throwaway — forward-port the report the same arc."*
|
||||
- Optionally extend `scripts/oss-drift-check.sh` to assert `RESULTS.md` headline parity between
|
||||
the monorepo evidence dir and the OSS `benchmarks/locomo/`.
|
||||
226
docs/analysis/loop-engineering-waggle-analysis-2026-06-29.md
Normal file
226
docs/analysis/loop-engineering-waggle-analysis-2026-06-29.md
Normal file
@@ -0,0 +1,226 @@
|
||||
# Loop Engineering as a Cron/Loop Layer for Waggle OS
|
||||
|
||||
### Can Cobus Greyling's loop-engineering be the scheduling/loop layer for Waggle — a knowledge-worker platform, not a coding tool?
|
||||
|
||||
**Status:** Lead analyst synthesis of four lenses (Capability Census · KW Pattern Translation · Cron-Layer Design · Strategic Fit) + direct re-verification of the load-bearing file:line claims.
|
||||
**Date:** 2026-06-29 · **Verdict confidence:** high (key claims verified against source, not CLAUDE.md).
|
||||
|
||||
---
|
||||
|
||||
## 1. Executive answer
|
||||
|
||||
**Yes — and it is closer to shipped than the framing implies.** Loop-engineering is not a new engine for Waggle; it is **one new `job_type:'loop'` case** that composes pieces Waggle already built for other reasons. The schedule spine (`CronStore` + `LocalScheduler`), run-logs (`cron_execution_history`), the maker/checker fan-out (`subagent-orchestrator.ts` + `judge.ts`), the autonomy gate (`confirmation.ts`), loop-bounding (`loop-guard.ts`), and the durable state substrate (the per-workspace `.mind`) are all in-tree and tested. What is genuinely missing is the **composition glue**, plus four small primitives. The coding-specific half of Cobus's framework — git worktrees, PR babysitting, CI sweeping — correctly does **not** translate, and its knowledge-work substitute (per-workspace mind isolation) already exists.
|
||||
|
||||
**The inversion thesis (the real differentiator):** Cobus treats **Memory/State as a bolt-on** — "a durable spine outside any conversation" attached to a loop whose true state is the git repo it reads each tick. Waggle inverts this: the per-workspace `.mind` (HybridSearch + KnowledgeGraph + IdentityLayer + AwarenessLayer) **is the product**, and three of the seven shipped cron job-types (`memory_consolidation`, `proactive`, `connector_fetch`) exist only to feed or mine it. A coding loop re-greps a log to remember tick N-1; a Waggle loop recalls it by *meaning*. Loops are the first feature that makes **scheduled work accumulate in the moat**.
|
||||
|
||||
**The honest boundary on that thesis:** memory is a genuine edge for the **recall/synthesis half** of KW loops (triage, brief, digest, "what changed since I last looked") and **neutral-to-negative for the pure-action half** (send the email, update the record), where the authoritative state is the external system (CRM, inbox) and dragging it through a memory substrate adds latency, dedup cost, and a staleness/poisoning surface. Sell the differentiator where it is true.
|
||||
|
||||
---
|
||||
|
||||
## 2. What loop-engineering is (sourced)
|
||||
|
||||
Cobus Greyling, `github.com/cobusgreyling/loop-engineering` (~3.9k stars, Jun 2026): *"Loop engineering is replacing yourself as the person who prompts the agent. You design the system that does it instead."* A **harness** equips one agent run; a **loop** keeps poking agents on a schedule, spawns helpers, verifies, persists state, decides the next action.
|
||||
|
||||
**Five building blocks + memory:** (1) Automations/Scheduling, (2) Worktrees (git isolation — coding-specific), (3) Skills (persistent project knowledge), (4) Plugins & Connectors (MCP), (5) Sub-agents (maker/checker), + Memory/State (the durable spine, treated as a bolt-on).
|
||||
|
||||
**Loop anatomy (10 steps):** schedule -> triage skill -> state read/write -> isolated worktree -> implementer sub-agent -> verifier sub-agent -> MCP/git/tickets -> human gate -> commit/PR/action -> loop back.
|
||||
|
||||
**Seven production patterns (all coding-centric):** Daily Triage, PR Babysitter, CI Sweeper, Dependency Sweeper, Changelog Drafter, Post-Merge Cleanup, Issue Triage.
|
||||
|
||||
**Operating concepts:** autonomy tiers **L1 Report / L2 Assisted / L3 Unattended**; **intent debt** (unarticulated goals piling up in loop prompts); **comprehension debt** (gap between what the loop ships and what humans understand — read-before-ship); denylist & auto-merge gates; MCP scopes; multi-loop coordination; cost-per-cadence; run-logs; CLI tools (`loop-init`, `loop-audit`, `loop-cost`).
|
||||
|
||||
---
|
||||
|
||||
## 3. What Waggle ALREADY has — verified capability census
|
||||
|
||||
Statuses corrected against source. **EXISTS** = shipped and usable. **PARTIAL** = built but not wired into the scheduled path. **ABSENT** = not present. **N/A** = coding-only, does not translate.
|
||||
|
||||
| # | Capability | LE term | Status | Evidence (verified file:line) | Gap |
|
||||
|---|---|---|---|---|---|
|
||||
| 1 | Schedule + triage on a cadence | Automations/Scheduling | **EXISTS** | `cron-store.ts:15` (`CronJobType` 7-member union), `:72-86` table, `:272` `getDue()`, `:279` `markRun()`; `cron.ts:126` 60s `tick()`; `automations.ts:149-336` user API | None for L1 |
|
||||
| 2 | Parallel execution isolation | Worktrees | **N/A (coding-only)** | KW analog = per-workspace `.mind`: `activateWorkspaceMindWithWeaver` (`index.ts:1867`) | Git worktrees do not translate; the analog already exists |
|
||||
| 3 | Persistent project knowledge / reusable modules | Skills | **PARTIAL** | `skill-audit.ts`, `skill-creator.ts`, `persona-data.ts` | Validated but not bound as durable scheduled config |
|
||||
| 4 | MCP integration | Connectors/MCP | **PARTIAL** | `mcp-catalog.ts`, `connectors/` (30 connectors), `permissions.ts` (whitelist/blacklist) | No per-MCP scope parameter |
|
||||
| 5 | Sub-agents (maker/checker) | Sub-agents | **PARTIAL (built, unwired)** | `subagent-orchestrator.ts:97` topological sort + `'reviewer'` preset; `judge.ts:76` rubric scoring | Exists in `packages/agent`; **not wired into the cron executor** (executor does a single chat call) |
|
||||
| 6 | Durable state/memory outside conversation | Memory/State | **EXISTS** | `hive-mind-core/src/mind/{frames,knowledge,identity,awareness}.ts`; `awareness.ts:91` `getByStatus('pending')`; `frames.ts:73` `createIFrame` | The spine — see §4 |
|
||||
| 7 | 10-step loop anatomy end-to-end | Loop Anatomy | **PARTIAL** | Steps 1-3,5,8-10 exist individually; **the verifier persona is FULLY built** (`judge.ts`) but unwired to cron; step 4 (worktree) is N/A | No integrated 10-step loop architecture; it is components, not a system |
|
||||
| 8 | Autonomy tiers | Autonomy L1/L2/L3 | **EXISTS (terminology differs)** | `confirmation.ts:202` `AutonomyLevel = 'normal'\|'trusted'\|'yolo'`; `:237` `isCriticalNeverAutopass`; `:271` `needsConfirmationWithAutonomy`; schema levels `manual/guided/medium/high` | Gate is **per-tool-call**, not **per-loop**; the named tiers map to L1/L2/L3 but aren't bound to a loop config |
|
||||
| 9 | Denylist / never-autopass | Denylist gates | **PARTIAL (corrected from ABSENT)** | Binary denylist `DENIED_BINARIES` (`system-tools-helpers.ts:7`, applied `system-tools.ts:1008`); `CRITICAL_NEVER_AUTOPASS` regex set (`confirmation.ts:220`) | Tool-level denylist exists; **no per-loop denylist config, no auto-merge gate, not bound to the headless path** |
|
||||
| 10 | Per-action MCP scope limiting | MCP Scopes | **ABSENT** | No `scope` field on `McpServer` | Net-new |
|
||||
| 11 | Intent debt tracking | Intent Debt | **ABSENT** | — | Concept/UX guardrail, not code |
|
||||
| 12 | Comprehension debt (read-before-ship) | Comprehension Debt | **ABSENT** | Raw material exists (`cron_execution_history.result_summary`, `formatTrustSummary`) | No plain-language "what this loop did/proposed" digest — the #1 non-technical-user risk |
|
||||
| 13 | Cost-per-cadence estimation | Cost-per-Cadence | **PARTIAL** | `cost-tracker.ts:55` soft/hard budget; **`getDailyTotal` is a per-session in-memory proxy** (`:135`) | Not persisted, not wired to cron; cannot bound a 24/7 loop |
|
||||
| 14 | Run-logs (per-tick record) | Run-logs | **EXISTS** | `cron-store.ts:89-102` `cron_execution_history` (`duration_ms/success/result_summary/error`); `getExecutionHistory()`; `GET /api/automations/:id/logs` (`automations.ts:312`); retention `pruneExecutionHistory(30)` (`:320`) | Surface, don't rebuild |
|
||||
| 15 | Loop bounding / infinite-loop detection | Loop Bounding | **EXISTS** | `loop-guard.ts`, `iteration-budget.ts`; `cron.ts:157` auto-disable after 5 consecutive failures | None |
|
||||
| 16 | Multi-loop coordination | Multi-loop Coordination | **ABSENT** | Only single-flight `this.ticking` guard (`cron.ts:127`) + 5-fail disable | No cross-loop conflict resolution — TEAMS-tier, later |
|
||||
| 17 | `loop-init`/`loop-audit`/`loop-cost` CLI | Loop CLI | **ABSENT** | `cli-tools.ts` is generic CLI discovery | Deliver as builder UI, not a developer CLI |
|
||||
|
||||
**Census bottom line:** ~75% of the loop-engineering primitive set is present (8 EXISTS, 4 PARTIAL, 1 N/A); the 4 true ABSENTs plus the cost-per-cadence gap are what a "Loop" layer must add. The coding-only block (worktrees) is correctly absent with a working analog.
|
||||
|
||||
---
|
||||
|
||||
## 4. The cron-layer opportunity: a "Loop" abstraction on top of CronStore
|
||||
|
||||
**A Loop is a `cron_schedules` row with one new `job_type:'loop'` and a structured `job_config`.** No new table, no new route, no new UI — it reuses the exact blob-on-`job_config` trick `automations.ts` already relies on, and `AutomationCenterApp` already renders any cron row + its history/logs/running tabs.
|
||||
|
||||
### Data model (everything beyond `CronSchedule` lives in `job_config`)
|
||||
```
|
||||
job_config (loop):
|
||||
goal: string // the recursive purpose ("keep my pipeline triaged")
|
||||
makerPrompt: string // the implementer sub-agent task (the "triage skill")
|
||||
checkerRubric?: string // verifier rubric -> judge.ts; omit = no checker
|
||||
autonomyTier: 'L1'|'L2'|'L3' // report / assisted / unattended
|
||||
stateKey: string // awareness namespace tag for cross-tick state
|
||||
budget?: { maxTicks?, maxSubagents?, maxTokens? }
|
||||
denylist?: string[] // L3 tool/connector denylist (binds to isCriticalNeverAutopass)
|
||||
notify: boolean
|
||||
```
|
||||
|
||||
### Execution sequence (the new `case 'loop':` in the `index.ts:1427` switch — the only new wiring)
|
||||
1. **Schedule fires** -> `LocalScheduler.tick` (exists, `cron.ts:126`).
|
||||
2. **Isolate** -> `activateWorkspaceMindWithWeaver(workspace_id)` (exists, `index.ts:1867`). *This is Waggle's worktree.*
|
||||
3. **Read prior state** -> `AwarenessLayer.getByStatus('pending')` (`awareness.ts:91`) + HybridSearch over frames tagged `stateKey`. **This is the step cron does not do today.**
|
||||
4. **Maker** -> `SubagentOrchestrator.runWorkflow` (`subagent-orchestrator.ts:97`), step `implement`.
|
||||
5. **Checker** -> a `verify` step (`dependsOn:['implement']`) scored by `judge.ts:76` -> pass/fail gate.
|
||||
6. **Human gate by tier:** L1 = `emitNotification` only (zero write side-effects); L2 = write a `pending` approval item; L3 = execute with `denylist` enforced by `isCriticalNeverAutopass` (`confirmation.ts:237`) as the hard floor.
|
||||
7. **Write next-tick state** -> `awareness.add/updateMetadata` + `FrameStore.createIFrame` (`frames.ts:73`) so tick N+1 sees what tick N did.
|
||||
8. **Record + loop back** -> `onJobComplete` -> `cron_execution_history` (exists), `markRun` recomputes `next_run_at`.
|
||||
|
||||
### Reuse vs build
|
||||
|
||||
| Loop step | Already there (cite) | Build new |
|
||||
|---|---|---|
|
||||
| schedule/tick | `LocalScheduler` `cron.ts:126` | — |
|
||||
| triage skill | persona/prompt in `job_config` | — |
|
||||
| **state read/write** | `awareness.ts:91` / `frames.ts:73` / HybridSearch | **wire it (cron ignores it today)** |
|
||||
| isolation | `activateWorkspaceMindWithWeaver` `index.ts:1867` | — |
|
||||
| maker subagent | `subagent-orchestrator.ts:97` | — |
|
||||
| checker subagent | `'reviewer'` preset + `judge.ts:76` | **glue verdict -> gate** |
|
||||
| connector action | `connector_fetch` `index.ts:1914`; `mcp/` + `tool-filter` | — |
|
||||
| human gate / autonomy | `needsConfirmationWithAutonomy` `confirmation.ts:271` | **map L1/L2/L3 -> levels + headless queue** |
|
||||
| loop bounding | `loop-guard.ts`, `iteration-budget.ts` | — |
|
||||
| cost-per-cadence | `cost-tracker.ts` | persist + pre-activation estimate |
|
||||
|
||||
**Net new code = one executor case (~150-250 LOC) + tests.** The maker/checker fan-out is ~90% there; scheduling/run-logs/auto-disable are 100% there; the gate is 100% there. The composition is the work.
|
||||
|
||||
### Why memory is the spine (the differentiator, grounded)
|
||||
Cobus's loops are stateful **because git is the state** — worktrees, branches, the diff a PR babysitter reads to know what it already touched. Knowledge work has no git, which is exactly why naive "agent on a cron" loops re-do and re-report the same thing every tick. Waggle already shipped the substitute:
|
||||
- **`AwarenessLayer`** (`awareness.ts:27`) is a typed, expiring, priority-ordered scratchpad with categories `task|action|pending|flag` and `{status,result}` metadata. `getByStatus('pending')` (`:91`) is literally "what did I leave open last tick" — the loop's working register.
|
||||
- **`FrameStore.createIFrame`** (`frames.ts:73`) + the harvest **dedup pipeline** mean tick N+1 recalls tick N's frames and doesn't re-emit them. "3 new at-risk deals" on day two means 3 *new* ones, because the prior 5 are already frames.
|
||||
- The contrast that sells it: Cobus's PR Babysitter remembers which PRs it nudged via git/ticket state. Waggle's pipeline loop remembers which leads it already drafted outreach for, and the contract loop remembers which clauses it already flagged — stored as awareness items + frames, deduped nightly by `memory_compact`/`memory_lane_extract` (`setup-crons.ts`). `cron_execution_history` can tell you a tick *ran*; the mind tells you what the tick *knows*. **L1 loops should be free precisely because they generate this memory.**
|
||||
|
||||
---
|
||||
|
||||
## 5. Knowledge-worker Loop catalog
|
||||
|
||||
### 5a. The 7 production patterns -> KW analogs
|
||||
|
||||
| # | Coding pattern | KW Loop | Cadence | CronJobType | Persona | Writes to memory (why it compounds) | Tier / gate | Translate? |
|
||||
|---|---|---|---|---|---|---|---|---|
|
||||
| 1 | Daily Triage | **Daily Desk Brief** — "what needs me today" across inbox/calendar/CRM | 1x/day | `proactive` (reuse `morning_briefing`) + `agent_task` enrich | executive-assistant | Brief frame; flags items into `awareness(category='task')` so they compound day-over-day | L1, free | ✅ clean |
|
||||
| 2 | PR Babysitter | **Awaiting-Reply Babysitter** — watch threads/deals you're waiting on; draft nudge when stale | 4x/day (polled) | `agent_task` | sales-rep/support-agent | Thread-state frame (who owes whom, last-touch) reused next tick | **L2, PRO** | ✅ but most blocked: wants event trigger + approval queue |
|
||||
| 3 | CI Sweeper | **Integration-Health Sweeper** — did nightly reports run? connectors still authed? | 15 min | `workspace_health` (reuse) | ops-manager | Health-status frame + alert | L1 | ⚠️ **mostly coding-only**; only the plumbing-health residue translates, strictly L1, no autonomous remediation |
|
||||
| 4 | Dependency Sweeper | **Doc/Policy Freshness Sweeper** — flag SOPs/contracts past review-by; propose patch-only edits | weekly | `agent_task` | legal/hr/ops | Freshness-ledger frame per doc | **L2 patch-only, PRO** | ⚠️ translates by metaphor (docs-as-dependencies) |
|
||||
| 5 | Changelog Drafter | **Weekly Wins / Status Digest** — "what shipped/closed/moved" from tasks+CRM+memory | weekly | `agent_task` (kin to `monthly_assessment`) | project-manager/PM | Digest frame -> compounds into the wiki (`compile_wiki`) | L2 | ✅ clean |
|
||||
| 6 | Post-Merge Cleanup | **Deal/Project Close-Out** — extract lessons-learned, archive, advance CRM stage | poll 1x/day | `agent_task` | project-manager/sales-rep | **Lessons-learned/post-mortem frame** (highest-compounding write) | L2->L3 | ⚠️ git mechanics N/A; the "hygiene-after-completion" pattern is high value |
|
||||
| 7 | Issue Triage | **Inbound Triage** — classify/route email, tickets, leads, NDAs, applicants | 2h | `agent_task` | support/recruiter/legal/sales | Triage-label + routing-decision frames; new KG entity per item | **L2 propose-only, PRO** | ✅ strong; matching skills exist (`customer-support:ticket-triage`, `legal:triage-nda`) |
|
||||
|
||||
**Coding-only SKIPs:** CI Sweeper's autonomous build-fix and the git/branch mechanics of Worktrees + Post-Merge Cleanup. Dependency Sweeper translates only by metaphor. The other four map cleanly.
|
||||
|
||||
### 5b. Net-new KW loops (no coding analog — these exist *because* memory + connectors are the platform)
|
||||
|
||||
| Loop | Purpose | Cadence | CronJobType | Why it compounds | Tier | New primitive? |
|
||||
|---|---|---|---|---|---|---|
|
||||
| **Inbox-to-memory harvest** | Pull gmail/slack/notion into the mind so context compounds passively | daily | `connector_fetch` (**already shipped**, `index.ts:1914`) | Raw frames in personal mind — the moat in motion | PRO, L3 read-only | **None — ships today** |
|
||||
| **Relationship-decay sweep** | "Who have I gone quiet on?" ranked by importance | weekly | `agent_task` | Per-contact cadence frame; decay model sharpens over weeks | L1 / L2 draft | Reuses cron; L2 send needs approval queue |
|
||||
| **Commitment tracker** | Scan sent mail + meeting notes for promises ("I'll send X by Fri") | daily | `agent_task` | Commitment entities in KG — impossible without the substrate | L1 | Benefits from checker (prune false promises) |
|
||||
| **Renewal / expiry radar** | Contracts/licenses/subs expiring in N days -> alert + prep pack | weekly | `agent_task` | Renewal-calendar frames reconciled each tick | L2 | L2 prep approval |
|
||||
| **Meeting-prep brief** | T-30 before each meeting: attendees, last threads, open items, CRM | per-meeting | `agent_task` | Brief frame keyed to attendees -> next meeting starts warm | L1 | **Calendar-derived scheduling** — NEW |
|
||||
| **Knowledge-gap / wiki-compile** | Periodically compile personal wiki + health-check -> surface gaps | weekly | `memory_consolidation` (reuse) | The mind audits itself | L1 | **None — reuses action convention** |
|
||||
| **Competitor / market-watch digest** | Weekly pull on tracked competitors -> "what changed" delta | weekly | `agent_task` | Per-competitor frame diffed vs last week | L1 | Reuses cron + web-fetch |
|
||||
| **Expense/invoice anomaly digest** | Weekly scan for anomalies + overdue | weekly | `agent_task` | Anomaly frames + learned baseline | L1 | Reuses cron |
|
||||
|
||||
**~60% of this catalog ships today on the existing cron substrate with zero new code** (persona + prompt + connectors + a `cron_expr`), at L1 or via the notification+`actionUrl` soft-L2 workaround.
|
||||
|
||||
---
|
||||
|
||||
## 6. Adopt / Build / Skip — concepts
|
||||
|
||||
| Concept | Verdict | Grounded why |
|
||||
|---|---|---|
|
||||
| Automations/Scheduling | **ADOPT (already built — reframe)** | `CronStore` + `LocalScheduler` + `/api/automations` shipped; work is vocabulary + UX |
|
||||
| Worktrees (git) | **SKIP** | Coding-only; KW analog = per-workspace `.mind` boundary (`index.ts:1867`) |
|
||||
| Skills | **ADOPT (already built)** | Skills + custom-personas + the §D2 verify loop; loops *are* scheduled skills |
|
||||
| Plugins/Connectors (MCP) | **ADOPT (already built)** | `mcp-catalog.ts` + `connectors/` + `connector_fetch` |
|
||||
| Sub-agents (maker/checker) | **ADOPT — wire to cron** | `subagent-orchestrator.ts:97` + `judge.ts` exist; the shipped `skill-audit.ts` (synth->run->judge->rewrite->badge) is a maker/checker loop for a KW artifact |
|
||||
| L1/L2/L3 autonomy | **ADOPT-AS-CONCEPT (relabel + per-loop bind)** | `confirmation.ts:202` `normal/trusted/yolo` = L1/L2/L3 but **per-tool-call**; build a thin per-loop autonomy field that maps onto it |
|
||||
| Denylist / never-autopass | **ADOPT (already built — bind to headless)** | `CRITICAL_NEVER_AUTOPASS` (`confirmation.ts:220`) + `DENIED_BINARIES` (`system-tools-helpers.ts:7`); bind to **DENY** in headless, not auto-pass |
|
||||
| Intent debt | **ADOPT-AS-CONCEPT** | UX guardrail: force a one-line goal + success criterion per loop. No code |
|
||||
| Comprehension debt | **BUILD (small)** | Raw material exists (`cron_execution_history.result_summary` + `formatTrustSummary`); assemble a plain-language "what this loop did/proposed" digest |
|
||||
| Cost-per-cadence | **BUILD** | `cost-tracker.ts:55` budgets exist but `getDailyTotal` (`:135`) is a per-session proxy; persist a per-loop budget + pre-activation $/day estimate |
|
||||
| Run-logs | **SKIP / DONE** | `cron_execution_history` + `pruneExecutionHistory(30)` already are run-logs; surface them |
|
||||
| Multi-loop coordination | **BUILD (later, TEAMS)** | Today: single-process guard + 5-fail disable (`cron.ts:127,157`); real conflict resolution is TEAMS-tier |
|
||||
| `loop-audit`/`loop-cost` | **ADOPT-AS-CONCEPT, BUILD as UI panel** | Pre-activation readiness/cost in the builder, not a developer CLI |
|
||||
| `loop-init` scaffold | **SKIP** | KW users don't scaffold YAML; the persona + template picker is the scaffold |
|
||||
| 7 production patterns | **SKIP as-is, translate** | All coding-centric; replace with the KW templates in §5a |
|
||||
|
||||
### Autonomy-tier proposal mapped to Waggle's trust-model
|
||||
- **L1 Report** = `emitNotification` only, zero write side-effects (today's scheduled `agent_task` is L1 *by construction* — it is toolless, `index.ts:1869-1885`). Default for v0.
|
||||
- **L2 Assisted** = maker drafts -> checker gates -> writes a `pending` approval item surfaced in the existing confirmation/notification UI; the human one-click approves. **Needs the new `pending_actions` store.**
|
||||
- **L3 Unattended** = executes with `denylist` enforced by `isCriticalNeverAutopass` as the hard floor; never inherits interactive auto-approve. Deferred past v0/v1 for write-capable KW workspaces.
|
||||
|
||||
---
|
||||
|
||||
## 7. Monetization & tier placement
|
||||
|
||||
**Split by what the loop *touches*, reusing the gate the code already made** — `connector_fetch` is **already PRO-gated** (`assertTierCapability(tier,'PRO')`, `index.ts:1919`). Lean on that precedent; do **not** invent a standalone "Loops" SKU (that would tax the moat-builder).
|
||||
|
||||
- **FREE — memory-directed loops (drives the moat).** Loops whose only side effect is writing to the mind: `memory_consolidation`, `proactive` recall, a capped daily `agent_task` digest. `spawnAgents` is already FREE (`tiers.ts`), and scheduled agents are agents on a clock. Cap by **cadence + count**, not by feature (e.g. FREE = up to 3 automations, daily-or-slower, no external write). Maximizes frames written = maximizes moat.
|
||||
- **PRO ($19) — connector-fed / connector-acting + verify loops (the upgrade trigger).** Anything reading a connector into a loop or acting through one, the maker/checker verify loop (already PRO-gated), custom-skill loops (`customSkills` starts at PRO), higher cadence (sub-hourly), and higher per-loop token budget. This is exactly "skills/connectors are the upgrade trigger."
|
||||
- **TEAMS ($49/seat) — shared, governed, multi-loop.** Shared-workspace loops, multi-loop conflict resolution, and full run-log audit (`auditLog:'full'`, `teamSkillLibrary` — both TEAMS-only). A team running 20 loops against a shared CRM needs coordination + audit; that is the governance value KVARK sells up-market.
|
||||
|
||||
Defensible because it keeps the moat (free memory loops compound the substrate), doesn't invent a new paywall (loops fall through the existing connector/skill/audit gates), and the cadence/cost cap is the natural "more, faster, acting" upgrade reason.
|
||||
|
||||
---
|
||||
|
||||
## 8. Risks & mitigations
|
||||
|
||||
**A. Unattended-action footgun — latent today, one wire from opening.** Scheduled `agent_task` is currently toolless — a plain `/v1/chat/completions` call that generates text and notifies (`index.ts:1869-1885`). So every scheduled loop is **L1 by construction** and *cannot* send an email. The danger is the obvious next feature: wiring the full agent loop (with tools) into the scheduler. The moment that happens, **`ConfirmationGate.confirm` returns `true` (auto-approve) when there is no `promptFn`** (verified `confirmation.ts:313`) — a headless tick would silently auto-approve `send_email` (otherwise always-critical). **Mitigation (hard requirement before any tool-enabled scheduled loop):** headless runs default L1; any gated action routes to the notification/approval queue (the `notifications` table already exists, `cron-store.ts:105`) as an async human gate; bind `isCriticalNeverAutopass` to **DENY** in headless, never auto-pass.
|
||||
|
||||
**B. Token-cost blowup on cadence — under-defended.** `CostTracker` has soft/hard daily budgets (`cost-tracker.ts:55`) but **`getDailyTotal()` is an in-memory per-session proxy** (`:135`) — it does not survive restarts and does not bound a per-minute loop. The only real defenses in-tree are the **20-hour frequency floor** (`index.ts:1945`) and the **5-consecutive-failure auto-disable** (`cron.ts:157`). **Mitigation:** generalize the frequency floor to all loop types, persist a per-loop daily budget with a hard cap (reuse `BudgetExceededError`), and show a pre-activation $/day estimate in the builder. Without this, a PRO user setting a 5-min triage loop on Opus is a surprise invoice.
|
||||
|
||||
**C. Comprehension debt — the sharpest KW-specific risk, least mitigated.** Cobus's "read-before-ship" assumes a developer reading a diff. Waggle's user is a salesperson who will not read a JSON run-log. The substrate exists (`cron_execution_history` per-tick rows, `result_summary`, `formatTrustSummary`'s plain-language prose) but is not assembled into a human story. **Mitigation:** default loops to **L2 "propose, don't act"** with a plain-language digest ("This automation drafted 3 follow-up emails and updated 2 deal stages — review?"). For non-technical users, comprehension debt is repaid by **propose-with-summary**, not better logs.
|
||||
|
||||
**D. Stale/poisoned memory feeding an acting loop.** A loop that recalls a poisoned frame then acts is the worst case. `connector_fetch` already injection-scans inbound frames and `skill-audit.ts` fences skill content as untrusted; that discipline must extend to *every* loop crossing recall->action. **Mitigation:** run recalled context through `scanForInjection` before it can reach a write tool.
|
||||
|
||||
**E. Scaling caveat (disclose, don't over-engineer for v0).** `LocalScheduler.tick` runs due jobs **sequentially, awaited in one process**, under a single-flight guard (`cron.ts:126-164`). A loop spawning maker+checker takes minutes; while it runs the whole tick is blocked and other due jobs wait. Fine for a handful of solo-desktop loops; it is not a fleet scheduler. Flag it; a job queue is a later concern.
|
||||
|
||||
---
|
||||
|
||||
## 9. Smallest shippable slice — "Loop v0"
|
||||
|
||||
**Loop v0 = "make `agent_task` stateful and verified," shipped as `job_type:'loop'`, L1 only.**
|
||||
|
||||
Scope, minimal:
|
||||
- **One new `case 'loop':`** in the `index.ts:1427` switch. Reads the `job_config` spec, activates the workspace mind, reads prior state from awareness + recall, runs a 2-step `SubagentOrchestrator` workflow (maker -> reviewer with `dependsOn`/`contextFrom`), scores the reviewer output with `judge.ts`, **emits a report notification** (L1: observe, zero writes), then **writes the result back** as frames + an awareness item tagged `stateKey`. `onJobComplete` already records the run-log.
|
||||
- **Builder/UI: none.** `AutomationBuilder` already POSTs an arbitrary `job_config`; `AutomationCenterApp` already renders the row + Running/History/Logs + Run-now. A 1-line "Loop" label is the only optional FE touch.
|
||||
- **Tests:** executor unit test + one e2e through `/api/automations` -> tick -> history. Both harnesses exist (`automations.test.ts`, `local-scheduler.test.ts`).
|
||||
|
||||
**Honest build cost: ~1-2 engineer-days.** Small *because* maker/checker, scheduling, run-logs, isolation, the gate, loop-guard, and the memory API are all already in-tree and tested. The risk is not code volume; it is the autonomy-ceiling and tier-gate decisions (§10), plus the §8E sequential-tick caveat.
|
||||
|
||||
**Deliberately deferred from v0:** L2 approval queue (next arc), L3 unattended writes, event/calendar triggers, multi-loop coordination, per-MCP scopes.
|
||||
|
||||
---
|
||||
|
||||
## 10. Open decisions for the founder
|
||||
|
||||
1. **Autonomy ceiling for v0** — L1-only (report + notify, safe-by-default, demoable) or also L2 (assisted: writes a `pending` approval surfaced in the existing confirmation UI)? *Recommendation: L1 only for v0, L2 next arc, L3 deferred as its own trust arc.*
|
||||
2. **Tier placement** — L1 memory-building loops FREE (aligned to the moat) with L2/L3 PRO, or "Loops" itself an upgrade trigger? *The gating primitive (`assertTierCapability`) is already wired; this is the one pricing call blocking a build.*
|
||||
3. **Vocabulary** — keep user-facing "Automations" (already shipped) or rebrand to "Loops" (borrows Cobus's mindshare)?
|
||||
4. **Event/calendar triggers** — lift the schedule-only restriction now or defer? *Only Babysitter (react-on-reply) and Meeting-prep (T-30) truly need it; everything else polls fine. Recommendation: defer.*
|
||||
5. **Scaling posture** — accept the sequential single-process tick for v0 and revisit a job queue only if TEAMS multi-loop demand materializes?
|
||||
|
||||
---
|
||||
|
||||
### Key files cited
|
||||
`packages/core/src/cron-store.ts` (store + run-logs) · `packages/server/src/local/cron.ts` (tick runner) · `packages/server/src/local/index.ts:1426-1957` (executor switch; `agent_task` one-shot at :1830; `connector_fetch` PRO-gated at :1914) · `packages/server/src/local/routes/automations.ts` (alias seam) · `packages/agent/src/subagent-orchestrator.ts:97` (maker/checker) · `packages/agent/src/judge.ts:76` (checker rubric) · `packages/agent/src/confirmation.ts:202,220,237,271,313` (autonomy gate + the headless footgun) · `packages/agent/src/system-tools-helpers.ts:7` (`DENIED_BINARIES`) · `packages/agent/src/cost-tracker.ts:55,135` (budget + per-session proxy) · `packages/agent/src/skill-audit.ts` (shipped maker/checker loop) · `packages/hive-mind-core/src/mind/{awareness.ts:91,frames.ts:73}` (state spine) · `packages/shared/src/tiers.ts` (tier gates).
|
||||
247
docs/analysis/odysseus-adoption-2026-06-28.md
Normal file
247
docs/analysis/odysseus-adoption-2026-06-28.md
Normal file
@@ -0,0 +1,247 @@
|
||||
# Odysseus → Waggle OS: Prioritized Adoption Report (Hardened + Deep-Trace Reconciled, Final)
|
||||
|
||||
> **License note (AGPL-3.0 — stated once, binding):** Odysseus is **AGPL-3.0**. We may read it for ideas and **port concepts clean-room** (math, control-flow shape, taxonomy, *edge-case knowledge*), authored fresh against Waggle's own files. We may **not** copy its code into Waggle's proprietary monorepo, **and we may not bundle or ship its binaries** (e.g. its `hwfit`/`llmfit` tool) alongside the closed product — distributing an AGPL binary with proprietary code is the worst-case AGPL trap. Where a spec below "lights up" an existing Waggle route that shells out to a tool, the implementation is a **clean-room TypeScript re-implementation of the algorithm**, never odysseus's binary. The §B hardware-detection port in particular is a port of the *edge-case checklist (knowledge)*, never the `hardware.py` code. No paste, no binary.
|
||||
|
||||
## 1. Framing & Honest Verdict
|
||||
|
||||
Odysseus is a mature self-hosted, multi-user, admin-console-grade workspace: a real (pre-SOTA) memory substrate, a full local-inference stack, a skill-lifecycle machine, a hardened scheduler, and a consumer email/calendar client. The founder's bar is unchanged and high — an item ADOPTs only if it moves one of four Waggle levers: **(1) KVARK funnel / sovereign narrative, (2) the memory+harvest MOAT, (3) the skills/connectors UPGRADE TRIGGER, (4) Waggle-funded PROXY COST.** Generic "good feature" = SKIP, the way Local-Agent-Studio was rejected wholesale.
|
||||
|
||||
**The decisive read: A/B/C already took the proxy-cost-margin lever this session.** Tool-output compression (A, `tool-output-compressor.ts` — subtractive, post-injection-scan, verified), Haiku-on-proxy routing (B, `model-class-router.ts` — `privacyRequired` fails closed, verified), and PRO-gated connector auto-fetch (C, `connector-harvest.ts` — injection-scanned, hash-skipped, frequency-floored, verified). So odysseus's *margin-side* cost items (tool-RAG schema slimming, compact-prompt mode, low-signal bypass, singleflight cache) are **the same lever A/B already took, at lower marginal value** — honest DEFER, not ADOPT.
|
||||
|
||||
**Odysseus's real value sits on the three levers A/B/C barely touched** (only C touched the moat). Four clusters clear the bar, scope-cut to their zero-regret core; one cheap fold; everything else defers or skips.
|
||||
|
||||
- **Proxy-cost via a *different mechanism* than A/B — the local-model on-ramp.** Waggle shipped the local-inference route + UI (`local-inference.ts`: `/hardware` `/models` `/status` `/pull`), but the ranking engine is delegated to an **absent `llmfit` binary** (verified: `detectHardwareViaLlmfit` shells out and returns `null`, falling to `detectHardwareBasic` with `hasGpu:false` + 4 hardcoded RAM-gated models, lines 90-167). B's `model-class-router` has a `localModel` passthrough but **assumes a local model already exists** (verified — `resolveModelForClass` takes a ready `localModel`, never creates one). Odysseus's `hwfit` ranking math is exactly the missing engine that *produces* a fit model. Genuinely NEW vs A/B/C, and double-levered (proxy-cost + KVARK sovereign-local). The deep pass sharpened the cost: the **ranking math is the cheap core; the per-vendor detection layer is the hard multi-week tail** (port it staged, as a clean-room checklist — see §B).
|
||||
- **Upgrade-trigger: skill VERIFICATION, not skill extraction.** Waggle already auto-distills skills (`skill-distillation.ts` — ≥5-tool success-gated, sign-gated, dedup-via-`search_skills`, verified) — **do not re-recommend auto-extraction.** The gap is that nothing *verifies or prunes* the library; `/api/skills/test` is a static "what would this inject" preview, not a run-and-grade loop (verified, `skills.ts:598`). A self-testing, "verified"-badged, self-pruning library is what a *paid* skills moat needs.
|
||||
- **Moat: the cheapest win — extend C to email.** C's auto-fetch substrate landed this session; gcal + github are wired, gmail/outlook are not (verified). outlook `list_emails` is `riskLevel:'low'`, param-free (verified) — wiring its `harvestAction` lands inbox content into the substrate in a few lines.
|
||||
- **Moat-hardening + KVARK narrative: a taint-preserving untrusted-content artifact.** Waggle's injection defense scans at three chokepoints with hard drops, and recall already carries a "this is memory data, attribute honestly" preamble (verified, `orchestrator.ts:773-808`). So Waggle is **not** "detect-block only" — but tool output that *passes* the scan is still returned verbatim with no structural data/instruction delimiter (verified, `tool-executor.ts:114`). The deep pass found the genuinely novel part isn't the delimiter wrapper but **taint-preservation through message-normalization** (the trust bit survives the lossy turn-merge — see §C), which lifts the *security* merit a notch above "purely incremental." Its decisive value remains the **KVARK/EU-AI-Act narrative artifact** ("external content is structurally non-authoritative") plus a published THREAT_MODEL.md — a sales/compliance asset for the sovereign funnel.
|
||||
|
||||
**Does anything clear the bar? Yes — four clusters, scope-cut hard, plus one cheap fold.** None overlaps A/B/C; three hit moat/upgrade/KVARK, one is a *new-mechanism* proxy-cost lever:
|
||||
|
||||
1. **Email connector → memory harvest (§A)** — pure **moat**, lowest effort, highest certainty. Ship first.
|
||||
2. **Local-model recommend engine + token budget (§B)** — highest ceiling on **proxy-cost + KVARK**; clean-room TS port only; cheap ranking core, staged detection tail; kill the homelab serve fleet.
|
||||
3. **Skill verification & hygiene (§D)** — the **upgrade-trigger**; ADOPT the cheap hygiene judge, ADAPT the PRO "verified" badge.
|
||||
4. **Taint-preserving sandbox + THREAT_MODEL (§C, with G merged in)** — moat-hardening that clears **on the KVARK-narrative lever** with a now-stronger security floor; scope to the minimal wrapper + taint-preservation + the doc.
|
||||
|
||||
Plus one fold: **scope-gate the existing memory-mcp to read-only/owner (§E1)** — cheap moat *hygiene* (keeps poisoned external-agent writes out of the SOTA substrate; corroborated on both MCP servers).
|
||||
|
||||
Honest bottom line: **four scope-cut clusters + one cheap moat-hygiene fold.** I killed the draft's "governed outbound write API as the strategic prize" (letting external agents WRITE the substrate cuts against the dedup/quality discipline that makes it SOTA), demoted the event bus and distill-on-failure to DEFER (the event bus now carries a banked cost-safe design — §4), and removed the option to bundle odysseus's AGPL binary. Resist the consumer email client, the compare arena, the homelab serve fleet, and the deep-research re-build — real engineering, not Waggle's funnel.
|
||||
|
||||
---
|
||||
|
||||
## 2. Adoption Matrix
|
||||
|
||||
| Capability (Odysseus) | Waggle status | Verdict | Impact | Effort | Strategic fit |
|
||||
|---|---|---|---|---|---|
|
||||
| Email connector → memory harvest (extend C) | partial (substrate landed, not wired) | **ADOPT** | M–H* | S | **moat** |
|
||||
| VRAM/RAM-fit model-ranking math (lights up dead `llmfit`) | partial (route exists, engine absent) | **ADAPT** (clean-room TS) | H | L | **proxy-cost + KVARK** |
|
||||
| Multi-vendor HW *detection* (NVIDIA/Apple first; long tail) | partial (no GPU detect) | **ADAPT** (edge-case checklist, staged) | H | M–H | proxy-cost + KVARK |
|
||||
| Backend-aware serve-path gating | missing | **ADAPT** (fold into ranking math) | M | S | proxy-cost |
|
||||
| Adaptive input-token budget → discovered window | partial (128k default, unwired) | **ADOPT** (bundle w/ engine) | M | S | proxy-cost |
|
||||
| Skill necessity/redundancy/generic hygiene judge | partial (dedup at create only) | **ADOPT** | M | S | **upgrade-trigger** |
|
||||
| Autonomous skill-audit loop (run→judge→edit→retry→demote) | missing (test is static preview) | **ADAPT** (PRO) | H | L | upgrade-trigger |
|
||||
| Taint-preserving untrusted-content sandbox + THREAT_MODEL.md | partial (scan+drop, no wrapper/taint; no doc) | **ADAPT** | M | M | **moat-harden + KVARK narrative** |
|
||||
| Scope-gate the existing memory-mcp (read-only/owner token) | partial (ungated read+write, both servers) | **ADOPT** (minimal) | M | S | **moat hygiene + KVARK** |
|
||||
| Distill-on-FAILURE teacher-escalation | partial (success-only) | **DEFER** (fold later) | M | M | upgrade-trigger (needs §C wrapper) |
|
||||
| Governed OUTBOUND scoped agent write API + token taxonomy | partial (single device token) | **DEFER** (speculative arc) | H | L | cuts against substrate quality |
|
||||
| Self-delivering skill bundle (plugin.zip) | missing | **DEFER** (behind outbound API) | M | S | upgrade-trigger |
|
||||
| Event-counter trigger (shares cron `next_run`) + model-slot semaphore | missing (C24 deferred) | **DEFER** (cost-safe design banked) | M | M | proxy-cost + moat-freshness |
|
||||
| Per-query tool selection via HybridSearch lane (local-model enabler) | missing (sends all) | **DEFER** (§B-adjacent ADAPT-candidate) | M | M | proxy-cost (local-model; reliability tail) |
|
||||
| Multilingual email thread/quote parser (talon) | missing | **DEFER** (behind email harvest) | M | M | moat |
|
||||
| Compact prompt mode / low-signal bypass / mid-loop unlock | missing | **DEFER** | L | M | proxy-cost (taken lever) |
|
||||
| Fail-closed read-only/plan-mode gating | partial (fail-open denylist) | **DEFER** | M | S | hardening |
|
||||
| Vault audit-on-read + justification | partial (no per-access audit) | **DEFER** | M | S | KVARK |
|
||||
| URL credential redaction before logging | missing | **DEFER** | L | S | KVARK |
|
||||
| BYO consumer-subscription LLM (ChatGPT/Copilot OAuth) | missing | **DEFER** (bank the credential-resolver seam) | M | M | proxy-cost (ToS-gray) |
|
||||
| CalDAV SSRF/DNS-rebind validator | parity (OAuth, no URL) | **DEFER** | L | M | KVARK (custom-URL only) |
|
||||
| Injection-narrowed retrieval (RAG blast-radius) | partial | **DEFER** | M | M | moat (latent) |
|
||||
| GitHub SKILL.md importer / toolset-gated index | partial | **DEFER** | L | M | upgrade (cannibalizes marketplace) |
|
||||
| Scheduler hardening (zombie reap / overdue / IANA-tz) | partial | **DEFER** | L | S | reliability (no lever) |
|
||||
| Pinned-facts always-inject recall lane | partial (IdentityLayer covers it) | **DEFER**/near-SKIP | L | S | none (memory is free) |
|
||||
| ChromaDB dual-lane memory + Jaccard fallback | **ahead** (SOTA substrate) | **SKIP** | — | — | redundant |
|
||||
| IterResearch deep-research loop | parity (`retrieval-agent-loop`) | **SKIP** | — | — | parity + token liability |
|
||||
| Conversation compaction (summarize older half) | **ahead** (5-step pipeline) | **SKIP** | — | — | redundant |
|
||||
| Agentic email auto-triage pollers / email→cal extraction | missing | **SKIP** | — | — | consumer email client, proxy liability |
|
||||
| Blind A/B model-compare arena | missing | **SKIP** | — | — | off-brand (consumer arena) |
|
||||
| Multi-host SSH/tmux/vLLM serve fleet | missing | **SKIP** | — | — | homelab, off-brand |
|
||||
| nh3 HTML visual report / HF search / JSON-repair | parity | **SKIP** | — | — | redundant / babysits weak models |
|
||||
| In-process loopback token + reserved usernames | parity (in-process agent) | **SKIP** | — | — | solves a problem Waggle avoids |
|
||||
| Role-based per-USER tool RBAC | missing | **SKIP** | — | — | RBAC Phase 5 founder-DEFERRED |
|
||||
| Voice/STT/TTS/faces · standalone email client · themes · 2FA · mascot | n/a / off-brand | **SKIP** | — | — | off-brand B2B cockpit |
|
||||
|
||||
\* *Impact is connector-dependent: high for outlook (`list_emails` returns subject/from/preview); lower for gmail (`list_messages` returns ID stubs only — see §A). Stated honestly, not oversold.*
|
||||
|
||||
---
|
||||
|
||||
## 3. ADOPT / ADAPT Specs
|
||||
|
||||
### A. Email connector → memory harvest — extend C (moat) · ADOPT · Tier: PRO
|
||||
|
||||
**What:** Wire the low-risk email read action into the auto-fetch substrate that landed this session. Odysseus has deep IMAP/CalDAV connectors but feeds **none** of it to memory — that anti-pattern is the lesson; closing it is the win.
|
||||
|
||||
**Precision correction (the draft overclaimed "the single richest personal corpus"):**
|
||||
- **outlook `list_emails`** (verified `outlook-connector.ts:55-66`) is `riskLevel:'low'`, optional-only params, and returns real content (subject/from/receivedDateTime, preview via `$select`). **ADOPT cleanly now** — add `harvestAction = { action: 'list_emails' }`.
|
||||
- **gmail `list_messages`** (verified `gmail-connector.ts:23-35`) is `riskLevel:'low'` and param-free, but the Gmail API returns only `{ id, threadId }` stubs — **no subject/body**. Harvesting it alone lands near-empty frames. The actual content needs `get_message` (requires an `id` param → outside the param-free `harvestAction` contract). So gmail's value is **gated behind a small list→get enrichment** (a two-step harvest variant), not a one-line wire. Ship outlook now; treat gmail as a fast-follow once the enrichment lands.
|
||||
|
||||
**Files:**
|
||||
- `packages/agent/src/connectors/outlook-connector.ts:55-66` — add `harvestAction`.
|
||||
- Pattern mirror: `packages/agent/src/connectors/gcal-connector.ts:24` and `github-connector.ts:23` (both wired in C).
|
||||
- `packages/server/src/local/connector-harvest.ts:143-211` — `runConnectorFetch` already injection-scans each frame (`:191`), hash-skips unchanged (`:184`), frequency-floors on the last real sweep (`:154`). **No new harvest code.**
|
||||
- `packages/agent/src/connector-sdk.ts:45-50` — `harvestAction` contract.
|
||||
|
||||
**Cost/security:** Bounded — `hashItems` skips unchanged; injection-scan runs per frame (and the §C wrapper stacks on top). PRO-gated like C → zero FREE proxy exposure; "your inbox becomes searchable memory automatically" is a clean upgrade trigger. **Prerequisite for the §4 thread parser.**
|
||||
|
||||
---
|
||||
|
||||
### B. Cookbook local-model recommend engine — light up the dead `llmfit` route (proxy-cost + KVARK) · ADAPT (clean-room TS) · Tier: FREE/TRIAL
|
||||
|
||||
**What:** **Clean-room re-implement** odysseus's `hwfit` ranking math in TypeScript so Waggle's already-shipped local-inference route stops returning a no-op. This is the on-ramp B needs: B routes to a local model *if one exists*; this is how a FREE/TRIAL user *obtains and picks* one that actually fits their machine — and every such user stops burning the Waggle-funded Anthropic proxy. Doubles as the KVARK sovereign-local story: "scan your machine → run a model that fits → agent + memory now run free, on-device, your data never leaves."
|
||||
|
||||
**AGPL caveat (load-bearing):** Do **NOT** bundle odysseus's `hwfit`/`llmfit` binary as the tool `local-inference.ts` shells out to — that ships an AGPL binary with the proprietary product. Re-implement the algorithm in TS as a sidecar function (drop the `execFile`/`callLlmfit` indirection entirely, or point `LLMFIT_PATH` at our own clean-room TS CLI). The route, types, and UI already exist (`local-inference.ts` lines 21-63, 200-259), so effort is L only because of the math, not the surface.
|
||||
|
||||
**Effort/risk reframing (the deep pass corrected this — the cost is honest now):** the two halves have very different cost profiles. The **ranking math is the cheap, high-value core** — `fit.py`/`models.py` are pure functions (quant bytes-per-param, MoE active-param math, harmonic CPU-offload tok/s, composite score) that port cleanly to TS and are unit-testable (effort L). The **detection layer is the hard, multi-week reliability tail** — `hardware.py` (~900 LOC) is a per-vendor bug graveyard you cannot guess: WSL non-interactive shells hide `nvidia-smi` from PATH; driver-mismatch strings must be disambiguated from "no GPU"; Grace-Blackwell unified memory reports `memory.total=[N/A]`; Strix Halo's BIOS UMA carveout shows only in `mem_info_vis_vram_total` and must NOT be capped at system RAM; Apple needs `recommendedMaxWorkingSetSize` fractions; Windows WMI's 32-bit `AdapterRAM` caps at 4 GB so you must read the registry `qwMemorySize`; consumer RDNA is GGUF-only-serve truth. **Port the detection as a documented edge-case CHECKLIST (clean-room — port the KNOWLEDGE, never the AGPL code) and STAGE it: ship NVIDIA + Apple-Silicon + basic-RAM first (covers ~all Waggle desktop users), then work the long tail iteratively.** Keeps §B the #2 pick while making its cost honest — the engine is cheap; detection reliability is the real spend.
|
||||
|
||||
**Port (scope-cut to the laptop cockpit):**
|
||||
1. **Fit/quant/offload ranking** (odysseus `fit.py`/`models.py`) — the cheap core: per-quant bytes-per-param, MoE active-param math, GPU→CPU-offload harmonic walk with context halving, weighted quality/speed/fit composite + arch-age bonus. Surfaces too-tight rows instead of hiding them. Replaces `basicModelRecommendations` (verified: 4 hardcoded models gated on RAM only, lines 160-168).
|
||||
2. **Hardware detection** — staged NVIDIA + Apple-Silicon + basic-RAM first, then the long tail as a clean-room edge-case checklist (see reframing above). Real VRAM detection is the prerequisite: `detectHardwareBasic` hardcodes `hasGpu:false` (verified `:151`), so the ranker today can't tell a laptop iGPU from a 4090.
|
||||
3. **Backend-aware serve-path gating** — Apple/Windows/consumer-AMD (RDNA) → GGUF-only; never recommend an AWQ repo a Mac can't load. Folds into the ranker at low marginal cost.
|
||||
|
||||
**Bundle — adaptive input-token budget (proxy-cost, effort S, ADOPT):** `context-compressor.ts:404` defaults `maxContextTokens: 128000`, and the live call site **passes no override** (verified `chat.ts:1254` — only `budgetModel`/`litellmUrl`/`litellmApiKey`), so a local 4k/8k model is sized as if it had a 128k window → blowout and mis-sized compaction. Pass an override derived from the discovered window (`window*0.85`, clamp, conservative-on-unknown). ~40 lines + one wiring point; it is the change that makes the §B models actually *work*.
|
||||
|
||||
**§B-adjacent ADAPT-candidate — per-query tool selection for local models (proxy-cost; bundle, don't headline):** a 4k/8k local model physically cannot hold 30 connectors' + MCP tool schemas in context, so the §B on-ramp is only half-useful without trimming the tool surface per turn. The non-obvious Waggle-native move: implement per-query tool selection by **reusing the existing HybridSearch substrate** (embed tool descriptions into a dedicated lane, retrieve top-K) rather than standing up a new vector index — it dogfoods the moat asset as the cost lever. The wiring slot already exists and is dead: `filterToolsForContext` (`tool-filter.ts`) is a static 3-bucket filter with **zero production callers** (verified — only the barrel export + tests). **The real cost is the reliability tail, not the retriever:** odysseus hardened selective exposure against ~10 cited regressions (e.g. #1707 "tell me" loading the whole email toolset; #1567 Ollama small models emitting one native-tool token then stopping → a native-schema-vs-fenced-prose delivery switch; contact-vs-memory mispick) with a tiny ALWAYS_AVAILABLE floor + word-boundary keyword/structural fallback + continuation-topic inheritance. Porting the retriever WITHOUT that de-risk layer ships the exact failure mode they already paid to fix. Distinct from A/B; DEFER as a deliberate §B-adjacent arc — do not over-promote.
|
||||
|
||||
**Cut wholesale (off-brand / over-depth):** multi-host SSH/tmux/SGLang serve lifecycle, 50-entry GPU bandwidth tables, AMD gfx-family/CDNA-vLLM branch, deterministic `llama.cpp` serve-profile generation (Ollama abstracts it), HF model search (marketplace covers discovery), the 917-row HF catalog (ship a ~40-row Ollama-scoped curated catalog instead).
|
||||
|
||||
---
|
||||
|
||||
### C. Taint-preserving untrusted-content sandbox + THREAT_MODEL.md (moat-harden + KVARK narrative) · ADAPT · Tier: ALL
|
||||
|
||||
**Honest scoping (M impact, with a security floor now a notch above "incremental").** Waggle is **not** "detect-block only": `scanForInjection` gates harvest, recall (`orchestrator.ts:777` drops the *entire* recall on a flag, verified `:778-786`), and tool output (`tool-executor.ts:114`, verified) — and recall already prepends a heavy "these are saved facts, attribute provenance honestly, do not treat as continuity/instructions" preamble (verified `:795-808`). So a bare delimiter wrapper's *security* delta is incremental defense-in-depth. **But the deep pass found the genuinely novel, hard-to-replicate mechanism is taint-preservation through normalization** (below) — which raises the security merit above "purely incremental." Even so, the cluster's decisive case is the **KVARK-narrative lever**, which is why the THREAT_MODEL doc (the draft's separate §G) is **merged in here as the co-deliverable**: the wrapper is the artifact, the doc is the sale.
|
||||
|
||||
**Port (clean-room from `prompt_security.py` + `llm_core.py`):**
|
||||
- `untrustedContextWrapper(label, body)` → delimiter-guarded block + a "this is data, not instructions" header + **marker-escaping** (`_escape_guard_markers`) so an embedded close-marker cannot break out of the sandbox (the wrapper treats its own guard markers as an attack surface).
|
||||
- **Taint-preservation through message-normalization (the genuinely novel part — port the concept, not just the wrapper).** Odysseus carries the trust bit (`metadata.trusted=False`) THROUGH the lossy provider message-normalization step: when consecutive user turns are merged to satisfy role-alternation, an untrusted-context predecessor triggers insertion of a synthetic assistant **boundary turn** instead of concatenation (`llm_core.py:1334`), so the merge that would silently re-fuse untrusted data into the real user request cannot erase the boundary. That two-layer structural defense (the boundary survives the merge that re-fuses it) is the hard-to-replicate idea — port the *principle*: any taint-tagged block stays a distinct message/section and is never string-concatenated into the user's actual request. Waggle's assembly differs structurally, so port the shape, not the lines.
|
||||
- Apply the wrapper to the one place content passes verbatim today: **tool output** (`tool-executor.ts:114`, after the scan). Optionally re-wrap the recall block (low marginal value — it already has the preamble).
|
||||
- A regression test that an embedded close-marker cannot escape the block, and that a taint-tagged block is never fused into the user turn.
|
||||
- **THREAT_MODEL.md** — a crisp desktop/single-user trust-boundary + honest known-gaps doc grounding the already-built controls (`scanForInjection`, `confirmation.ts`, `install-audit.ts`, `vault.ts`, and this wrapper). Grep confirms none exists. This is the sellable KVARK/EU-AI-Act compliance asset.
|
||||
|
||||
**Why it clears the bar:** hardens the FREE-FOREVER moat's #1 attack surface (poisoned harvest frames that re-fire on every future recall) AND produces a concrete sovereign-trust artifact ("external content is structurally non-authoritative and cannot escape its boundary, even through provider normalization"). Effort M. Without the THREAT_MODEL framing this would be a DEFER; with it, plus the taint-preservation floor, it is a KVARK-funnel asset.
|
||||
|
||||
---
|
||||
|
||||
### D. Skill verification & hygiene layer (upgrade-trigger) · ADOPT (cheap subset) + ADAPT (PRO loop)
|
||||
|
||||
**Do NOT re-recommend auto-extraction** — `skill-distillation.ts` already does success-gated, sign-gated, dedup-via-`search_skills` distillation (verified `:31-79`). The gap is *verification and pruning*. A "verified"-badged, self-pruning library is what converts a pile of unverified drafts into a paid moat.
|
||||
|
||||
**D1 — Necessity/redundancy/generic hygiene judge (ADOPT, effort S — the 20% that delivers most):** a periodic **single LLM call per skill** (no agent re-run) asking "is this still necessary / redundant with peers / too generic," demoting the loser to **draft (never delete)** and flagging it on the card. Waggle only dedups at *creation*; an auto-growing library bloats without this.
|
||||
- Files: `packages/server/src/local/routes/skills.ts` (where distillation lands); reuse `packages/agent/src/judge.ts` for the verdict; write the advisory flag to a usage sidecar so `SKILL.md` doesn't churn.
|
||||
|
||||
**D2 — Autonomous skill-audit loop (ADAPT, PRO, effort L):** run each skill via the agent loop against a synthesized test task → `judge.ts` grades → auto-rewrite the `SKILL.md` to fix flagged issues → retry → demote-to-draft on persistent failure → surface a **"verified" badge + confidence** on the card. Today `/api/skills/test` (verified `skills.ts:598-633`) is a static prompt-injection *preview*, not a run-and-grade loop. Reuse `judge.ts` + `iterative-optimizer.ts`. **PRO-gated and batched** (agent re-run + judge + rewrite per skill burns proxy). The "verified" badge is the sellable artifact.
|
||||
|
||||
**D3 — Distill-on-FAILURE teacher-escalation (DEFER, fold later):** Waggle distillation is explicitly success-only (verified `skill-distillation.ts:35-37`, "a failed/refusal turn has no recipe yet"), so the "learn the fix when you fail" axis is missing — a real gap, but lower priority and it **needs the §C wrapper** to safely capture a failed trace. Reframe as **in-proxy model-class escalation** (Haiku→Opus via B) that captures the Opus fix as a durable skill. Drop odysseus's English-only regex give-up tier. Revisit after D1/D2 ship.
|
||||
|
||||
---
|
||||
|
||||
### (folded in) E1. Scope-gate the existing memory-mcp — read-only / owner token (moat hygiene + KVARK) · ADOPT (minimal)
|
||||
|
||||
**What:** add **owner-scoped + read-only token modes** so Claude Code/Codex can be granted *recall-only* access to one workspace's mind, instead of today's full read+write to `~/.waggle`. Verified on BOTH MCP servers: `memory-mcp/src/index.ts:64-72` registers `registerMemoryTools` + `registerCleanupTools` (write/delete) with **no auth/scope**, and `hive-mind-mcp-server/src/tools/memory.ts` registers `save_memory` (WRITE, `:20`) and `recall_memory` (READ, `:73`) **in the same file with identical exposure** — the `scope` enum there is *search breadth, not access control*. The MindDB already keys by workspace, so the gate is cheap, and it aligns with the **mind-isolation durable pin**.
|
||||
|
||||
**Concrete low-effort mechanism (port these two ideas, not the HTTP bundle):** (a) **scope-gate MCP tool *registration*** at server start (`HIVE_MIND_SCOPES=memory:read` ⇒ register `recall_memory` but never `save_memory`), so a read-only token literally cannot mutate the substrate; (b) **write-implies-read scope expansion** (`ensure_before` — granting `memory:write` auto-inserts `memory:read`), the ~15-line correctness detail that makes a granular scope model usable. Feed grants into the existing `install-audit.ts`.
|
||||
|
||||
**Why it clears (and why the bigger version doesn't):** read-only scoping is **moat hygiene** — it keeps a poisoned or buggy *external* agent from writing junk into the SOTA substrate. That protects the moat. The draft's larger **E2/E3 — a "governed outbound scoped *write* API" framed as "the strategic prize"** — is **DEMOTED to DEFER**: letting external agents WRITE the substrate by design cuts directly against the dedup/quality discipline that makes it LoCoMo-87.66 SOTA, and the full token taxonomy + middleware is a speculative KVARK-narrative arc, not a now-build. Ship the read-only gate; design the write API later, if ever.
|
||||
|
||||
---
|
||||
|
||||
## 4. SKIP / DEFER (one-line reasons)
|
||||
|
||||
**SKIP (off-brand / parity / no lever):**
|
||||
- **ChromaDB dual-lane memory + Jaccard fallback** — substrate is LoCoMo-87.66 SOTA (HybridSearch + cross-encoder + KG bridge); strictly ahead. (Deep memory-retrieval trace confirms Waggle ahead on every retrieval property — RRF vs linear blend, CE reranker, read-side *blocking* vs *framing*.)
|
||||
- **IterResearch deep-research loop** — parity with `retrieval-agent-loop.ts` (checkpoint/resume + cost halts); odysseus's is less hardened and token-heavy = a proxy-cost *liability*. (The deep-research/compare deep-trace agent failed on schema retries, but the breadth pass already settled this area — no rescue needed.)
|
||||
- **Conversation compaction** — Waggle's 5-step pipeline + messages-compressor + long-task context-manager subsume summarize-older-half.
|
||||
- **Agentic email auto-triage pollers / email→calendar extraction** — textbook consumer email client; an LLM call per inbound message on a poller is a direct hit on the Waggle-funded Anthropic proxy; off-brand.
|
||||
- **Blind A/B model-compare arena** — consumer/LMArena feature; Waggle is a B2B cockpit; model selection is automated (B) and quality is judged by `judge.ts`, not user voting.
|
||||
- **Multi-host SSH/tmux/vLLM serve fleet + llama.cpp serve-profiles** — homelab-grade; Waggle's user is a single laptop on Ollama (which autotunes `n_gpu_layers` behind its modelfile); Ollama-pull covers it.
|
||||
- **Built-in MCP tool-server packaging / image-gen fit / nh3 HTML report / HF model search / weak-model JSON-repair** — parity, off-brand (consumer media), or babysitting weak local models. (Bank the npx-cache-precheck + anyio-cancel-scope defensive nugget for if/when Waggle auto-spawns npx MCP servers.)
|
||||
- **In-process loopback token + reserved usernames** — solves an out-of-process privilege-crossing problem Waggle's in-process Node agent doesn't have.
|
||||
- **Per-USER tool RBAC** — a real TEAMS idea, but **RBAC Phase 5 is founder-DEFERRED** (don't re-raise); bank the fail-closed `is_public_blocked_tool` detail for when it reopens.
|
||||
- **Voice/STT/TTS/faces · standalone email client · gallery/image editor · Theme Studio · 2FA/TOTP · companion mascot** — off-brand for a B2B cockpit + demand-gen funnel; Hive DS brand consistency is deliberate.
|
||||
|
||||
**DEFER (real, but gated behind a trigger):**
|
||||
- **Governed outbound scoped *write* API + token taxonomy (E2/E3)** — the draft's headline "prize"; demoted because external write to the substrate cuts against the quality discipline that makes it SOTA. Revisit as a deliberate KVARK arc *after* E1's read-only gate proves the demand. (The owner-attribution context-swap `_as_owner` is a clean TEAMS-multi-tenant pattern to remember; nothing to build single-user.)
|
||||
- **Event-counter trigger (C24) — DEFER, but with a concrete cost-safe design now banked.** Verified `cron-store.ts` is schedule-only (DDL has no `trigger_type`/`trigger_event`/`trigger_counter`; `getDue` = `enabled AND next_run_at<=now`; zero `event`/`trigger` matches). The cost-safe mechanism that answers the proxy objection: a named-event counter lives in the SAME row as cron's `next_run_at`; on threshold the bus persists `counter=0, next_run_at=now` to the DB **before** invoking the in-memory scheduler — so the trigger is reboot-durable and replays through the ordinary `next_run<=now` poll (cron + event unified on one path) — paired with a **model-slot semaphore** so pure-code reactions (index reconcile, prune) fire freely while LLM reactions serialize one-at-a-time. Net: idle FREE/TRIAL workspaces fire zero maintenance LLM calls, and memory gets tidied right after a harvest burst instead of up to 24h later. Ship it when memory-freshness-between-cron-ticks becomes a real complaint; the design is recorded so it isn't re-derived. (Waggle's `SignalBus` already carries the events — it's display-only today; this is the reactive half.)
|
||||
- **Per-query tool selection / compact prompt / delivery-format switch** — reframed by the deep pass from "same lever A/B took" to a **§B-adjacent ADAPT-candidate** (see §3.B): a *local-small-model* enabler (reuse the HybridSearch substrate as the tool retriever; native-schema-vs-fenced-prose delivery for non-API Ollama models), bundled with §B and gated by the same reliability tail (~10 cited regressions). On the Anthropic proxy path prompt-caching + B blunt the win; the concentrated value is the local/sovereign path. DEFER with §B.
|
||||
- **Distill-on-FAILURE (D3)** — fold onto B's escalation after D1/D2; needs the §C wrapper first.
|
||||
- **Multilingual email thread/quote parser (talon)** — becomes load-bearing the instant §A ships (else a 10-deep thread stores the same paragraph 10×). DEFER until email harvest is live; then ADAPT as a harvest pre-pass.
|
||||
- **Fail-closed read-only gating** — Waggle's `isReadOnly` persona filter is fail-OPEN; flip to inverse-allowlist + static mutator backstop opportunistically when persona governance is next touched.
|
||||
- **Vault audit-on-read + justification / URL credential redaction (`redactUrl`)** — cheap EU-AI-Act hygiene (strip userinfo+query+fragment from LiteLLM/connector endpoint URLs before logging); fold into the next compliance/connector-logging pass. Plus a sensitive-basename deny list (.ssh/.env/id_rsa) + fix the prefix-weak `startsWith(root)` in `file-store.ts:59` to a real segment-boundary containment check — near-free desktop-fs hardening for the sovereign story.
|
||||
- **BYO consumer-subscription LLM (ChatGPT/Copilot OAuth)** — keep DEFER: genuine proxy relief, but ToS-gray, brittle, ban-risk, and widens off the deliberate Anthropic-only proxy; the `privacyRequired`→local path already gives a sanctioned zero-proxy escape. **Bank the reusable primitive underneath, though:** Waggle's `ProviderEntry.apiKey` is a static string (verified `model-router.ts:8`); odysseus's value is a **refreshable runtime-credential resolver seam** (per-call OAuth refresh = JWT-`exp` decode + skew + per-id refresh lock + a reauth/ratelimit/notfound error taxonomy). The ChatGPT/Copilot backends are just two instantiations; the seam itself is reusable for any *sanctioned* OAuth-refreshing connector/provider (Copilot now, enterprise model-gateway / Anthropic-OAuth SSO later — a KVARK-adjacent sovereign story). Bank the seam; ship neither consumer backend now.
|
||||
- **CalDAV SSRF/DNS-rebind validator** — bank the harness for if/when a custom-URL/self-hosted (KVARK-sovereign) connector ships.
|
||||
- **Injection-narrowed retrieval (RAG blast-radius)** — its only concrete trigger is email auto-reply (off-brand, won't ship); note the pattern.
|
||||
- **GitHub SKILL.md importer / toolset-gated index** — a free arbitrary-GitHub importer competes with the *paid* marketplace; revisit only as a community on-ramp that funnels into marketplace discovery.
|
||||
- **Scheduler hardening (zombie reap / overdue / IANA-tz)** — genuine reliability, no business lever; cherry-pick overdue-`next_run`-advance only on a reported duplicate-cron bug; IANA-tz only when TEAMS cross-zone scheduling lands.
|
||||
- **Memory pinned-facts always-inject lane** — the only thing odysseus's recall has that Waggle's `recallMemory` lacks (a deterministic user-pinned "core facts" block injected every turn without retrieval). Memory is free in Waggle's model → moves no lever; `IdentityLayer`/`AwarenessLayer` already cover the always-on need. DEFER/near-SKIP.
|
||||
- **`bg_jobs`/`bg_monitor` auto-continue for long shell commands** — genuinely elegant (restart-safe exit-code file, idempotent follow-up), but a dev/power-user ergonomic that *adds* proxy cost (an extra agent run per completed job); lever-less. SKIP-leaning DEFER.
|
||||
|
||||
---
|
||||
|
||||
## 5. Ranked Top Recommendations
|
||||
|
||||
The bar culls hard; odysseus clears it on the three levers A/B/C left open. **Four clusters clear cleanly, plus one cheap fold**, ranked by strength-of-case × certainty, scope cut to the bone:
|
||||
|
||||
1. **Email connector → memory harvest (§3.A).** Highest certainty, lowest effort, pure **moat**. Extends the auto-fetch substrate that *landed this session*; **outlook `list_emails` is a one-line `harvestAction` wire** that lands inbox content into the mind (gmail needs a small list→get enrichment first — don't overclaim it). PRO-gated = cost-safe + upgrade trigger. **Ship first.**
|
||||
|
||||
2. **Cookbook local-model recommend engine + adaptive token budget (§3.B).** Highest strategic ceiling on **proxy-cost + KVARK**, and genuinely NEW vs A/B/C — B routes to a local model, this *creates* one. Resurrects an already-shipped-but-dead route. **Clean-room TS only — never bundle the AGPL binary.** Scope discipline is the whole game: the **ranking math is the cheap core**; the **detection layer is the multi-week reliability tail — port it as a clean-room edge-case checklist and STAGE it (NVIDIA + Apple + basic-RAM first, long tail iteratively)**; bundle the auto-derived token budget; **reject the SSH/tmux serve fleet, the bandwidth tables, the CDNA depth, the 917-row HF catalog, and llama.cpp profile generation.** (Per-query tool selection via the HybridSearch lane rides alongside as a §B-adjacent local-model enabler — DEFER, not headline.)
|
||||
|
||||
3. **Skill verification & hygiene layer (§3.D).** The **upgrade-trigger** play — **only the verification half** (auto-extraction already shipped). ADOPT the cheap necessity/dedup judge (D1, single call/skill) now; ADAPT the PRO-gated run-and-grade audit loop (D2) for the sellable "verified" badge. Distill-on-failure (D3) defers.
|
||||
|
||||
4. **Taint-preserving sandbox + THREAT_MODEL.md (§3.C).** Defense-in-depth on the moat's #1 attack surface (poisoned harvest) **plus** the concrete KVARK/EU-AI-Act trust artifact. The deep pass lifted this from "incremental" to a real security floor by naming **taint-preservation-through-normalization** (the boundary survives the turn-merge that re-fuses it) as the concept to port — but it still clears the bar on the **narrative lever**, so the doc is the co-deliverable, not an afterthought. Scope tight.
|
||||
|
||||
**Folded in, not headlined:** scope-gate the memory-mcp to read-only/owner (§3.E1) — cheap moat hygiene (write-implies-read + scope-gated tool registration) that keeps external-agent writes out of the SOTA substrate.
|
||||
|
||||
**Everything else defers behind explicit triggers or skips.** Do not let odysseus's well-built but off-strategy surfaces — the consumer email client, the compare arena, the homelab serve fleet, the deep-research re-build, the "let external agents write memory" outbound API — pull scope. They are real engineering, not Waggle's funnel.
|
||||
|
||||
---
|
||||
|
||||
## 6. Critique Deltas (what changed, and why)
|
||||
|
||||
1. **Corrected the §A email overclaim with verified API behavior.** The draft said wiring gmail+outlook "lands the single richest personal corpus … in a few lines." Verified: Gmail's `list_messages` returns only `{id, threadId}` stubs (no subject/body) — harvesting it alone writes near-empty frames; real content needs `get_message` (requires an `id` param, outside the param-free `harvestAction` contract). **outlook `list_emails` returns real content and ADOPTs cleanly now; gmail is gated behind a small list→get enrichment.** Same ADOPT verdict, honest about which half ships in one line.
|
||||
|
||||
2. **Banned bundling odysseus's AGPL binary in §B; mandated clean-room TS.** The draft offered "port to TS **or** bundle it as the `llmfit` binary." The bundle option ships an AGPL binary alongside the proprietary product — the worst-case AGPL trap. Removed it; the spec is now a clean-room TS re-implementation of the fit math, and the license note at the top is strengthened to forbid binaries explicitly.
|
||||
|
||||
3. **Demoted §C from H to M impact and merged §G into it.** Verified that Waggle is *not* "detect-block only": recall already injection-scans with a **full drop on flag** AND carries a substantial "this is memory data, attribute honestly, do not treat as instructions" preamble (`orchestrator.ts:773-808`). A bare structural wrapper's *security* gain is incremental — so the cluster clears the bar **via the KVARK-narrative lever**, which is why the THREAT_MODEL.md (draft's standalone §G) is folded in as the co-deliverable that makes it sell. Scoped the wrapper to the one verbatim-pass site (tool output).
|
||||
|
||||
4. **Killed the draft's "governed outbound *write* API as the strategic prize" (E2/E3 → DEFER); kept only the cheap read-only gate (E1).** Verified memory-mcp is local stdio with ungated read+write. Letting external agents *write* the substrate by design cuts directly against the dedup/quality discipline that makes it LoCoMo-87.66 SOTA — so the big version is moat-*risky*, not moat-deepening, and the token-taxonomy middleware is a speculative arc. The honest win is the **read-only owner-scoped gate** (moat hygiene, effort S), aligned with the mind-isolation pin.
|
||||
|
||||
5. **Demoted the event bus (§F) and distill-on-failure (§D3) from ADAPT-fold to DEFER.** Both are real but thin: the event bus is a proxy-free moat-*freshness* nicety whose lever is marginal; D3 needs the §C wrapper first and is lower priority than D1/D2. Neither is a differentiator on its own.
|
||||
|
||||
6. **Held §B's grounding as the strongest survivor — and verified it end-to-end.** Confirmed the dead `llmfit` shell-out + `hasGpu:false` basic fallback (`local-inference.ts:90-167`), that `model-class-router` *assumes* a local model exists (so the on-ramp is genuinely new), and that the token-budget bug is live (`chat.ts:1254` passes no `maxContextTokens` override → 128k default for every local model). Kept it at #2.
|
||||
|
||||
7. **Re-counted the bar-clearers honestly: four clusters + one fold.** The accurate, restrained framing: **A (moat), B (proxy-cost+KVARK), C (moat-harden+KVARK narrative), D (upgrade-trigger)** clear cleanly; **E1** folds in cheap; everything else defers or skips.
|
||||
|
||||
8. **Held all SKIPs.** Re-tested every SKIP against the bar — all correctly skipped against the SOTA substrate, the Anthropic-only proxy cost model, and the B2B-cockpit brand. No false negatives to rescue.
|
||||
|
||||
**Deep mechanism-trace deltas (this revision — folding the 7 deep dossiers into the hardened breadth brief):**
|
||||
|
||||
9. **(A) Made §B's cost honest — engine cheap, detection is the multi-week tail.** The breadth brief lumped HW detection at effort "M" beside the ranking math. The `hwfit-detection` dossier shows `hardware.py` (~900 LOC) is an un-guessable per-vendor bug graveyard (WSL PATH holes, driver-mismatch strings, Grace-Blackwell `[N/A]` unified memory, Strix Halo UMA carveout, Apple working-set fractions, Windows WMI 4 GB `AdapterRAM` cap → registry `qwMemorySize`, RDNA-GGUF-only). Reframed: the fit math (`fit.py`) is the cheap, high-value core (effort L); detection ports as a **clean-room edge-case CHECKLIST**, STAGED (NVIDIA + Apple + basic-RAM first, long tail iteratively). Matrix split into a math row (L) and a detection row (M–H); §B stays #2.
|
||||
|
||||
10. **(B) Upgraded §C from "wrap tool output" to "preserve taint across normalization."** The `security-sandbox` dossier found the genuinely novel mechanism: odysseus carries `trusted=False` through the lossy provider message-merge by inserting a synthetic assistant boundary turn (`llm_core.py:1334`) instead of concatenating, plus `_escape_guard_markers` delimiter-breakout escaping. Named that two-layer structural defense as the concept to port; security merit nudged a notch above "purely incremental" while the verdict/lever (moat-harden + KVARK narrative) holds.
|
||||
|
||||
11. **(C) Promoted the event-trigger from "thin DEFER" to "DEFER with a banked cost-safe design."** The `scheduler-events` dossier supplied the mechanism that answers the cost objection — an event counter sharing cron's `next_run_at` row, persisted before the in-memory dispatch (reboot-durable; unifies cron+event on one poll), plus a model-slot semaphore so pure-code reactions fire freely and idle workspaces cost nothing. Verified `cron-store.ts` is schedule-only. Recorded the design; kept it gated, not headlined.
|
||||
|
||||
12. **(D) Reframed tool-economy from "taken lever" to a §B-adjacent local-model enabler.** The `agent-loop-smallmodel` dossier shows per-query tool selection is a small-LOCAL-model unlock (a 4k/8k model can't hold 30 connectors' schemas) implementable by reusing the existing HybridSearch substrate — distinct from A/B, and `filterToolsForContext` is the dead socket (verified zero production callers). Bundled it with §B, flagged the ~10-regression reliability tail (and the native-schema-vs-fenced-prose delivery switch) as the real cost; did not over-promote.
|
||||
|
||||
13. **(E+F) Corroborated E1 with the second MCP server; banked the credential-resolver seam under BYO-subscription; confirmed pinned-facts DEFER.** `integrations-scope` confirmed `hive-mind-mcp-server/src/tools/memory.ts` exposes `save_memory`+`recall_memory` ungated → added "write-implies-read + scope-gated tool registration" as E1's concrete mechanism (verdict unchanged, ADOPT). `mcp-providers` identified the **refreshable runtime-credential resolver seam** (vs static `apiKey`, verified `model-router.ts:8`) as the reusable primitive worth banking while BYO-subscription stays DEFER. `memory-retrieval` confirmed the pinned-facts recall lane is the only delta vs `recallMemory` and moves no lever (memory is free) → DEFER/near-SKIP.
|
||||
|
||||
---
|
||||
|
||||
## 7. Deep Mechanism Appendix — what's actually hard to replicate
|
||||
|
||||
The deep mechanism-trace pass surfaced the genuinely non-trivial engineering behind the picks above — the "why this took a real team to build" evidence for the founder. Each is clean-room-portable as *knowledge*, never as AGPL code.
|
||||
|
||||
1. **Calibrated memory-bandwidth tok/s model with a harmonic CPU-offload blend** (`fit.py`). `raw_tps = (bw/model_gb)·0.55`; when a model spills to RAM, `eff_bw = 1/(frac/cpu_bw + (1-frac)/gpu_bw)` so the slow CPU portion dominates as it grows — **empirically calibrated** ("DeepSeek-Coder-V2-Lite Q4_K_M light offload → ~59 t/s est vs 59.8 measured"). You can read the formula; you cannot fake the calibration. (§B core.)
|
||||
2. **Per-vendor hardware-detection bug graveyard** (`hardware.py`, ~900 LOC). WSL PATH holes hiding `nvidia-smi`; driver-mismatch string disambiguation; Grace-Blackwell unified-memory `[N/A]`; Strix Halo BIOS UMA carveout (`mem_info_vis_vram_total`, must not cap at system RAM); Apple `recommendedMaxWorkingSetSize` fractions; Windows WMI 32-bit `AdapterRAM` 4 GB cap → registry `qwMemorySize`. Each line is a fixed bug — the multi-week reliability tail behind §B (port as a staged checklist).
|
||||
3. **Serving-path realism** (`fit.py`/`models.py`). It models *what actually serves on what*: vLLM/SGLang can't shard GGUF → single-GPU VRAM for GGUF, full multi-GPU for AWQ/GPTQ; consumer RDNA → GGUF-only; Apple/Windows → GGUF-only; multi-GPU dense → BF16 default. Operational ecosystem knowledge, not spec sheets. (§B serve-gating.)
|
||||
4. **Taint-preservation through message-normalization** (`llm_core.py:1334` + `_escape_guard_markers`). Carries `trusted=False` THROUGH the lossy role-alternation merge: an untrusted predecessor forces a synthetic assistant boundary turn instead of concatenation, so the normalization that re-fuses turns can't erase the data/instruction boundary. Everyone wraps; almost nobody preserves the taint across the pass that silently undoes it. (§C concept.)
|
||||
5. **Reboot-durable event-counter sharing cron's `next_run` + a model-slot semaphore** (`event_bus.py:99-105`, `task_scheduler.py`). Counter reset + `next_run=now` persisted to the DB *before* the in-memory dispatch, so a restart mid-queue replays through the ordinary poll; pure-code reactions bypass the `Semaphore(1)` that serializes LLM reactions. (§4 cost-safe event-trigger design.)
|
||||
6. **Per-query tool-retrieval hardened against ~10 named regressions + a native-schema-vs-fenced-prose delivery switch** (`tool_index.py`, `agent_loop.py`). Word-boundary keyword hints (not substring — "fix"/"serve"/"reply" must not fire inside "prefix"/"observe"/"replying"), structural regexes, continuation-topic inheritance, a tiny ALWAYS_AVAILABLE floor, and a per-endpoint switch because Ollama small models emit one native-tool token then stop (#1567). The de-risk layer is the hard part, not the embedding retrieval. (§B-adjacent concept.)
|
||||
7. **Refreshable runtime-credential resolver seam** (`endpoint_resolver.py` + `chatgpt_subscription.py`). A provider credential as a *refreshable OAuth session* — JWT-`exp` decode + skew, per-auth-id refresh lock (no double-refresh / reuse-burn), reauth/ratelimit/notfound taxonomy — vs Waggle's static `apiKey` string. The reusable primitive under BYO-subscription (§4 / delta 13).
|
||||
8. **Embedding-lane fingerprint-gated re-embed with rollback** (`embedding_lanes.py`). A sha256 fingerprint of `lane|url|model|dim` detects an embedding-config change, then preserves docs, recreates the collection, and re-embeds — **rolling back to the old vectors if the re-embed write fails**. The reusable lesson for when a Waggle user swaps embedding model (sqlite-vec also fixes dimension on first insert). Memory-store hardening, not a headline lever.
|
||||
37
docs/analysis/odysseus-impl-plan-2026-06-28.md
Normal file
37
docs/analysis/odysseus-impl-plan-2026-06-28.md
Normal file
@@ -0,0 +1,37 @@
|
||||
# Odysseus Adoption — Implementation Plan (2026-06-28)
|
||||
|
||||
Source brief: `docs/analysis/odysseus-adoption-2026-06-28.md`. Branch: `codex/fix-ai-os-proof-plumbing`.
|
||||
AGPL: every port is clean-room TS (concept/knowledge only — no odysseus code, no binary).
|
||||
|
||||
## Phase 1 — high-confidence ADOPTs (this arc · all TDD-able · file-disjoint)
|
||||
|
||||
| # | Item | Lever | Files (primary) | Tier | Effort |
|
||||
|---|---|---|---|---|---|
|
||||
| 1 | **§A Email→memory harvest** (outlook) | moat | `packages/agent/src/connectors/outlook-connector.ts` (+ mirror gcal/github `harvestAction`); wired via existing `connector-harvest.ts` | PRO | S |
|
||||
| 2 | **§B-budget adaptive input-token budget** | proxy-cost | `packages/server/src/local/routes/chat.ts:~1254` pass `maxContextTokens` override (window·0.85, clamp, conservative-on-unknown); helper `compute_input_token_budget` clean-room in agent | all | S |
|
||||
| 3 | **§E1 memory-mcp read-only scope-gate** | moat hygiene | `packages/memory-mcp/src/index.ts`, `packages/hive-mind-mcp-server/src/tools/memory.ts` — scope-gated tool registration + write-implies-read | all | S |
|
||||
| 4 | **§D1 skill hygiene judge** | upgrade-trigger | `packages/server/src/local/routes/skills.ts` + reuse `packages/agent/src/judge.ts`; advisory flag → usage sidecar (no SKILL.md churn) | all | S |
|
||||
| 5 | **§C untrusted-content wrapper + THREAT_MODEL.md** | moat-harden + KVARK | new `packages/agent/src/untrusted-context.ts`; apply at `tool-executor.ts:114` (post-scan); investigate taint-preservation in message assembly; `THREAT_MODEL.md` | all | M |
|
||||
|
||||
**Gate before commit:** `npx tsc --noEmit` on shared/core/agent/server (+ memory-mcp, hive-mind-mcp-server) · `vitest run` on every touched package + new tests · multi-lens review (security + ts + founder-bar) · 0 regressions vs touched-area baseline.
|
||||
|
||||
## Phase 2 — L-effort builds (staged)
|
||||
|
||||
| # | Item | Status | Notes |
|
||||
|---|---|---|---|
|
||||
| 6 | **§B Cookbook ranking engine** | ✅ SHIPPED (`e46ff6b0`) | clean-room TS port of `fit.py` math (pure fns → TDD): quant bytes/param, MoE active-param, harmonic CPU-offload tok/s, composite score, serve-path gating. Then **staged detection** (NVIDIA `nvidia-smi` + Apple + basic-RAM first; long tail iteratively, as a checklist). Replaces `basicModelRecommendations` in `local-inference.ts`. |
|
||||
| 7 | **§D2 PRO autonomous skill-audit loop** | ✅ SHIPPED (this commit) | `skill-audit{,-store}.ts` (synth→run→judge→rewrite→retry), `skills-audit.ts` route (PRO + vault-key gated), GET `/api/skills` badge merge + staleness-on-read, `SkillRow` "verified · NN%" badge + per-row Verify trigger (`adapter.auditSkills`). **Advisory-by-default**: `autoRewrite`/`autoDemote` OFF, fail-safe taxonomy (a flaky judge can never mint a false badge nor demote a good skill). Reuses `LLMJudge` + `demoteSkillToDraft` (single active/draft owner); fences skill content via the §C `untrustedContextWrapper`. 87 tests; tsc 0 ×3. |
|
||||
| 8 | **§B-adjacent per-query tool selection** | ⏸ DEFER (per brief) | reuse HybridSearch lane as tool retriever; the dead `filterToolsForContext` is the socket; port the ~10-regression de-risk layer. The brief explicitly DEFERs this (§B-adjacent, "do not over-promote") — the reliability tail is the real cost. Revisit as a deliberate local-model arc. |
|
||||
|
||||
### §D2 open items (founder decisions / follow-ups, non-blocking)
|
||||
- **F3 (founder call):** "verified" is a same-model self-grade (the user's one key synthesizes the task, runs the skill, and grades it). Defensible (catches gross brokenness; the card shows confidence %, not a bare check) but consider relabel ("self-check passed") OR adversarial held-out test + a different model class. Brief names it "verified", so kept as-is pending a call.
|
||||
- **F5 / restore coupling:** `restoreSkillToActive` doesn't reset the audit badge's `consecutiveFails` → a restored skill can re-demote on the next confident fail. Benign while `autoDemote` defaults OFF.
|
||||
- **T3 (TOCTOU):** `recordAuditBadge` read-modify-write isn't linearizable under concurrent same-skill POSTs (safe direction: missed increment → no false demote). Single-user/sequential-batch makes it a non-issue today.
|
||||
- **F8 (pre-existing):** `skills.ts` CRUD `onChange` reloads from `loadSkills` (drafts included), not `loadActiveSkills` — a demoted draft re-enters the live prompt until the next hygiene/audit run. Out of D2 scope.
|
||||
|
||||
## Method
|
||||
|
||||
1. **Design (workflow, parallel):** per Phase-1 item → exact edits + failing tests + risks (grounded in real files).
|
||||
2. **Implement (main tree, sequential, TDD):** test-first, targeted `tsc`+`vitest` after each.
|
||||
3. **Review (workflow, parallel):** security-reviewer + typescript-reviewer + founder-bar/correctness.
|
||||
4. **Fix → full gate → confirm → commit per phase.** No commit until gate + user confirm.
|
||||
173
docs/analysis/openhuman-adoption-2026-06-28.md
Normal file
173
docs/analysis/openhuman-adoption-2026-06-28.md
Normal file
@@ -0,0 +1,173 @@
|
||||
# OpenHuman → Waggle OS: Prioritized Adoption Report (Hardened, Final)
|
||||
|
||||
## 1. Framing & Honest Verdict
|
||||
|
||||
OpenHuman is a genuine mature peer — Rust core, real memory substrate, shipped voice/meet/screen surfaces. So the question is not "is it serious," it's the founder's bar: does any item move a Waggle-specific lever — **KVARK funnel, the memory+harvest moat, the skills/connectors upgrade trigger, or Waggle-funded proxy cost** — or is it the same "nice but not differentiating" class the founder just rejected wholesale (Local-Agent-Studio)?
|
||||
|
||||
**Where OpenHuman is genuinely ahead (verified against grounding):**
|
||||
- **Pre-LLM per-tool-result compression** (TokenJuice). Confirmed gap: tool output is appended to the messages array *verbatim* (`agent-loop.ts:511` `r.content`; `tool-executor.ts:161-166` raw result returned, no compaction). Waggle only compresses *after* the conversation crosses 50% (`context-compressor.ts`).
|
||||
- **Capability-aware model routing.** Confirmed: `resolveUsableModel` (`model-availability.ts:86-109`) is provider-*readiness* fallback only; `capability-router.ts` routes tool **names** to sources, not **tasks** to model classes. No "lightweight → cheap, reasoning → frontier" policy exists.
|
||||
- **Scheduled auto-fetch connector→memory loop.** The grounding flags this as *"the ONLY significant gap in the memory substrate"* — cron infra (`cron-store.ts`) and harvest (`harvest/pipeline.ts`, pull-only) both exist but are never wired together.
|
||||
- Idle background cognition (Subconscious), trigger triage, memory-diff — real, but each carries a real-cost or prerequisite problem (below).
|
||||
|
||||
**Where Waggle already matches or leads (do not touch):**
|
||||
- **Memory substrate** — hierarchical trees (`wiki-compiler`), Obsidian/Notion export, 4-profile RRF + reranker + chunk-level scoring (`search.ts`/`scoring.ts`), KG bridge. This is the LoCoMo-87.66-SOTA moat; at-parity-or-ahead on every memory item.
|
||||
- **Approval gate** (`confirmation.ts` — risk taxonomy + autonomy tiers + never-autopass blacklist) — *more* sophisticated than OpenHuman's.
|
||||
- **Warm-start memory** (`orchestrator.recallMemory`, Hermes `session-start`), cron/automations, iteration-budget, loop-guard, awareness, Composio + 30 connectors, vault, Hive DS — all present.
|
||||
|
||||
**Does anything clear the bar? Yes — three items, heavily scope-cut, not the draft's three:**
|
||||
|
||||
1. A **pure-code tool-result compression subset** (JSON-table crusher + live search dedup) — a clean, zero-added-cost margin lever that reuses `dedup.ts`.
|
||||
2. **Deterministic capability-aware routing** of *known-lightweight internal calls* to Haiku-on-proxy / local — a real proxy-cost lever that works even for a vanilla FREE user with no Ollama.
|
||||
3. **PRO-gated auto-fetch connector→memory** — the *only* item that touches the actual memory+harvest **moat** rather than just margin; the grounding calls it the sole substrate gap; gating to PRO makes it simultaneously a **tier trigger** and **cost-safe**.
|
||||
|
||||
**What I cut from the draft as still-too-loose:** the token-aware-truncation **LLM-summarization fallback** (largely redundant with existing message-level compression, and summarizing-on-truncation can *add* budget-model proxy cost on the very FREE/TRIAL tier it claims to protect — a hard slice is free); the **standalone "savings metering" ADOPT** (an internal `cost-tracker` accumulator is fine; a user-facing "we saved you N tokens" panel is exactly the nice-but-not-differentiating scope creep the founder rejects); and the routing layer's **arbitrary-user-task complexity classifier** (needs its own classifier = cost + risk; the deterministic internal-call subset captures most of the win with none of it).
|
||||
|
||||
Honest bottom line: **two tightly-scoped cost levers + one PRO-gated moat-deepener, with metering folded in as internal telemetry. Defer four real-but-blocked items behind explicit triggers; skip the rest.** Resist the Rust engine, the 96-rule overlay, the ML compressor, the mascot, the Meet agent.
|
||||
|
||||
---
|
||||
|
||||
## 2. Adoption Matrix
|
||||
|
||||
| Capability (OpenHuman) | Waggle status | Verdict | Impact | Effort | Strategic fit |
|
||||
|---|---|---|---|---|---|
|
||||
| Per-result: JSON-table crusher (pure code) | none (verbatim) | **ADAPT** | M–H* | S | cost lever |
|
||||
| Per-result: live search-result dedup/merge | partial (ingest-only) | **ADAPT** | M | S | cost lever |
|
||||
| Per-result: token-aware truncation (no LLM) | partial (char-only) | **ADAPT (minor)** | L–M | S | cost lever |
|
||||
| Per-result: LLM-summarization-on-truncation fallback | partial (msg-level only) | **DROP→DEFER** | L | M | redundant + can add cost |
|
||||
| Content-aware kind classifier (deterministic, feeds crusher) | none | **ADAPT** | — | S | cost lever (input only) |
|
||||
| Capability-aware routing — known-lightweight internal calls → Haiku/local | partial (readiness-only) | **ADAPT** | H | M | cost lever |
|
||||
| Capability-aware routing — arbitrary user-task complexity | partial | **DROP** | M | M | speculative (classifier cost) |
|
||||
| Privacy-required-on-device flag | none | **ADAPT (bundle w/ routing)** | L | S | KVARK narrative |
|
||||
| Savings tracking / cost attribution | none | **ADOPT (internal only)** | L | S | instrumentation |
|
||||
| Auto-fetch connector→memory loop (PRO-gated, dedup-capped) | partial (cron infra, no job type) | **ADOPT** | M–H | M | **moat + tier trigger** |
|
||||
| Trigger triage pipeline (drop/ack/react/escalate) | missing (event triggers deferred C24) | **DEFER** | H | L | blocked on webhook infra |
|
||||
| Subconscious idle cognition + durable per-thread goal | partial (read-only daemons) | **DEFER (cost-negative)** | M | L | burns proxy $ on free tier |
|
||||
| Taint-origin background safety | partial (autonomy tiers exist) | **DEFER (bundle)** | L | S | polish |
|
||||
| MCP live registry discovery (Smithery) | static 200+ + Composio on-demand | **DEFER (near-SKIP)** | L–M | M | redundant w/ Composio |
|
||||
| Memory-diff (git-backed change tracking) | missing | **DEFER** | M | L | `compliance/` already covers audit |
|
||||
| SuperContext first-turn scout | **has** (warm-start) | **SKIP** | — | — | redundant |
|
||||
| Trees / Obsidian / scoring / E2GraphRAG | **has / ahead** | **SKIP** | — | — | redundant w/ SOTA moat |
|
||||
| Pluggable external memory backend | partial (export-only) | **SKIP** | — | — | KVARK does sovereign on-prem |
|
||||
| 90k-entry skills aggregation | curated marketplace | **SKIP** | — | — | cannibalizes tier trigger |
|
||||
| Native voice (STT/TTS + lip-sync) | missing | **SKIP** | — | — | off-brand (B2B cockpit) |
|
||||
| Desktop mascot (Rive) | missing | **SKIP** | — | — | off-brand |
|
||||
| Google Meet agent (CEF/CDP) | missing | **SKIP** | — | — | multi-quarter, fragile, diff product |
|
||||
| Screen intelligence (macOS Vision + Ollama) | partial (browser only) | **SKIP** | — | — | macOS-only, commodity |
|
||||
| iOS companion / 18 messaging channels | missing | **SKIP** | — | — | mobile v2+; off-funnel |
|
||||
| OS keyring | **has** (`vault.ts` AES-256-GCM) | **SKIP** | — | — | vault better for server/KVARK |
|
||||
| Theme Studio | **has** (Hive DS tokens) | **SKIP** | — | — | brand consistency intentional |
|
||||
| Kanban / approval / cron / iteration-budget / loop-guard / awareness | **has** | **SKIP** | — | — | already shipped |
|
||||
|
||||
\* *Impact is workload-dependent: high for tool/connector-heavy sessions (JSON list responses, web research); low for memory-recall-dominated sessions. Stated honestly, not oversold.*
|
||||
|
||||
---
|
||||
|
||||
## 3. ADOPT / ADAPT Specs
|
||||
|
||||
### A. Tool-Result Compression — pure-code subset only (cost lever)
|
||||
|
||||
**What to build:** one pure-TS module `packages/agent/src/tool-output-compressor.ts`, invoked in `tool-executor.ts` **between** `tool.execute()` and the return, under a hard contract — **never enlarge output, never throw, fall through to passthrough; passthrough below a ~2KB gate** (exactly TokenJuice's guard). Two compressors plus a deterministic kind-classifier. Explicitly **reject** tree-sitter, the 96-rule overlay, ModernBERT, and CCR retrieval markers.
|
||||
|
||||
1. **JSON-table crusher** — array-of-objects → pipe-delimited table; force-keep head/tail rows + any row containing `error`/`panic` or a numeric outlier (>2σ). Pure `JSON.parse` + format; ~95% reduction on API list responses. **No LLM.**
|
||||
2. **Live search-result dedup** — call the trigram fuzzy-dedup already in `harvest/dedup.ts` (75% threshold) on `web_search` snippets before formatting. The logic exists; it is simply never invoked on real-time results today. **No LLM.**
|
||||
3. *(minor)* **Token-aware truncation** — replace the blunt 10K-char cut in `web_fetch` (`system-tools.ts:701-730`) with a token-estimated budget so the cap is consistent across prose/code/JSON. **No LLM.**
|
||||
|
||||
**Explicitly NOT building:** the LLM-summarization-on-truncation fallback. It is largely redundant with the existing message-level summarizer (`context-compressor.ts` at 50%, `messages-compressor.ts` with `COMPACTION_PROMPT`), and replacing a free hard-slice with a budget-model call **adds** proxy cost on FREE/TRIAL — net-positive only when a large result is followed by many turns. If data-loss complaints actually appear, revisit then.
|
||||
|
||||
**Files:** new `packages/agent/src/tool-output-compressor.ts`; insert at `tool-executor.ts:161`; `system-tools.ts:701-730` (web_fetch path); reuse `packages/hive-mind-core/src/harvest/dedup.ts`.
|
||||
|
||||
**Tier:** ON for all tiers, ungated — pure margin protection where Waggle funds the proxy.
|
||||
|
||||
**Cost/security:** Net reduction, zero added LLM cost. Only risk is over-compression hiding signal — mitigated by the force-keep rule + never-enlarge contract. Compressed output still passes the existing `scanForInjection()` (already runs post-tool).
|
||||
|
||||
### B. Capability-Aware Routing — deterministic internal-call subset (cost lever)
|
||||
|
||||
**What to build:** route a **fixed allowlist of known-lightweight internal calls** — the compaction summarizer, the kind-classifier from §A, tool-name selection, short structured-extraction — to the cheapest ready class: **Haiku on the built-in Anthropic proxy** by default, **local Ollama** when configured. No new classifier; the call sites are known a priori, so routing is deterministic and low-risk.
|
||||
|
||||
**Why this is a real FREE-tier lever:** the built-in proxy is Anthropic-only, so the universal win is **Haiku-on-proxy for lightweight work** (~10–12× cheaper than Sonnet, far cheaper than Opus) — it materializes for a vanilla FREE user with *no* local model. Ollama/on-device is the bonus for configured users.
|
||||
|
||||
**Bundle the `privacyRequired` flag:** forces on-device, no cloud fallback. This is the only piece with a KVARK-funnel angle — surface as a TEAMS/ENTERPRISE-flavored capability ("sensitive tasks never leave the machine"), reinforcing the sovereign narrative with zero KVARK work. Keep it honest: it's a narrative asset, not KVARK itself.
|
||||
|
||||
**Explicitly NOT building:** classification of *arbitrary user-task* complexity — that needs its own (cost-bearing) classifier and risks mis-routing real reasoning to a weak model. The deterministic internal-call subset captures most of the savings with none of the risk.
|
||||
|
||||
**Files:** extend `model-availability.ts:86-109` (`resolveUsableModel` gains a `class` arg); `routes/litellm.ts` (already aggregates 13 providers incl. Ollama); add a model-capability dimension alongside the source dimension in `capability-router.ts`. Quality fallback: if a local result looks like a refusal/garbage, retry on cloud — unless `privacyRequired`.
|
||||
|
||||
**Tier:** routing-to-cheap universal; `privacyRequired` surfaced as a paid-tier capability.
|
||||
|
||||
**Synergy:** B is the prerequisite that makes item C (auto-fetch) cost-safe — its extraction step routes here.
|
||||
|
||||
### C. Auto-Fetch Connector→Memory Loop — PRO-gated (moat + tier trigger)
|
||||
|
||||
**What to build:** a `connector_fetch` cron job type wiring the existing scheduler to the existing harvest pipeline, on a **frequency-capped** schedule (daily, not 20-min), so a user's mind stays current without manual re-harvest. This is the *only* item touching the actual memory+harvest moat — a mind that silently stays fresh is stickier (deeper lock-in) than one that goes stale.
|
||||
|
||||
**Why it clears the bar where metering doesn't:** the grounding names this *the* substrate gap; the infra already exists; and **PRO-gating resolves every objection at once** — it removes FREE proxy exposure, turns "your mind stays fresh automatically" into a concrete **upgrade trigger**, and deepens the **moat** for paying users. Triple fit (moat + tier trigger + cost-safe) — the most on-strategy item in this report.
|
||||
|
||||
**Cost is bounded, not open-ended:** harvest's `harvestSetHash` skips unchanged sources (steady-state cost is only incremental new data), and the extraction LLM routes through §B to the budget model. The expensive first ingest stays user-triggered.
|
||||
|
||||
**Files:** add job type in `packages/core/cron-store.ts`; wire execution in `routes/automations.ts`; invoke `packages/hive-mind-core/src/harvest/pipeline.ts`; gate via tier check.
|
||||
|
||||
**Tier:** PRO+ only. Do **not** ship on FREE.
|
||||
|
||||
### (folded in) Savings telemetry — internal only
|
||||
|
||||
Extend `cost-tracker.ts` (per-model pricing already lives there) with a `tokensSaved` / `by_compressor` / `by_model` accumulator to validate A and B internally. **No user-facing "we saved you N tokens" panel** — that is speculative scope creep. Build only enough to prove the cost arc to the founder.
|
||||
|
||||
---
|
||||
|
||||
## 4. SKIP / DEFER (one-line reasons)
|
||||
|
||||
**SKIP:**
|
||||
- **SuperContext first-turn scout** — redundant; Waggle warm-starts memory synchronously before the LLM (`orchestrator.recallMemory`, Hermes `session-start`). A scout sub-agent adds a round-trip for marginal gain.
|
||||
- **Memory substrate (trees / Obsidian / scoring / E2GraphRAG / pluggable backend)** — at-parity-or-ahead; the SOTA-benchmarked moat. Pluggable backend is a real enterprise-sync gap, but that's precisely what KVARK's sovereign on-prem covers; desktop is local-first by design.
|
||||
- **90k skills aggregation** — a free external firehose undercuts the curated marketplace that *is* the upgrade trigger.
|
||||
- **Voice + lip-sync / Rive mascot** — off-brand for a B2B cockpit + demand-gen funnel; OpenHuman's own analysis calls them commodity.
|
||||
- **Google Meet agent** — multi-quarter Rust CEF/CDP build, breaks on every Meet UI change, different product than a memory cockpit.
|
||||
- **Screen intelligence** — macOS-only, Ollama-heavyweight, commodity OCR+vision; computer-use can wait.
|
||||
- **iOS companion / 18 messaging channels** — mobile is v2+; consumer chat platforms are off-funnel.
|
||||
- **OS keyring** — `vault.ts` (AES-256-GCM, icacls-hardened) is already stronger for server/Docker/KVARK; keyring is end-user convenience, not a moat.
|
||||
- **Theme Studio** — Hive DS brand consistency is a deliberate moat; user theming dilutes it.
|
||||
|
||||
**DEFER (real, but gated):**
|
||||
- **Trigger triage pipeline** — adopt the *design* (drop/ack/react/escalate on a fast model) only once the event-trigger/webhook layer it depends on actually exists (C24 is explicitly schedule-only v1). Blocked on a prerequisite, not on merit.
|
||||
- **Subconscious idle cognition + durable per-thread goals + taint-origin** — genuine capability gap, but idle agent loops **burn Waggle-funded proxy on FREE/TRIAL** — actively *against* the cost discipline that justifies this whole report. Defer until there's a PRO tier-trigger case *and* a quiet-tick/local-eval zero-cost model; that cost model is the real prerequisite.
|
||||
- **MCP live registry discovery** — near-redundant with Composio's on-demand discovery (grounding: Composio "exceeds static-only registries"). Revisit only if catalog staleness becomes a stated sales objection; no evidence it is today.
|
||||
- **Memory-diff (git-backed change tracking)** — a genuinely moat-adjacent idea for a memory product, but L effort and `compliance/` already covers audit/EU-AI-Act; fold into a future compliance sprint.
|
||||
|
||||
---
|
||||
|
||||
## 5. Ranked Top Recommendations
|
||||
|
||||
The bar culls hard. Three items clear it — ranked by strength of case, with scope cut to the bone:
|
||||
|
||||
1. **PRO-gated auto-fetch connector→memory (§3.C).** The only item touching the actual **memory+harvest moat**, not just margin. Grounding-flagged as the sole substrate gap; infra already exists (`cron-store` + `harvest/pipeline`); PRO-gating makes it cost-safe **and** a tier trigger in one move. Highest strategic ceiling. M effort; depends on connectors being connected, so size it as a deliberate PRO-feature bet, not a quick win.
|
||||
|
||||
2. **Tool-result compression — pure-code subset (§3.A).** Lowest effort/risk, cleanest pure-margin cost lever. Verified gap (`agent-loop.ts:511` verbatim append). JSON-table crusher + live search dedup, both **zero added LLM cost**, reusing `harvest/dedup.ts`. Compounds across accumulating turns. **Scope discipline is the whole game: ship the crusher + search dedup + token-aware truncation; reject tree-sitter, the 96-rule overlay, the ML compressor, CCR, and the LLM-summarization fallback.**
|
||||
|
||||
3. **Deterministic routing of internal-lightweight calls (§3.B).** Complementary cost lever via **Haiku-on-proxy** (works for vanilla FREE users, no Ollama needed), and the enabler that makes #1's extraction step cheap. Bundle the `privacyRequired` on-device flag as a free KVARK-sovereignty narrative asset.
|
||||
|
||||
**Folded in, not headlined:** internal savings telemetry via `cost-tracker.ts` — build enough to prove #2/#3, no user-facing panel.
|
||||
|
||||
**Everything else: defer behind explicit triggers (triage, Subconscious, MCP discovery, memory-diff) or skip.** Do not let OpenHuman's impressive-but-off-strategy surfaces (Meet, mascot, voice, screen, mobile, 90k skills) pull scope — real engineering, not Waggle's funnel.
|
||||
|
||||
---
|
||||
|
||||
## 6. Critique Deltas (what I changed vs the draft and why)
|
||||
|
||||
1. **Split the compression layer; dropped the LLM fallback.** The draft bundled a genuine zero-cost win (JSON crusher + trigram search dedup, both pure code) with **token-aware-truncation-with-LLM-summarization**. I demoted the summarization fallback to DEFER because it is (a) largely redundant with Waggle's *existing* message-level compression (`context-compressor.ts` at 50%, `messages-compressor.ts`), and (b) cost-perverse on the target tier — a hard slice is free, a budget-model summary spends proxy tokens, net-positive only for long post-result conversations. The ruthless cut sharpens the rec to its zero-added-cost core.
|
||||
|
||||
2. **Demoted "savings metering" from a co-equal top-3 ADOPT to internal telemetry.** A user-facing "we saved you N tokens" panel is exactly the nice-but-not-differentiating scope creep the founder rejects. Kept only the near-free internal `cost-tracker` accumulator needed to validate the arc. This freed the #3 slot for a real moat item.
|
||||
|
||||
3. **Elevated auto-fetch connector→memory from mid-DEFER to ADOPT (PRO-gated).** This is the biggest change and the one place I make the *strongest* case. The grounding names it the **sole** substrate gap; it's the only candidate touching the actual **memory+harvest moat** rather than margin; PRO-gating eliminates the FREE cost exposure the draft worried about *and* converts it into a **tier trigger**. Triple strategic fit beats every cost-only item. The draft's cost objection is over-stated: `harvestSetHash` bounds steady-state cost, and routing (rec B) makes extraction cheap.
|
||||
|
||||
4. **Scoped routing down to deterministic internal calls; cut the task-complexity classifier.** The draft proposed `classifyTaskComplexity(intent)` over arbitrary user tasks — that needs its own cost-bearing classifier and risks mis-routing real reasoning. I kept only the deterministic allowlist (summarizer, kind-classifier, tool-selection → Haiku/local), and made explicit that **Haiku-on-built-in-proxy** (not Ollama) is the universal FREE-tier lever — the draft over-weighted Ollama, which requires user setup most FREE users won't have.
|
||||
|
||||
5. **Corrected a file pointer.** `agent-loop.ts:504` → **`:511`** per the authoritative grounding ("line 511, r.content added verbatim").
|
||||
|
||||
6. **Re-characterized MCP live discovery as near-redundant with Composio.** Grounding states Composio on-demand discovery "exceeds static-only registries," so Smithery live discovery is closer to SKIP than DEFER — kept DEFER but flagged the redundancy and the lack of any evidence catalog staleness is a real objection.
|
||||
|
||||
7. **Sharpened Subconscious as cost-negative.** The draft deferred it neutrally; I flagged that idle background loops *burn the Waggle-funded proxy on the exact moat tiers*, cutting directly against the cost discipline that justifies the rest of the report — so it's not just "later," it's "not on FREE, ever, without a zero-cost tick model."
|
||||
|
||||
8. **Reframed the honest verdict.** Replaced the draft's "one cluster of three small modules (compression + routing + metering)" with the more accurate and more strategic framing: **two tightly-scoped cost levers + one PRO-gated moat-deepener (auto-fetch), metering folded in.** Same restraint, but the third item now touches the moat instead of being instrumentation.
|
||||
|
||||
9. **Held all SKIPs.** Re-tested every SKIP (voice, mascot, Meet, screen, mobile, keyring, Theme Studio, pluggable backend, 90k skills, scout, substrate) against the bar — all correctly skipped; no false negatives to rescue.
|
||||
Reference in New Issue
Block a user