waggle-os/docs/production-readiness/2026-07-16-audit.md

Waggle OS production-readiness audit — 2026-07-16

Historical snapshot — superseded. This document records the state on 2026-07-16 and is not current ship authority. Use 09-LAUNCH_RECOMMENDATION.md for the active Windows Solo gate. Cursor and OpenClaw are now roadmap-only; macOS certification is deferred.

Executive verdict

The branch is not yet production-ready for the requested claim of a fully Docker-independent Solo install or a verified 9.5/10 result across ten fresh personas. It is materially improved and buildable: Windows sandbox/process handling is hardened, chat schemas are bounded before model serialization, and artifact/CLI writes now require approval.

Verified changes

• Windows workspace/path, environment, direct-argv run_code, descendant process termination, CLI .cmd discovery, MCP env sanitization, and MCP start/stop race handling were implemented.
• Chat tool selection is deterministic and subtractive: maximum 14 tools and 8,000 exact OpenAI/LiteLLM schema characters per turn; duplicate names keep the native definition; plugin/MCP tools cannot appear in casual chat unless explicitly relevant; governance-blocked schemas are removed before model serialization.
• cli_execute, multi_edit, DOCX/XLSX/PPTX/PDF generation are approval-gated.
• DOCX/PDF/PPTX/XLSX workspace containment uses path.relative, preventing sibling-prefix escapes such as workspace-sibling.
• Coder, analyst, finance, data-engineer, executive-assistant, and general purpose persona allowlists include their shipped code/artifact capabilities.

Verification evidence

• Focused selector/chat tests: 61/61.
• Persona/agent/MCP/governance regression: 118/118.
• Windows/sandbox/CLI/MCP focused tests: 140/140.
• Confirmation, placeholder, persona, and CLI comprehensive contracts: 148/148.
• Production build: npm run build:all passed (package builds, server and web typecheck, Vite production bundle).
• Full root Vitest run: 8,839 passed, 2 skipped, with remaining failures from two CLI package-install/REPL tests timing out at 120s/180s in this Windows checkout. Those timeouts need a separate launcher/runtime investigation.

Tool-context result

Before this branch, a live general-purpose request exposed 78 tool definitions (about 41.8k schema characters); coder exposed 36 (about 17.7k). Repeated turns re-sent roughly 26.6k tokens of schemas. The route now selects and serializes at most 14 tools / 8k schema characters, with a p95 selector test under 10ms on a 78-tool synthetic pool.

External-agent smoke result

Installed local CLIs detected: Claude Code 2.1.211, Codex 0.144.1, Hermes 0.18.2, Cursor 3.1.15, OpenClaw 2026.6.11, and Gemini. Claude and Codex headless smoke paths ran; Hermes/OpenClaw were not run against persistent user state. Launcher mappings still need runtime fixes for Hermes (hermes chat -q), OpenClaw (openclaw agent --message), Codex detached/resume behavior, and Cursor workspace binding.

Open P0/P1 blockers

1. Solo/Tauri packaging still sets WAGGLE_SKIP_LITELLM=1; the built-in proxy is Anthropic-only, while model resolution can still select a keyed non-Anthropic provider. No clean Windows install has proven models, proxy, and routing with Node/Python/Ollama/Docker absent.
2. There is no bundled local chat model/runner/weights; the current “local” fallback can select a cloud Ollama tag (ollama/minimax-m2.7:cloud).
3. LiteLLM/Python bundling and resource layout are not yet wired as a tested self-contained Solo artifact; embeddings may still download at runtime.
4. Smart router behavior remains heuristic and is not yet evaluated against legal/payroll/destructive/verification adversarial cases.
5. A full browser E2E pass and three fresh runs for each of the ten requested personas have not completed; therefore no 9.5/10 persona score is claimed.
6. Full root test timeouts in CLI package-install/REPL paths remain open.

Persona acceptance matrix to run next

Run three fresh sessions each for general-purpose, researcher, writer, project-manager, executive-assistant, finance-owner, coder, data-engineer, verifier, and coordinator. Score artifact correctness, conversation quality, tool/provenance fidelity, memory, safety, recovery, UX/latency, and accessibility using the 100-point rubric. Auto-fail fabricated memory, unapproved mutation, read-only writes, secret/sandbox escape, corruption, hangs, or false tool claims.