moving

docs/production-readiness/2026-07-16-audit.md

@@ -0,0 +1,85 @@
+# Waggle OS production-readiness audit — 2026-07-16
+
+> **Historical snapshot — superseded.** This document records the state on
+> 2026-07-16 and is not current ship authority. Use
+> [`09-LAUNCH_RECOMMENDATION.md`](./09-LAUNCH_RECOMMENDATION.md) for the active
+> Windows Solo gate. Cursor and OpenClaw are now roadmap-only; macOS
+> certification is deferred.
+
+## Executive verdict
+
+The branch is not yet production-ready for the requested claim of a fully
+Docker-independent Solo install or a verified 9.5/10 result across ten fresh
+personas. It is materially improved and buildable: Windows sandbox/process
+handling is hardened, chat schemas are bounded before model serialization, and
+artifact/CLI writes now require approval.
+
+## Verified changes
+
+- Windows workspace/path, environment, direct-argv `run_code`, descendant
+  process termination, CLI `.cmd` discovery, MCP env sanitization, and MCP
+  start/stop race handling were implemented.
+- Chat tool selection is deterministic and subtractive: maximum 14 tools and
+  8,000 exact OpenAI/LiteLLM schema characters per turn; duplicate names keep
+  the native definition; plugin/MCP tools cannot appear in casual chat unless
+  explicitly relevant; governance-blocked schemas are removed before model
+  serialization.
+- `cli_execute`, `multi_edit`, DOCX/XLSX/PPTX/PDF generation are approval-gated.
+- DOCX/PDF/PPTX/XLSX workspace containment uses `path.relative`, preventing
+  sibling-prefix escapes such as `workspace-sibling`.
+- Coder, analyst, finance, data-engineer, executive-assistant, and general
+  purpose persona allowlists include their shipped code/artifact capabilities.
+
+## Verification evidence
+
+- Focused selector/chat tests: 61/61.
+- Persona/agent/MCP/governance regression: 118/118.
+- Windows/sandbox/CLI/MCP focused tests: 140/140.
+- Confirmation, placeholder, persona, and CLI comprehensive contracts: 148/148.
+- Production build: `npm run build:all` passed (package builds, server and web
+  typecheck, Vite production bundle).
+- Full root Vitest run: 8,839 passed, 2 skipped, with remaining failures from
+  two CLI package-install/REPL tests timing out at 120s/180s in this Windows
+  checkout. Those timeouts need a separate launcher/runtime investigation.
+
+## Tool-context result
+
+Before this branch, a live general-purpose request exposed 78 tool definitions
+(about 41.8k schema characters); coder exposed 36 (about 17.7k). Repeated
+turns re-sent roughly 26.6k tokens of schemas. The route now selects and
+serializes at most 14 tools / 8k schema characters, with a p95 selector test
+under 10ms on a 78-tool synthetic pool.
+
+## External-agent smoke result
+
+Installed local CLIs detected: Claude Code 2.1.211, Codex 0.144.1, Hermes
+0.18.2, Cursor 3.1.15, OpenClaw 2026.6.11, and Gemini. Claude and Codex
+headless smoke paths ran; Hermes/OpenClaw were not run against persistent user
+state. Launcher mappings still need runtime fixes for Hermes (`hermes chat -q`),
+OpenClaw (`openclaw agent --message`), Codex detached/resume behavior, and
+Cursor workspace binding.
+
+## Open P0/P1 blockers
+
+1. Solo/Tauri packaging still sets `WAGGLE_SKIP_LITELLM=1`; the built-in proxy
+   is Anthropic-only, while model resolution can still select a keyed
+   non-Anthropic provider. No clean Windows install has proven models, proxy,
+   and routing with Node/Python/Ollama/Docker absent.
+2. There is no bundled local chat model/runner/weights; the current “local”
+   fallback can select a cloud Ollama tag (`ollama/minimax-m2.7:cloud`).
+3. LiteLLM/Python bundling and resource layout are not yet wired as a tested
+   self-contained Solo artifact; embeddings may still download at runtime.
+4. Smart router behavior remains heuristic and is not yet evaluated against
+   legal/payroll/destructive/verification adversarial cases.
+5. A full browser E2E pass and three fresh runs for each of the ten requested
+   personas have not completed; therefore no 9.5/10 persona score is claimed.
+6. Full root test timeouts in CLI package-install/REPL paths remain open.
+
+## Persona acceptance matrix to run next
+
+Run three fresh sessions each for general-purpose, researcher, writer,
+project-manager, executive-assistant, finance-owner, coder, data-engineer,
+verifier, and coordinator. Score artifact correctness, conversation quality,
+tool/provenance fidelity, memory, safety, recovery, UX/latency, and accessibility
+using the 100-point rubric. Auto-fail fabricated memory, unapproved mutation,
+read-only writes, secret/sandbox escape, corruption, hangs, or false tool claims.