oleg.m/waggle-os

Fork 0

Files

Oleg Maslov 0c3e2ead3b

Installer Smoke / installer-smoke (push) Has been cancelled

Details

moving

2026-09-02 10:10:29 +02:00

159 KiB

Raw Blame History

Complete UX and Usage Audit - 2026-07-08

Status: Phase 1 implementation completed and verified on 2026-07-08. Follow-up UX trust corrections and verification refreshed on 2026-07-11. The original analysis below is retained as the baseline finding packet; the evidence sections record the current post-fix state.

Objective from the goal: analyze the complete UX and usage scenarios, define everything that should be corrected, then wait for approval before implementation. Target acceptance is five different judge personas scoring the product at least 9/10.

Packet index: docs/audits/2026-07-08-ux-packet-index.md.

Approval brief: docs/audits/2026-07-08-ux-approval-brief.md.

Analysis completion audit: docs/audits/2026-07-08-analysis-completion-audit.md.

Companion execution checklist: docs/audits/2026-07-08-ux-route-scenario-manifest.md.

State and failure matrix: docs/audits/2026-07-08-ux-state-failure-scenario-matrix.md.

Non-main surface scope: docs/audits/2026-07-08-ux-non-main-surface-scope.md.

Five-persona scoring packet: docs/audits/2026-07-08-five-persona-judge-scorecards.md.

Five-persona judge runbook: docs/audits/2026-07-08-five-persona-judge-runbook.md.

Master correction register: docs/audits/2026-07-08-ux-correction-register.md.

Web Guidelines line findings: docs/audits/2026-07-08-web-guidelines-line-findings.md.

Source inventory consistency audit: docs/audits/2026-07-08-source-inventory-consistency-audit.md.

Browser Companion T19 analysis: docs/audits/2026-07-08-browser-companion-t19-analysis.md.

Desktop wrapper T14 analysis: docs/audits/2026-07-08-desktop-wrapper-t14-analysis.md.

Admin/CLI utility T15 analysis: docs/audits/2026-07-08-admin-cli-utility-t15-analysis.md.

AI-tool hook lifecycle T16 analysis: docs/audits/2026-07-08-ai-tool-hook-t16-analysis.md.

Developer/substrate T17 analysis: docs/audits/2026-07-08-developer-substrate-t17-analysis.md.

Ops/deployment/CI/judging T18 analysis: docs/audits/2026-07-08-ops-deploy-ci-judging-t18-analysis.md.

First-run onboarding T1/T2/T12 analysis: docs/audits/2026-07-08-first-run-onboarding-t1-t2-t12-analysis.md.

Shell overlay T10/T12 analysis: docs/audits/2026-07-08-shell-overlays-t10-t12-analysis.md.

Phase 1 implementation plan and verification log: docs/superpowers/plans/2026-07-08-ux-phase-1-corrections.md.

Phase 1 Implementation Evidence

Phase 1 closed the main P0 blockers that were approved for immediate implementation: local accountless Clerk/CSP console health, mobile Settings and first-run onboarding reachability, active Solo/Teams/Enterprise copy, Ctrl+Shift+N plus Workspace Switcher route behavior, visual baseline drift, and codified route evidence for thin Phase 1 judge paths.

Current post-fix verification:

Check	Result	Notes
`npm run typecheck:web`	Pass	Web TypeScript gate.
`npx tsc --noEmit --project packages/server/tsconfig.json`	Pass	Server-side type gate after local rate-limit harness config.
`npm run ux:contrast`	Pass	60 enforced token/surface pairs.
`npm run ux:color-guard`	Pass	96 off-token offenses, all frozen in the existing baseline; no new raw-color violations.
Full `apps/web` Vitest	Pass	Fresh 2026-07-11 run: 207 files, 1726 tests, zero failing tests.
Cache-first return lane	Pass	8 focused files / 71 tests cover instant return repaint plus silent refresh for Home cache, Marketplace, Agents, Connectors, MCP Hub, Artifacts, Approvals, Launcher, and the shared Home cache contract.
Model setup recovery lane	Pass	Shared onboarding + Settings `ModelGate` shows an actionable provider-catalog failure and Retry path instead of an empty key grid; focused ModelGate/onboarding coverage passes 27/27 and web typecheck passes.
API-key rendered setup lane	Pass	Fresh port `34382` Playwright coverage passes 2/2: onboarding and Settings both select a provider, focus the key field, validate, submit the Vault-bound provider payload, show saved feedback, and leave onboarding Continue enabled.
Live provider model catalog lane	Pass	Model inventories are fetched from configured provider APIs, exposed as stable `provider/model` ids, merged into Chat/Spawn, and never filled from a baked-in model list. Claude/Gemini pagination, stale-catalog recovery, app-focus refresh, managed-router hot loading, default/model/fleet resolution, and exact-id completion are deterministic regression gates. One paid live credential request remains external release verification.
Create Workspace mobile disclosure lane	Pass	Focused shell-overlay contracts pass 15/15; the rendered `J-mobile` journey passes 1/1 on fresh port `34380`, proving agent assignment is collapsed by default at 390 x 844, expands on demand, and remains reachable without displacing primary setup.
`npm run build`	Pass	Build succeeds. The prior Tailwind motion-token ambiguity warnings, `shape-selection.ts` dynamic/static import warning, and Vite large-chunk warning are fixed. Latest build is 423.61 kB minified / 114.63 kB gzip for the startup JS chunk; PostHog is lazy-loaded as a separate chunk.
Focused Phase 1 browser lane	Pass	10/10 on port `34146`: console health, wiring, first-run onboarding, mobile onboarding, and cockpit visuals.
`tests/e2e/user-journeys.spec.ts`	Pass	Latest expanded trust-flow run: 24/24 on port `34177`, including mobile Settings, mobile first-run onboarding, Workspace Switcher, route evidence tests, Approvals `J3d`, Artifact `J3e`, Memory Center `J3f`, Wiki export `J3g`, and Settings backup/restore `J3h`. Focused re-checks also passed: mobile first-run Profile reachability 1/1 on port `34194`, mobile Settings 1/1 on port `34195`.
Full combined browser gate	Pass	156/156 on port `34150`: full-product, full-wiring, phase-ab, power-user stress, user journeys, and visual snapshots.
Runtime accessibility axe gate	Pass	2/2 on port `34333`: 41 routes across desktop and 390px mobile have zero axe violations in `tests/e2e/runtime-a11y.spec.ts`; the matrix now includes Settings tabs, Memory tabs, workspaces/tasks, automations, skills, room, channels-adjacent surfaces, marketplace, team, benchmarks, and platform.
T4 shortcut/switcher route contract	Pass	`Ctrl+Shift+N` route contracts passed 2/2 on port `34197`; Workspace Switcher/sidebar traversal guard passed 1/1 on port `34198`.
T5 visual snapshot lane	Pass	`tests/visual/views.spec.ts` passed 14/14 on port `34199` for the seven tracked desktop dark/light views.
T6 marketplace local/audit lane	Pass	Manual sync disable regression passed in Vitest, the manual/background sync slice passed 3/3, `packages/server` typecheck passed, and the focused marketplace Playwright slice passed 4/4 on port `34203`.
Coverage Compass browser integration state	Pass	Browser AI extensions now honestly render as `partial` with popup/capture coverage and native toolbar/context-menu work called out as pending; focused component guard passes 1/1.

Remaining blockers to the final five-persona 9/10 claim are outside the completed Phase 1 scope unless explicitly deferred: broader unsampled accessibility metadata/focus-return cleanup, deeper workflow-state evidence for non-main surfaces, public launch funnel, desktop wrapper/package release evidence, registry/deploy evidence for utility packages, AI-tool packaged desktop hook-status proof, developer/substrate command hygiene, ops/deploy/CI/judge evidence, and Browser Companion native-toolbar/context-menu/rendered-memory/package proof.

Executive Verdict

Waggle OS is broadly functional in the tested built app. The main shell, Home, Chat, Memory, Agents, Library, Settings, workspace routes, and the deeper zone apps are not blank or fundamentally broken. The app has a coherent desktop direction and strong Home continuity.

It is not ready for an honest 9/10 UX judgment yet. The blockers are mostly trust, consistency, responsive layout, and verification quality rather than missing core functionality:

Accountless local Clerk/CSP console health is now verified fixed for sampled first-run and initial-load paths; explicit Clerk-enabled auth still needs its own state evidence.
The original mobile Settings and first-run Profile reachability blockers are now verified fixed, but broader mobile polish evidence still needs ownership for Memory/workspace chat tab strips and the selected overlay path.
Main cockpit pricing/gating copy now uses Solo/Team/Enterprise; remaining active Pro language is limited to explicit legacy billing contexts, while public legal-page Pro copy remains in launch-funnel T13.
The visual regression suite now passes for the seven tracked desktop views in both dark and light modes; it still does not prove full-surface visual quality by itself.
Several destructive and recovery flows use native confirm/alert/prompt dialogs instead of in-app, accessible, branded flows.

Current estimated judge readiness after Phase 1 is materially higher for the main installed cockpit lane, but the final 9/10 claim is still not made. The remaining caps are mostly trust-critical dialogs, accessibility semantics, non-main surface proof, and packaged/release evidence rather than the Phase 1 console, responsive, copy, shortcut, visual, or route-evidence blockers.

The main judge gate is scoped to the installed cockpit experience, but the final "complete UX" claim must also handle public launch/download/checkout/legal, desktop wrapper, installer/update, active sidecar-startup, utility, hook lifecycle, Browser Companion extension capture, developer API/background/substrate, and ops/deployment/CI/benchmark/judging gates. Those non-main surfaces are now mapped in docs/audits/2026-07-08-ux-non-main-surface-scope.md as T13/T14/T15/T16/T17/T18/T19 and must be fixed or explicitly deferred before final completion.

Evidence Collected

Environment:

Repo: D:\Projects\waggle-os
Date: 2026-07-08
Built app server: http://127.0.0.1:3377
Test data dir: temporary WAGGLE_DATA_DIR
Render mode: built Vite app served by local sidecar, LiteLLM skipped, mock embeddings.
Design standard checked against: Vercel Web Interface Guidelines, https://raw.githubusercontent.com/vercel-labs/web-interface-guidelines/main/command.md

Commands and results:

Check	Result	Notes
`npm run typecheck:web`	Pass	No TypeScript errors for the web app.
`npm run ux:contrast`	Pass	60 enforced token/surface pairs passed. Some info-only dim text pairs remain below 4.5.
`npm run ux:color-guard`	Pass	No new off-token text colors beyond the frozen baseline.
Targeted `apps/web` Vitest set	Pass	12 files, 145 tests.
Full `apps/web` Vitest	Pass	174 files, 1575 tests. Warnings are noisy.
`npm run build`	Pass	Build succeeds but main chunk is very large.
Rendered Playwright audit subset	Mixed	51 passed, 15 failed.
Broader non-visual Playwright sweep	Mixed	247 passed, 4 failed, 1 flaky, 1 skipped across 253 tests.
Browser/mobile rendered inspection	Mixed	Home is usable on mobile; Settings is not.
`npm run test -w apps/www`	Pass	Current public-site suite passes 7 files / 19 tests: persona card, Pricing checkout links/recovery, Stripe checkout route cancel URL, layout hydration contract, legal launch-copy guard, controlled download path, mobile/desktop download label detection, and public-site deployment workflow guard.
`npm run build:www`	Pass	Next build succeeds and includes the dynamic public routes. The public-site workflow now uses Vercel prebuilt deployment instead of GitHub Pages/static `apps/www/dist`.
Public-site env-shape check	Mixed	`apps/www/.env.local` has Clerk test keys different from the root Clerk pair and a live-shaped Stripe secret; values were not printed.
Public-site local prod route/API smoke	Improved	Binding `next start` to `127.0.0.1` caused Next middleware proxy failures to `localhost:<port>`. Current post-fix `next start --hostname localhost --port 34205` returned 200 for `/`, `/?checkout=cancelled`, and `/docs/methodology`; signed-out `GET /api/stripe/checkout?tier=teams&billing=monthly` returned 303 to sign-in.
Public-site Playwright interaction smoke	Historical mixed	Earlier regular Playwright against `http://localhost:3416` showed annual toggle and mobile nav worked, while signed-out `Get Team` showed inline `Sign in required`, `/docs/methodology` passed, and `/methodology` plus `/pricing?checkout=cancelled` 404ed. Current Browser/unit evidence supersedes the checkout dead-end and cancel-route findings.
Public-site Browser interaction refresh	Improved	Fresh in-app Browser loaded `http://localhost:34204/?checkout=cancelled#pricing`. Pricing rendered, cancelled-checkout recovery notice appeared, monthly Team CTA and retry link used `/api/stripe/checkout?tier=teams&billing=monthly`, annual toggle updated both links to annual, the Next dev issue badge disappeared after the layout fix, and console warnings/errors were empty. Browser `domSnapshot()` still failed with the known `incrementalAriaSnapshot` plugin mismatch, so evidence used targeted DOM probes plus screenshots.
`npx tsc --noEmit --project app/tsconfig.json`	Pass	Desktop wrapper scripts/config typecheck.
Targeted `app/` Vitest set	Pass	4 files, 85 tests for updater, installer, signing, and runtime bundling helpers.
Local Tauri installer bundle	Pass locally	`npm run tauri:build:local --prefix app` produced fresh MSI and NSIS artifacts on 2026-07-10. Both local artifacts are unsigned debug builds; installed UX, signed release trust, and public availability remain open.
Static Tauri config/update tests	Pass	`npx vitest run packages/server/tests/tauri-config.test.ts app/tests/auto-update.test.ts` passed historically, and the focused `tauri-config.test.ts` refresh now passes 1 file / 19 tests including the release package-before-sidecar guard.
`node scripts/check-sidecar-resources.mjs`	Pass	Staged Node runtime, native deps, and sidecar `node_modules` are present.
Targeted app service E2E set	Pass	3 files, 11 tests for startup, settings, chat SSE, workspace/session, and memory-scope API flows.
Utility/admin surface Vitest slice	Improved mixed	Admin-web package/root tests, launcher package-local tests, root-directed CLI, `@waggle/cli` package-local tests, memory MCP package-local tests, hive-mind MCP package-local tests, marketplace targeted tests, and hive-mind CLI package-local tests pass. Admin-web now owns a package-local test command and its focused suite no longer emits React `act(...)` warnings. Remaining mixed status is from broader rendered/auth/error/publish gaps, not this test lane.
Utility/admin TypeScript and builds	Mixed	`admin-web`, `cli`, `marketplace`, `memory-mcp`, `hive-mind-mcp-server`, and `hive-mind-cli` pass `tsc --noEmit`; all inspected package build scripts pass; `packages/launcher` has no `tsconfig.json` and is built via `tsup`.
Utility CLI help/runtime smoke	Mixed	Source help for launcher, CLI, hive-mind CLI, and marketplace works. Built and clean installed marketplace help/invalid-command paths pass with no DB side effect; built `@waggle/cli` help plus the bin wrapper pass with no `.waggle` side effect; local package-closure installed `npx waggle --help` passes; local package-closure installed `npx waggle --local` starts the REPL, renders the local prompt, runs `/help`, `/mode`, `/whoami`, `/models`, `/cost`, and `/clear`, creates `default.mind`, exits via `/exit`, and completes a streamed chat turn through a mock LiteLLM-compatible endpoint while verifying auth/model/stream payload shape; built and clean installed launcher help/startup-recovery paths no longer print service banners before help or create `.waggle`; and local package-closure installed `hive-mind-cli status --help` passes without `personal.mind` creation.
MCP protocol smoke	Improved mixed	Built `@waggle/hive-mind-mcp-server` and built legacy `waggle-memory-mcp` both complete official MCP client handshakes and write-scope save/recall roundtrips from temp data dirs. Both MCP packages now also have clean local package-closure installed read-only startup proof. Registry-only proof remains open.
Hive-mind CLI and marketplace negative/package paths	Improved mixed	Hive-mind CLI sampled subcommand help now prints focused help and does not create `personal.mind` from source, built, and local package-closure installed paths. Marketplace unknown commands now print help, exit 1, avoid DB creation, and its package manifest points at emitted files included by `npm pack --dry-run`; installed packed-CLI help and invalid-command paths are guarded. Every-subcommand sampling and registry-only proof remain open.
Admin-web rendered package smoke	Pass	`npm run test:rendered --workspace @waggle/admin-web` builds the package and runs Playwright against the built preview. It covers all seven admin pages at desktop and 390px mobile widths with typed authenticated mock API data, real local bearer-auth middleware wrong-token/valid-token behavior through protected Fastify routes, hash URL state, `aria-current`, document scroll width, labelled table scroll regions, overflow checks, labelled controls, clean app console/pageerror collection, mobile shell keyboard navigation, page-level keyboard traversal from connection fields into dashboard, members, capabilities, jobs, audit, and settings controls/table regions, full-page desktop/mobile visual snapshots, browser back/forward hash traversal, capability governance edit/add/decision forms, malformed analytics response recovery, all-page initial API-failure recovery with accessible alerts, and rendered mutation/destructive-failure recovery for capability policy save, capability override create/remove, capability request decision, member invite, member role change, member removal, and team settings save. Live deployed auth remains a launch/deploy evidence item, not an admin-web package-local blocker.
AI-tool hook route/contract tests	Pass with warning noise	Shared/agent/server T16 slice plus package runtime proof passed 7 files / 96 tests, with an additional registry-aware adapter launch slice passing 5 files / 108 tests. It covers manifests, third-party manifest loader, hook-capable cohort, launcher, backend tool routes, process paths, launchable third-party adapters, prompt template application, and packed package `npx` install/verify/uninstall lifecycle for all six hook-capable packages. Server setup emits expected mock embedding warnings.
Launcher hook UI, prompt, and adapter tests	Pass with deprecation warning	`npm run test -w apps/web -- src/components/os/apps/LauncherApp.test.tsx src/lib/launcher-prompt-args.test.ts src/lib/adapter.launcher.test.ts --reporter=dot` passed 3 files / 28 tests and emitted Node `punycode` warnings.
Launcher rendered hook state smoke	Pass, codified plus Browser partial	Fresh `npm run build` passed, then the in-app Browser loaded `/launcher?watch=1&skipOnboarding=true&skipBoot=true&skipBriefing=true` against a mock local API on port 54147. It rendered installed/not-installed/hooks-active/running/Phase 4 states, prompt summary, install success, verify failure, and live output with no current-port console errors. Artifacts: `output/playwright/launcher-t16-54147/`. Browser DOM snapshot failed with `TypeError: o.incrementalAriaSnapshot is not a function`, so evidence used targeted DOM evaluation plus screenshots. Codified Playwright rendered coverage now also proves all six hook-capable cards through install, verify, uninstall, and refreshed hooks-active transitions.
AI-tool hook/shim package tests	Pass with warning noise	Root-run hook/shim tests passed 60 files / 573 tests with 1 skipped. Output includes expected fail-open warnings, sidecar-unreachable signal drops, package logs, and hook verify negative-path logs.
AI-tool hook/shim typechecks, bin smokes, and package-local tests	Pass with warning noise	Official package typechecks pass 8/8, Claude Desktop stub build passes, 6/6 compiled hook-bin help smokes pass, local package-pack `npx --yes @waggle/hive-mind-hooks-<id> install/verify/uninstall` lifecycle passes for all six hook-capable packages, and hook/shim package-local `npm run test --workspace ...` scripts pass. The shim package-local integration lane also verified the CLI ESM resolver fix for the MCP server entry. Registry-only proof after actual package publication remains external launch/deploy evidence.
First-run onboarding smoke	Mixed	`npm run build` passed and a clean-data sidecar on port 3431 completed desktop onboarding through first-task auto-send into workspace chat; the original mobile pass found Profile Continue below the 390 x 844 viewport and T1 Clerk/CSP console errors. Current focused checks are green: mobile Profile reachability passed 1/1 on port `34194`, sampled accountless console health passed 3/3 on port `34196`, high-volume import CTA coverage passed 2/2, model-ready copy passed 5/5, and first-task composer handoff passed in the 12/12 chat input suite.

Rendered Playwright details:

Passed: API health checks, desktop shell, Home, Chat, Room, Agents, Files, Approvals, Settings, Vault, Mission Control, Timeline, Usage, Events, Team Governance, Skills Hub, Connector Hub, MCP Hub, Marketplace, and user journey checks J1-J12.
Historical failure now rechecked: tests/e2e/full-product-audit.spec.ts "no critical console errors on load" passed in the focused 3/3 console-health bundle on port 34196.
Historical failure now rechecked: all 14 visual snapshots in tests/visual/views.spec.ts originally failed for the 7 tracked views across dark and light; the current visual suite passes 14/14 on port 34199.
Visual suite scope remains 7 views only (chat, memory, events, skills/capabilities, home, mission-control, settings). The route registry exposes roughly 30 app routes/route wrappers, so the green visual suite does not by itself prove full-surface visual quality.

Broader Playwright sweep details:

Command: node node_modules/playwright/cli.js test tests/e2e/full-wiring-audit.spec.ts tests/e2e/waggle-complete.spec.ts tests/e2e/user-behavior.spec.ts tests/e2e/power-user-stress.spec.ts tests/e2e/phase-ab-verification.spec.ts tests/e2e/polish-verification.spec.ts tests/e2e/light-mode-polish.spec.ts tests/e2e/boot-screen-skip.spec.ts --project=chromium --reporter=list
Environment: fresh built server on http://127.0.0.1:3388, WAGGLE_E2E_SKIP_LITELLM=1, mock embeddings, temporary data dir.
Result: 247 passed, 4 failed, 1 flaky, 1 skipped.
Historical failures now rechecked:
- tests/e2e/full-wiring-audit.spec.ts:553: original workspace-switcher traversal interception now passes in traverse all sidebar views - zero critical JS errors on port 34198.
- tests/e2e/phase-ab-verification.spec.ts:132 and tests/e2e/power-user-stress.spec.ts:215: original Ctrl+Shift+N route failures now pass 2/2 on port 34197.
- tests/e2e/phase-ab-verification.spec.ts:271: original inline-script CSP console error now passes in the focused 3/3 console-health bundle on port 34196.
Flaky:
- tests/e2e/full-wiring-audit.spec.ts:501: marketplace search API returned non-OK once, then passed on retry.
Important harness notes:
- Historical result: the run triggered a live marketplace sync via POST /api/marketplace/sync; that path took 34.4s and logged external-source errors despite the harness disabling background marketplace sync.
- Current recheck: manual /api/marketplace/sync now respects WAGGLE_DISABLE_MARKETPLACE_SYNC=1, and the focused marketplace Playwright slice passed 4/4 on port 34203.
- A "without LLM" graceful-degradation test still used the local Ollama provider because it was healthy.
- Cost tracking warned that ollama/minimax-m2.7:cloud has no pricing entry and was estimated with Sonnet pricing.
- The test server was stopped after the run; no listener remained on port 3388.

Route/test coverage notes:

apps/web/src/App.tsx registers the app shell plus /auth, /home, /workspaces, /workspaces/:workspaceId/:tab?, /memory/:mindScope?, /artifacts, /files, /agents, /automations, /skills, /room, /waggle-dance, /approvals, /connectors, /mcps, /marketplace, /launcher, /team, /settings, /settings/vault, /settings/profile, /settings/mission-control, /settings/timeline, /settings/events, /settings/usage, /benchmarks, /platform, /payment-success, /payment-cancelled, and catch-all.
Simple route-string search in tests/ plus apps/web/src/test shows heavy coverage for /home, /workspaces, /memory, /marketplace, /settings, and /skills.
Thin route-level coverage is now codified for the priority gaps: /payment-cancelled, /benchmarks, /platform, /launcher, /launcher?watch=1, /waggle-dance, /artifacts, /settings/profile, /settings/timeline, /payment-success, /automations, /mcps, /settings/usage, and /files have J-route-coverage owners, with the focused route coverage run passing 2/2 on port 34200. Deeper workflow/state owners remain tracked separately under T10/T12/T16.
Source inventory also found app files that are embedded or retired rather than top-level routes: DashboardApp.tsx and VoiceApp.tsx are unrouted/retargeted, MissionControlApp.tsx is superseded by CockpitApp, and BackupApp.tsx plus the files/storage subapps are judged through Settings or /files.
Existing five-persona harness: tests/vision/personas.spec.ts verifies live persona chat workspaces, persistence, screenshots, and no cross-persona leakage, but it does not score every routed surface or the requested 9/10 UX rubric.
Companion manifest: docs/audits/2026-07-08-ux-route-scenario-manifest.md converts these coverage gaps into a route-by-route and scenario-by-scenario evidence checklist.
State/failure matrix: docs/audits/2026-07-08-ux-state-failure-scenario-matrix.md converts account, tier, disclosure, model, data, offline, destructive, responsive, accessibility, and performance state combinations into judge evidence requirements.
Judge scorecards: docs/audits/2026-07-08-five-persona-judge-scorecards.md defines the scoring caps, required screenshots, route evidence, and correction mapping for the five-persona 9/10 gate.
Judge runbook: docs/audits/2026-07-08-five-persona-judge-runbook.md turns the scorecards into an executable evidence protocol with state bundles, route sequences, required screenshots, score caps, deferral records, and an evidence folder shape.
Source inventory consistency audit: docs/audits/2026-07-08-source-inventory-consistency-audit.md rechecks current route, command, app, package, public-site, and Browser Companion extension inventory against source.
Route evidence T11 analysis: docs/audits/2026-07-08-route-evidence-t11-analysis.md adds the AppShell route registry, direct route-reference matrix, route-table command evidence, component evidence for command-only routes, current all-route built-preview smoke evidence, and zero/thin route correction candidates.
Mobile Executive T2/T12 analysis: docs/audits/2026-07-08-mobile-executive-t2-t12-analysis.md adds 390 x 844 screenshots for Home, Settings general/models/billing/profile, Memory, workspace chat, Command Center, and Workspace Switcher. The original pass proved critical visible controls can clip even when document-level horizontal overflow is clean; current focused Settings verification passes 1/1 on port 34195.
First-run onboarding T1/T2/T12 analysis: docs/audits/2026-07-08-first-run-onboarding-t1-t2-t12-analysis.md adds clean-data desktop and mobile onboarding evidence without skip flags. It confirms the desktop path is functionally complete; sampled T1 console health, mobile Profile reachability, high-volume import CTA risk, model-ready copy, and first-task composer handoff are now focused fixed.
Shell overlay T10/T12 analysis: docs/audits/2026-07-08-shell-overlays-t10-t12-analysis.md adds fresh desktop/mobile overlay smoke evidence for Keyboard Shortcuts, Persona Switcher, Spawn Agent, Workspace Switcher, Notification Inbox, Create Workspace, and Upgrade Modal, plus source-only findings for Context Rail, Onboarding Tooltips, and Trial Expired Modal.
Browser Companion T19 analysis: docs/audits/2026-07-08-browser-companion-t19-analysis.md adds extension syntax/manifest/server typecheck evidence, disconnected/connected popup screenshots, direct sidecar save evidence, current toolbar/extraction probes, Memory UI confirmation, CORS/auth findings, and line-level T19 findings.
Desktop wrapper T14 analysis: docs/audits/2026-07-08-desktop-wrapper-t14-analysis.md adds refreshed Rust cargo check, resource/runtime evidence, app TypeScript evidence, web-side Tauri test command evidence, app service E2E evidence, and a native event consumer matrix.
Public launch funnel T13 analysis: docs/audits/2026-07-08-launch-funnel-t13-analysis.md adds www test/typecheck/build evidence, fresh localhost route/API smoke, rendered Browser checkout-recovery evidence, live canonical-domain DNS failure evidence, GitHub Releases download-target evidence, local checkout fixes, and line-level launch-funnel correction candidates.

Non-main surface scope notes:

Public site apps/www is a real user-facing launch funnel, not just marketing decoration. It has a coherent homepage, skip link, mobile menu semantics, Solo/Team/Enterprise pricing, GET-based Team checkout CTA, cancelled-checkout recovery notice, guarded legal placeholder copy, OS-aware download CTA, historical Lighthouse evidence, and focused component/route tests.
Public site command evidence now shows 7-file component/route/legal/download/deployment tests, direct TypeScript, and npm run build:www passing. The download guard includes mobile/desktop OS-label detection so iOS/Android visitors are not promised desktop installers. The prior GitHub Pages/static artifact mismatch is fixed locally: deploy-www.yml now uses Vercel production pull/build/deploy for the dynamic Next app.
Public site local production route smoke is host-sensitive: --hostname 127.0.0.1 caused Next middleware proxy failures to localhost:<port> on Windows, while --hostname localhost rendered the homepage, legal pages, auth pages, /download, /docs/methodology, and unauthenticated checkout/account redirects. The fresh Browser refresh on localhost:34204 had 0 console warnings/errors for the fixed pricing recovery state. Real issues remain: canonical DNS failure, missing signed installer/release artifacts, missing deployed Vercel/DNS smoke, formal legal sign-off, and missing deployed Clerk/Stripe checkout evidence.
Public site gaps found by source inspection and current smoke: the canonical waggle-os.ai domain does not currently resolve from the audit environment; a controlled local /download status page has replaced the empty GitHub Releases dead end, but real signed installer artifacts are not published; deployment target and real deployed checkout coverage are still thin; formal legal sign-off is not proved. Locally fixed: signed-out Team checkout now uses GET auth continuation, checkout cancel recovery points to /?checkout=cancelled#pricing with a retry notice, legal Day-0/stale Pro placeholder copy is guarded, the public Download CTA no longer points to releases/latest, mobile/tablet visitors keep a generic Download label instead of a false desktop OS label, and the intentional html.js hydration mismatch no longer raises a Next dev issue badge.
Desktop wrapper app/ has strong build-resource preflights, static tests, TypeScript checks, Rust cargo check, web-side Tauri binding tests, passing service-level startup/chat/workspace E2E, a release workflow guard that builds packages before sidecar packaging, a focused tray-source guard, and a fresh local debug bundle that produced both MSI and NSIS artifacts. Native source directly handles tray Open/focus, close-to-tray, Ctrl+Shift+W window visibility, and tray Quit; Settings is bridged to /settings; unsupported Pause/About tray actions are hidden. Installed-app UX evidence, signed release trust, published artifact availability, and packaged tray/service/update interaction proof remain open.
Source search now leaves waggle://update-available, waggle://service-status, and waggle://service-restart-needed as the remaining unconsumed native event families. Tray waggle://navigate has a web bridge for /settings; waggle://pause-agents, waggle://quit, and /about tray emissions were removed or replaced with native handling.
The active installed sidecar path is packages/server/src/local/service.ts bundled by scripts/build-sidecar.mjs, staged into app/src-tauri/resources/service.js, and launched by app/src-tauri/src/service.rs. The top-level sidecar/ JSON-RPC code should not be treated as the primary installed desktop runtime unless a current launch path imports it.
Admin web and CLI/MCP utility surfaces are real user-facing/admin paths: packages/admin-web, packages/cli, packages/launcher, packages/marketplace, packages/memory-mcp, packages/hive-mind-mcp-server, and packages/hive-mind-cli. T15 now has its own supplement: docs/audits/2026-07-08-admin-cli-utility-t15-analysis.md.
Utility evidence is no longer merely shallow: marketplace CLI built/installed help/invalid-command behavior plus package manifest/packed-file alignment, launcher built help/invalid-port/occupied-port recovery plus packed first-command, clean installed occupied-port startup recovery, and clean installed long-running /health startup, built, packed, and local package-closure installed @waggle/cli npx help plus installed local REPL startup/slash-command/exit and streamed chat/provider plumbing, sampled hive-mind CLI source/built/installed subcommand help, legacy memory MCP read/write startup plus installed read-only startup, hive-mind MCP write-scope roundtrip plus installed read-only startup, and admin-web unit plus rendered package coverage with local bearer-auth behavior are now locally fixed and guarded. Registry-only proof after internal package publication remains open.
AI-tool hook packages are real user-facing integration surfaces through Launcher hook install/verify/uninstall. Current package and manifest tests are strong, local packed-package npx lifecycle now proves install/verify/uninstall for all six hook-capable packages, focused rendered Launcher smoke covers mixed hook states, a codified rendered Playwright spec covers every hook-capable tool's install/verify/uninstall UI transitions, a gated real-tool Playwright smoke proves OpenClaw rendered detection plus observed launch/output/exit/process cleanup, and a gated route Playwright smoke proves real /api/tools/hooks install/verify/uninstall for all six hook-capable tools against an isolated profile with cleanup. The packet still has not proved packaged desktop integration for hook status transitions. claude-desktop is intentionally the only non-hook-capable built-in; packages/hive-mind-hooks-claude-desktop is a stub/no-bin package, while codex-desktop is a thin re-export of the Codex hook package.
Developer API/background/substrate now has its own supplement: docs/audits/2026-07-08-developer-substrate-t17-analysis.md. Current evidence is broad and mostly green, including 13/13 direct no-emit typechecks, agent/core/optimizer/weaver package tests, root-run substrate/SDK/shared/WaggleDance/worker/compiler tests, a dedicated server performance lane passing 13/13, a named server release lane passing 185 files / 2128 tests with one worker, hermetic marketplace-sync coverage, and a fixed Playwright webServer startup path. T17 remains open for warning hygiene and developer recovery journeys that are not fully sampled.
Ops/deployment/CI/benchmark/judging now has its own supplement: docs/audits/2026-07-08-ops-deploy-ci-judging-t18-analysis.md. Config syntax, secret tracking, safe Compose scans, benchmark TypeScript, package-local benchmark tests (29/325), production Compose fail-closed credential guards, explicit Render sidecar-mode guards, root-run benchmark tests, the public-site Vercel workflow guard, and a new blocking five-journey browser smoke pass; deployed Vercel/DNS proof, no live infra lane, LiteLLM live routing, and stale judge artifacts keep T18 open.
Browser Companion extension apps/browser-ext is a real capture surface: a Chrome MV3 popup/content/background flow that saves current page or selection into memory through the local sidecar and GET /api/browser-ext/health. Current July evidence now proves syntax, manifest shape, server typecheck, direct sidecar frame creation, content-script extraction from a normal page, secure-default extension token bootstrap during save, background save through chrome.runtime.sendMessage, /api/memory/frames imported-frame confirmation, direct popup Save selection/Save page button-click saves, context-menu handler behavior, Memory search provenance consistency for imported captures, and rendered Memory UI confirmation after secure popup saves. T19 still does not prove native toolbar-bubble exposure, native context-menu click behavior, packaged pairing, or any separate global-recall provenance shape if it enters scoring.
These findings are tracked as T13, T14, T15, T16, T17, T18, and T19 in the correction register. They are not Phase 1 unless a Phase 1 verification command directly forces a small supporting fix, but they do block the final "complete UX" claim unless the user explicitly defers them from the score.

Build warnings:

Fixed 2026-07-09: Tailwind ambiguous motion-token classes duration-[var(--mo-base)], duration-[var(--mo-fast)], and ease-[var(--mo-ease)] were replaced with named motion utilities and guarded by motion-class-hygiene.test.ts.
Fixed 2026-07-09: shape-selection.ts is no longer dynamically imported by the adapter while also being statically imported by Settings; build-warning-hygiene.test.ts guards this boundary.
Fixed 2026-07-09: route surfaces, closed shell overlays, ChatHost, and PostHog analytics are lazy-loaded. The startup JS chunk dropped from index-BtS0I3SX.js at 1,847.38 kB minified / 495.99 kB gzip to index-D-wAFouW.js at 421.96 kB minified / 114.08 kB gzip, and the Vite large-chunk warning no longer appears.
Several persona image assets are roughly 395 to 653 kB each.

Static Web Interface Guidelines scan:

Line-level supplement: docs/audits/2026-07-08-web-guidelines-line-findings.md.
Native browser dialogs in trust-critical UI: current follow-up scan finds 0 high-confidence production confirm/alert/prompt calls after excluding comments, sanitizer tests, and local callbacks named confirm.
Scoped production UI scan found: 34 transition-all hits, 57 outline-none/focus:outline-none hits, 15 autoFocus hits, 11 raw <img> hits in cockpit UI, and 114 new Date(...) / toLocale* hits needing user-visible locale review.
transition-all appears 34 times in scoped cockpit/admin/public UI. Animation polish should replace broad transitions in touched user-facing surfaces with explicit properties.
outline-none appears 57 times in the original scoped scan; many have focus-ring replacements, and the supplement now lists 0 current high-confidence weak or missing visible focus replacements in the reviewed list after focused fixes.
No user-scalable=no, maximum-scale=1, or paste-blocking onPaste hits were found.
Static <img> search found all inspected hits have alt, but image elements commonly rely on CSS sizing instead of explicit width/height; above-fold and repeated persona/logo images should be included in the performance/CLS pass.
Loose form metadata scan found 145 text-like controls missing name, 107 missing autoComplete, and 115 without an obvious same-tag id/ARIA label hook. Treat these as a historical T10 audit queue, because custom Input components and wrapping labels require contextual inspection; regenerate the counts after the focused fixes before final closeout.
Implementation update 2026-07-08 T10: Settings model/trust/team/KVARK controls, Profile identity/preferences/brand fields, Chat composer, Vault add-secret controls/actions, Launcher refresh/prompt, Approvals refresh/revoke, Cockpit refresh, WaggleDance refresh, ComplianceDashboard report actions, AgentCard custom delete, All Workspaces search, Wiki search, and Files storage/browser scroll regions now have focused accessible-name/metadata/focus fixes and tests.
Implementation update 2026-07-09 T10: WorkspaceActionsMenu rename/delete confirmation inputs now have explicit accessible labels, name, autocomplete="off", and visible focus rings guarded by workspace-actions-menu.test.tsx; the focused menu test now passes 13/13 without React act(...) warnings.
Implementation update 2026-07-09 T10: Command Center search now has a visible token focus ring guarded by p7-b3-command-center.test.tsx; the focused Command Center test passes 6/6.
Implementation update 2026-07-09 T10: warm AskBar input now has name, autocomplete="off", and a visible token focus ring guarded by warm-primitives.test.tsx; the focused warm primitive test passes 13/13.
Implementation update 2026-07-09 T10: workspace TasksTab add-task input now has aria-label, name, autocomplete="off", and a visible token focus ring guarded by workspace-tasks-tab.test.tsx; the focused TasksTab test passes 5/5.
Implementation update 2026-07-09 T10: Timeline event-type filter now has aria-label, name, hidden decorative icon semantics, and a visible token focus ring guarded by timeline-app.test.tsx; the focused Timeline test passes 1/1.
Implementation update 2026-07-09 T10: Create Workspace template creator Description and Starter Memory textareas now have associated labels, name, autocomplete="off", and visible token focus rings guarded by shell-overlay-contracts.test.tsx; the focused shell overlay contract passes 10/10.
Implementation update 2026-07-09 T10: Workspace Switcher focus movement, Tab trap, Escape close, and focus return are now guarded by shell-overlay-contracts.test.tsx; the focused shell overlay contract passes 10/10.
Implementation update 2026-07-09 T10: MissionControlApp refresh/pause/resume/stop icon-only controls now have accessible action names and visible token focus rings guarded by mission-control-a11y.test.tsx; the focused Mission Control test passes 1/1.

Runtime accessibility smoke:

Focused supplement: docs/audits/2026-07-08-runtime-a11y-t10-analysis.md.
Fresh built-app axe/DOM smoke on port 3423 rendered /home, /settings, /settings/profile, /memory, /workspaces/default-workspace/chat, /launcher, /mcps, /files, /approvals, and Command Center in desktop and 390 x 844 mobile contexts with the standard E2E skip harness.
Runtime findings from the pre-fix smoke: Settings had critical button-name and select-name axe findings; Profile had critical select-name; Launcher and Approvals had critical unnamed icon buttons; Files had a serious keyboard-inaccessible scroll region; workspace chat had tabpanel semantics and image/preview accessible-text findings; Command Center opened/focused/closed with Control+k/Escape but logged a dialog-description warning and overflowed long subtitles on mobile. These sampled findings are now fixed in source and focused tests, and the expanded runtime axe gate is green across 41 desktop/mobile routes.
The first axe attempt without skip parameters is retained only as a harness caveat because desktop routes can render first-run/auth state instead of the intended app surface.

Judge Personas

The five judges should test the product as distinct users, not as abstract reviewers. Each judge scores 10 points, with no category below 8 allowed for a pass.

Judge	Primary scenario	Current estimate	Why not 9 yet
Solo founder/operator	First run, accountless mode, model setup, Home Start Here, create workspace, first chat	8.6	Core flow works, sampled accountless console health is green, and first-run import/model/first-task handoff polish is focused fixed; remaining caps are legacy pricing-copy/state evidence and broader unsampled failure-state proof.
Power developer/agent wrangler	Command center, agents, launcher, skills, connectors, MCPs, files, keyboard workflows	8.1	Core routes and keyboard contract now pass; remaining caps are marketplace/search determinism, broader keyboard/focus proof, and non-main integration evidence.
Team admin/security reviewer	Billing, Team, Approvals, Vault, governance, audit/compliance, restore/backup	7.8	Known production native dialogs, sampled console health, and active cockpit pricing copy are fixed; remaining caps are explicit auth-enabled evidence, launch/legal copy, and governance failure-state depth.
Memory-heavy researcher	Home continuity, Memory Center, search, frames, wiki, provenance, timeline	8.5	Functional and well covered; Artifact permanent delete, Memory Center delete/erase/allow re-import, Wiki export destinations, and seven-view visual snapshots are green. Remaining caps are broader persona-bundle coverage and deeper Memory/chat mobile-state proof.
Mobile/tablet executive	Review Home, settings, billing/profile, light/dark, touch/narrow viewport	7.4	Home renders, Settings is now green in the focused 390px route check, first-run Profile Continue reachability is green, and Command Center has a described, non-overflowing 390px keyboard path. Remaining cap: Memory/chat tab-strip ownership plus broader touch-state evidence.

Target pass condition:

All five judges score at least 9/10.
No P0 findings remain.
Full web Vitest, build, rendered audit, and visual snapshots are green or have explicitly approved baseline updates.
Manual desktop and mobile screenshots show no text clipping, incoherent overlap, or unusable navigation.
Five-persona live judge gate runs and produces at least 9/10 for every persona. Existing candidate harness: tests/vision/personas.spec.ts, which drives five persona workspaces against a real local LLM and writes persona artifacts under tests/vision/artifacts/personas.

Route and Surface Coverage Matrix

This matrix is the current route/app inventory used for coverage. "Evidence" means at least one current test or manual rendered inspection touched the surface in this audit pass; it does not mean the surface is 9/10.

Surface	Route	Primary component	Main user job	Evidence	Correction focus
Auth/accountless	`/auth`	`AuthRoute` + Clerk/accountless screens	Sign in or continue local-first	Focused accountless console-health checks pass 3/3 on port `34196`; auth component tests cover no-key and explicit-enabled key shape	Explicit Clerk-enabled rendered state evidence.
Home	`/home`	`HomeCockpit`	Daily briefing, Start Here, recent work	Rendered desktop/mobile + tests	Preserve as anchor surface.
All workspaces	`/workspaces`	`AllWorkspacesApp`	Browse/create/manage workspaces	Route inventory + passing tests	Modal/native confirm polish.
Workspace	`/workspaces/:workspaceId/:tab?`	`WorkspaceDesktopApp` + `ChatSlot`	Chat, files, memory, tasks per workspace	Passing journey/API tests	Shortcut contract, workspace context clarity.
Memory	`/memory/:mindScope?`	`MemoryCenterApp`	Search, trust, wiki, provenance, evolution	Passing tests, rendered `J3f` delete/erase/re-import confirmations on port `34164`, rendered `J3g` Wiki export forms on port `34167`, and expanded user journeys pass 24/24 on port `34177`	Visual baseline, form/a11y pass, failure states, and broader persona-bundle evidence.
Artifacts	`/artifacts`	`ArtifactCenterApp`	Agent-produced documents/files	Passing smoke plus rendered `J3e` permanent-delete confirmation on port `34161`; latest expanded user journeys pass 24/24 on port `34177`	Archive/empty/error states and broader persona-bundle evidence.
Files	`/files`	`StorageAndFilesApp`	Browse workspace files/storage	Passing smoke/stress; cross-workspace file copy now passes API and drag-interaction regression tests	Folder cross-workspace copy is explicitly constrained to files; file action confirmations and empty states remain in the broader judge lane.
Agents	`/agents`	`AgentsApp`	Create/manage agents/groups/templates	Passing smoke/stress	Form labels, async test warnings.
Automations	`/automations`	`AutomationCenterApp`	Scheduled jobs	Passing unit/route tests; delete confirmation now has focused component evidence	Builder form semantics and broader automation states.
Skills	`/skills`	`CapabilitiesApp`	Installed skills and skill marketplace	Passing smoke; marketplace flake	Pricing copy, marketplace loading determinism.
Room	`/room`	`RoomApp`	Watch agents work together	Passing smoke	Empty/running state polish.
WaggleDance	`/waggle-dance`	`WaggleDanceApp`	Shared signal/swarm view	Route/API coverage	Clarify discoverability and value copy.
Approvals	`/approvals`	`ApprovalsApp`	Review agent actions/grants	Passing smoke plus rendered `J3d` revoke-all confirmation on port `34159`; latest expanded user journeys pass 24/24 on port `34177`; refresh and per-grant revoke icon buttons have focused accessible-name coverage	Approve/deny and individual revoke states still need final persona-bundle evidence.
Connectors	`/connectors`	`ConnectorsApp`	Connect external apps/accounts	Passing smoke/API	Credential setup clarity, error state polish.
MCP Hub	`/mcps`	`MCPHubApp`	Manage MCP servers	Passing smoke	Pro/Solo/Team copy, custom MCP form labels.
Marketplace	`/marketplace`	`MarketplaceApp`	Browse/install extensions	Passing smoke; one API flake	Pro copy, live sync determinism.
Launcher	`/launcher`	`LauncherApp`	Launch linked AI tools/hooks	Current all-route smoke renders `/launcher` and `/launcher?watch=1`; focused T16 Browser smoke renders mixed hook states against a mock local API; Playwright rendered coverage proves standard install changed-file/pointer/backup/recovery labels, all six hook-capable install/verify/uninstall transitions, sidecar-offline Retry, long-stderr summarization, non-built-in adapter launch-only/prompt behavior, and real OpenClaw observed launch/output/exit/process cleanup; refresh and optional prompt controls have focused accessible-name/metadata coverage; packed-package `npx` lifecycle is proven for all six hook-capable packages; gated route evidence proves real `/api/tools/hooks` install/verify/uninstall for all six hook-capable tools against an isolated profile	Packaged app hook-status integration remains.
Team	`/team`	`TeamGovernanceApp`	Team rules/permissions	Passing smoke/API	Gating copy, admin trust states.
Settings	`/settings`	`SettingsApp`	General/model/billing/backup/system settings	Passing desktop/mobile focused checks plus rendered `J3h` backup/restore trust path on port `34171`; latest expanded user journeys pass 24/24 on port `34177`; Prompt Shape, telemetry, daily budget, mutation gate, team, and KVARK controls have focused accessibility metadata coverage	Deeper mobile overlay paths, remaining settings tabs/states, and persona-state evidence.
Vault	`/settings/vault`	`VaultApp`	Store API keys/secrets	Passing smoke/API	Error/success states, labels.
Profile	`/settings/profile`	`UserProfileApp`	Identity/profile preferences	Runtime axe gate and focused tests cover identity, preference, brand, writing-style, and language controls	Broader profile-state edge cases.
Mission Control	`/settings/mission-control`	`CockpitApp`	Health/cost/activity cockpit	Runtime axe gate covers route; Cockpit refresh and ComplianceDashboard actions are named	Visual baseline and responsive fit.
Timeline	`/settings/timeline`	`TimelineApp`	Activity history	Passing smoke	Empty/loading state polish.
Events	`/settings/events`	`EventsApp`	Live logs/events	Passing smoke; visual failure	Filter ergonomics and visual baseline.
Usage & Cost	`/settings/usage`	`TelemetryApp`	Spend/token visibility	Current all-route smoke renders Team-tier gate but logs a 403 resource error	Unknown local model pricing entry and expected-vs-error resource semantics.
Benchmarks	`/benchmarks`	`BenchmarkApp`	Capability/memory benchmark view	Component tests plus codified `J-route-coverage` route smoke passing on port `34200`	Non-sidebar discoverability and benchmark interpretation depth.
Platform	`/platform`	`PlatformApp`	Platform/roadmap info	Component tests plus codified `J-route-coverage` route smoke passing on port `34200`	Non-sidebar discoverability and judged-scope/value clarity.
Payment success	`/payment-success`	`PaymentSuccessApp`	Checkout return state	Current all-route smoke renders no-checkout fallback	Legacy Pro display context and completed Teams checkout-return state.
Payment cancelled	`/payment-cancelled`	redirect to Settings billing	Recover from cancelled checkout	Codified `J-route-coverage` proves redirect to `/settings?tab=billing` and billing recovery copy on port `34200`	Completed checkout-cancel recovery state depth.
Not found	`*`	`NotFound`	Recovery from bad route	Current all-route smoke renders branded recovery but logs the expected bad route as a console error	Keep recovery link visible and classify expected 404 logging outside critical console failures.
Command Center	`Ctrl+K` overlay	`CommandCenter`	Search/routes/actions	Passing journey/stress plus described/no-overflow 390px mobile proof	Pro wording in pinned group.
Workspace Switcher	Sidebar/Ctrl+Tab overlay	`WorkspaceSwitcher`	Pick/create workspace	T4 traversal guard passes 1/1 on port `34198`; mobile open/close evidence exists	Broader less-common overlay states and focus-return evidence.
Persona Switcher	`Ctrl+Shift+P` overlay	`PersonaSwitcher`	Switch active persona/group	Fresh shell overlay smoke passes desktop open/Escape close	Keep current labelled/focus-trapped behavior covered.
Spawn Agent	New Agent overlay	`SpawnAgentDialog`	Start an agent run	Fresh shell overlay smoke passes desktop/mobile open and desktop Escape close	Form labels and model fallback clarity.
Notification Inbox	Status/sidebar entry	`NotificationInbox`	Review alerts	Fresh shell overlay smoke fails Escape close and semantics	Add labelled panel/dialog semantics, named icon actions, and keyboard close proof.
Create Workspace	Workspace Switcher/New workspace	`CreateWorkspaceDialog`	Create a workspace	Fresh shell overlay smoke fails Escape close/semantics; mobile first viewport is template-heavy	Add modal semantics/focus handling, named icon actions, and mobile-first creation hierarchy.
Context Rail	Chat/context action	`ContextRail`	Inspect selected context	Source-only overlay finding	Add labelled landmark, named close action, and keyboard/open-close evidence.
Onboarding Tooltips	Post-onboarding coach marks	`OnboardingTooltips`	Explain first actions	Source-only overlay finding	Decide modal vs non-modal semantics and prove it never blocks task flow.
Onboarding Wizard	`forceWizard`/first run	`OnboardingWizard`	First launch setup	Passing tests plus clean-data smoke on port 3431; focused T1/mobile checks pass on ports `34196`/`34194`; high-volume import CTA coverage passes 2/2; model/first-task handoff tests pass 17/17	Desktop completion works; remaining first-run risk is broader state/recovery evidence, not the sampled CTA/copy/handoff polish.

Usage Scenarios Covered

First Run and Account

Covered surfaces:

Boot/briefing skip paths
Accountless app behavior
Optional Clerk wrapper
Onboarding Wizard
Model gate
Memory import
Template selection
First task setup

Risk:

The optional Clerk wrapper is conceptually right: no key means accountless, key means Clerk. The original built local app picked up a real publishable key and then hit CSP/script loading errors; current sampled accountless console-health checks pass 3/3 on port 34196.

Correction:

Make deterministic local/accountless audit mode impossible to accidentally override with apps/web/.env.local.
Either permit Clerk's configured frontend API in CSP when Clerk is intended, or keep Clerk fully disabled in desktop/local/offline accountless mode.
Re-run the console-error audit until it passes.

Home and Daily Continuity

Covered surfaces:

Home route
Start Here
While you slept
Recent workspace
Mobile Home

Result:

Home is one of the strongest parts of the product. Desktop and mobile screenshots were coherent and usable. This should remain the anchor surface for first daily use rather than creating a new surface for continuity work.

Correction:

Preserve Home as the default daily cockpit.
Do not introduce a separate "dashboard" unless a specific missing workflow proves Home cannot own it.

Workspace, Chat, Files, and Artifacts

Covered surfaces:

Workspace route
Chat textarea and model/persona visibility
File manager smoke
Artifact center smoke
Workspace management overlays

Result:

Automated journey tests pass. The shell opens the primary working surfaces reliably.

Risks:

Some creation/edit flows rely on dense modal content and keyboard-only handlers.
The current production native-dialog scan is clean; sampled Artifact permanent-delete, Memory Center delete/erase/re-import, Settings telemetry/backup/restore, Wiki export, Automation delete, compliance template delete, admin member removal, BackupApp restore, and Create Workspace custom-template delete paths now use in-app confirmations/forms/status.

Correction:

Standardize destructive workspace/file/artifact actions on the in-app confirmation pattern with focus management and clear consequences.

Memory, Wiki, Trust, and Provenance

Covered surfaces:

Memory route
Empty/non-empty memory states
Memory Center
Wiki export forms
Trust management references
Evolution tab references

Result:

The memory app is functional in tests and appears integrated with the main navigation.

Risks:

Memory Center delete, GDPR erase, allow re-import, and Wiki Obsidian/Notion export destinations now use in-app confirmations/forms.

Correction:

Replace remaining prompt/confirm flows with in-app modals or dedicated inline forms.
Keep destructive actions on the branded confirmation pattern and allow safe cancel/undo where possible.

Agents, Personas, and Automations

Covered surfaces:

Agents route
Spawn Agent dialog
Persona Switcher
Automation Center
Agent groups/builders

Result:

Tests cover and pass important agent/automation surfaces.

Risks:

Test output includes React act(...) warnings in several app surfaces, including workspace, model gate, settings, automation, and templates.
Some form labels are visually present but not consistently associated with inputs.

Correction:

Fix test async warnings so regressions are easier to see.
Run a focused accessibility pass over forms, labels, and keyboard flows.

Skills, Connectors, MCP, Marketplace, and Launcher

Covered surfaces:

Skills Hub
Connector Hub
MCP Hub
Marketplace
Launcher
Command catalog

Result:

Deep app smoke tests pass. The high-level IA is understandable: the sidebar stays simple and command center exposes hidden depth.

Risks:

Original source scans found user-facing Pro gate copy in cockpit Marketplace/MCP/Skills/Settings surfaces where current strategy says Solo/Team.
Current source search finds remaining cockpit Pro references only in explicit legacy billing compatibility, tests/comments, or model names. Public legal Pro copy remains tracked under launch-funnel T13.

Correction:

Audit all rendered billing/gating text and normalize it to Solo/Team/Enterprise.
Keep "legacy Pro" only where the user is already a legacy Pro subscriber and the context is explicitly billing history.

Team, Approvals, Billing, Vault, Backup, and Compliance

Covered surfaces:

Team Governance
Approvals
Settings billing
Settings vault
Backup/restore
Compliance dashboard/template flows

Result:

Smoke tests pass, but trust-critical flows need UX cleanup.

Risks:

The sampled Approvals revoke-all, Settings backup/restore, standalone BackupApp restore, Automation delete, compliance template delete, and admin-web member removal paths now use in-app confirmation/status patterns.
Settings and standalone BackupApp backup/restore success and failure now use inline status/approval patterns for the sampled paths.

Correction:

Move trust-critical feedback into the app UI.
For backup/restore, show stateful progress, failure details, and next required action in the page, not a browser alert.

Settings and Narrow Viewports

Covered surfaces:

Settings route
Settings tabs
Model/provider settings
Billing/profile/vault/system settings
Mobile Settings at 390 x 844

Result:

Desktop Settings is generally coherent.
Mobile Settings was a major original blocker: the tab rail and content panel stayed side by side, leaving a narrow content column. Current focused Settings verification is green on port 34195; broader mobile polish still needs Memory/workspace-chat tab ownership.

Correction:

Add a responsive Settings layout:
- On narrow widths, tabs should become a top segmented control, horizontal scroll tab bar, or drawer.
- Content should use the full width.
- Provider/model cards should stack to one column.
- No inner horizontal scrollbar should be required for core settings.

Findings and Correction Backlog

P0-1: Accountless Clerk/CSP console health (verified fixed)

Evidence:

tests/e2e/full-product-audit.spec.ts:360 asserts no critical console errors.
The built app failed this check.
Console errors included CSP inline script blocking, Clerk script blocking, and Clerk load timeouts.
apps/web/src/lib/clerk.ts:41 reads VITE_CLERK_PUBLISHABLE_KEY.
apps/web/src/providers/WaggleClerkProvider.tsx:21 mounts ClerkProvider when a key is present.
Current verification update: clean first-run onboarding loads without Clerk, CSP, or page errors, no console errors on initial load, and no critical console errors on load passed 3/3 on port 34196.

Why it matters:

Trust and first-run stability are part of the UX. A product can look usable and still fail a buyer/admin judge if it emits auth/security errors at boot.
The sampled accountless path no longer carries this cap; explicit Clerk-enabled rendered auth still needs state evidence.

Correction:

Define one deterministic local/offline/accountless mode for desktop and test harnesses.
Prevent local .env.local from accidentally enabling Clerk during accountless verification.
If Clerk is enabled, align CSP with the configured Clerk frontend API and remove inline script violations.
Re-run the full-product audit until "no critical console errors on load" passes.

Acceptance:

No critical console errors on initial load.
Auth/accountless behavior is visible and intentional.
No app blanking if Clerk is unavailable.
Current focused accountless evidence passes this acceptance on port 34196.

P0-2: Mobile Settings structural usability (verified fixed)

Evidence:

apps/web/src/components/os/apps/SettingsApp.tsx:221 renders Settings as flex h-full.
apps/web/src/components/os/apps/SettingsApp.tsx:225 keeps a fixed w-36 tab sidebar.
apps/web/src/components/os/apps/SettingsApp.tsx:259 renders content in the remaining width.
At 390 x 844, the content panel was squeezed to roughly half the screen and provider cards clipped.
Fresh mobile smoke on port 3419 captured output/playwright/mobile-executive-3419/settings.png, settings-tab-models.png, settings-tab-billing.png, and settings-profile.png. It found no document-level overflow, but Settings header disclosure controls, model controls, and billing controls still visibly overflowed or became unreadably narrow.
Current verification update: J-mobile: Settings is usable at 390px width passed 1/1 on port 34195; the test checks /settings, /settings?tab=models, /settings?tab=billing, and /settings/profile for document-level overflow and visible control bounds.

Why it matters:

A mobile/tablet judge cannot score this 9/10 even if desktop is polished.

Correction:

At small breakpoints, convert Settings to a single-column layout.
Move Settings tabs to a top tab strip, drawer, or select-style control.
Stack provider/model cards and remove internal horizontal scrolling for primary content.

Acceptance:

Mobile Settings has no document-level or main-panel horizontal overflow and no critical visible element overflow. Current focused evidence passes this acceptance on port 34195.
Labels, provider names, badges, and action buttons are readable at 390 px width.
Desktop layout remains unchanged except for intentional polish.

P0-3: Cockpit pricing copy uses Solo/Team/Enterprise (verified fixed)

Original evidence:

apps/web/src/components/os/apps/MarketplaceApp.tsx:65: marketplace MCP note says "Pro".
apps/web/src/components/os/apps/mcp/AddCustomMcpForm.tsx:79: custom MCP form originally said legacy Pro was required.
apps/web/src/lib/command-catalog.ts:113: command group heading originally used legacy Pro wording.
apps/web/src/components/os/overlays/LoginBriefing.tsx:351 and :358: account/team copy originally mixed legacy Pro with Teams/Enterprise.
apps/web/src/components/os/apps/skills/SkillRow.tsx:55: verified badge title says "PRO".
apps/web/src/components/os/apps/SettingsApp.tsx:335: local-first reassurance originally mixed legacy Pro with Teams.
Current verification update: source search over the named cockpit files and apps/web/src finds remaining Pro references only in explicit legacy billing contexts (Legacy Pro, Pro (legacy)), tests/comments, or model/provider names. Public legal Pro copy is tracked separately under T13.

Why it matters:

The current strategic model is Solo/Team/Enterprise, with legacy Pro mapped to Solo. Mixed copy harms comprehension and buyer confidence.

Correction:

Replace user-facing Pro copy with Solo/Team/Enterprise language.
Keep "legacy Pro" only in billing contexts where a legacy subscriber is being serviced.
Add/extend tests for rendered plan copy in Marketplace, MCP, LoginBriefing, Settings, command catalog, and skill rows.

Acceptance:

A text search for user-visible Pro copy returns only explicit legacy billing contexts and comments/tests.
Solo users understand what is free.
Team upgrade triggers are clear and consistent.
Current cockpit evidence passes this acceptance; public launch/legal copy remains separate.

P0-4: Visual regression suite for tracked views (verified fixed)

Evidence:

tests/visual/views.spec.ts:12 tracks chat, memory, events, capabilities, cockpit, mission-control, and settings.
tests/visual/views.spec.ts:139 asserts screenshots for both dark and light.
All 14 snapshots failed in the original rendered run, then passed in the current T5 verification run: 14/14 on port 34199.
Fresh T5 rerun on port 3463 preserved artifacts under output/playwright/visual-t5-3463/test-results/; exact classification lives in docs/audits/2026-07-08-visual-t5-classification.md.
Actual screenshots were coherent in desktop dark and light mode. The original failures were classified mostly as stale-baseline drift after intentional surface changes, not a broad UI break.
tests/visual/baselines/ also contains duplicate historical baseline families; the active runner uses the ASCII-hyphen Visual-Regression---... folders, and the duplicate-family finding is documented in the T5 classification note.

Why it matters:

The goal explicitly requires polished, well-designed UX. A failing visual suite removes objective evidence that the app stayed polished.

Correction:

Keep current intentional surfaces instead of rolling UI back to old snapshots.
Preserve the stabilization-only masking in tests/visual/views.spec.ts so visual snapshots compare layout and presentation, not volatile Home copy.
Keep the duplicate baseline-family note in docs/audits/2026-07-08-visual-t5-classification.md until the historical folders are intentionally archived or removed.

Acceptance:

Visual suite passes in dark and light; current focused evidence is 14/14 on port 34199.
Baseline updates and volatile-field masking are intentional and documented.
The active baseline family is documented; duplicate historical folders are not used by the current runner.

P0-5: `Ctrl+Shift+N` route contract (verified fixed)

Evidence:

tests/e2e/phase-ab-verification.spec.ts:132 expects Ctrl+Shift+N to navigate to /workspaces/:id/chat when any workspace exists.
tests/e2e/power-user-stress.spec.ts:215 expects the same shortcut to open the active workspace chat route and reveal the chat input.
Both tests failed on initial run and retry, then passed in the focused T4 re-check: 2/2 on port 34197.
apps/web/src/hooks/useKeyboardShortcuts.ts:75 calls onNewChatWindow for Ctrl+Shift+N.
apps/web/src/components/os/AppShell.tsx:140 routes to chat only when effectiveActiveWorkspaceId exists; otherwise it opens the Workspace Switcher.

Why it matters:

Power users rely on keyboard promises. A shortcut that sometimes opens a modal instead of the expected work surface feels broken, even when the fallback is intentional.

Correction:

Product contract now follows Option A: Ctrl+Shift+N opens the active or first available workspace chat rather than leaving the user in an ambiguous chooser state when a workspace exists.
Keep code, tests, keyboard help, and command-center copy aligned to that contract.

Acceptance:

Both E2E shortcut tests pass against the approved contract.
No modal remains open after a successful shortcut route.
The chat input is visible after the shortcut when a workspace is available.

P0-6: Workspace Switcher route traversal (verified fixed)

Evidence:

tests/e2e/full-wiring-audit.spec.ts:553 originally timed out while traversing sidebar views.
Error context shows the Switch Workspace dialog remained open and its backdrop intercepted clicks on the Memory sidebar item.
Screenshot artifact: test-results/e2e-full-wiring-audit-Full-adf22-s-—-zero-critical-JS-errors-chromium/test-failed-1.png.
Current verification: traverse all sidebar views - zero critical JS errors passed 1/1 on port 34198.

Why it matters:

This is a real UI state, not just a test artifact. A modal that persists across expected navigation blocks the primary shell and creates a "why won't the app move?" moment.

Correction:

Ensure route-changing sidebar actions close or supersede workspace-selection overlays when appropriate.
If the switcher opens because Chat has no explicit workspace, make its next step unambiguous and focus-safe.
Add regression coverage for "open workspace switcher -> click Memory/Home/Settings -> modal closes or navigation is intentionally blocked with clear reason."

Acceptance:

Full console traversal can visit key sidebar routes without pointer interception; current focused evidence passes on port 34198.
Users always have a visible close path and the shell never feels frozen behind the overlay.

P0-7: Mobile first-run onboarding primary action reachability (verified fixed)

Evidence:

Fresh clean-data onboarding smoke on port 3431 captured output/playwright/first-run-onboarding-3431/mobile-00-welcome.png and mobile-01-profile-empty.png.
Mobile Welcome fits and communicates the product promise clearly.
Mobile Profile is taller than the initial 390 x 844 viewport; the primary Continue button starts below the visible area.
Trace evidence records the Continue button bottom at 880 in an 844 px viewport.
The same smoke completed the desktop first-run path through /workspaces/research-hub/chat, so this is a responsive onboarding issue, not a general onboarding dead end.
Current verification update: J-mobile: first-run onboarding keeps primary actions reachable at 390px width passed 1/1 on port 34194; the test asserts the Profile Continue button is within the 844 px viewport, checks visible overflow, and guards Clerk/CSP/page errors.

Why it matters:

First-run setup is a primary Solo Founder and Mobile Executive route. The hidden-primary-action cap and sampled T1 console-health cap are currently closed for the Profile/accountless path, but remaining handoff polish can still cap the journey.

Correction:

Make the Profile step mobile-first: reduce vertical density, split optional details, or use a sticky action row inside the wizard.
Keep progress/title context and primary action visible or clearly reachable at 390 x 844.
Add visible-bounds evidence for first-run onboarding, not only skipped-shell routes.

Acceptance:

390 x 844 clean-data onboarding Profile has no clipped critical controls and the primary Continue action is visible, sticky, or deliberately signposted. Current focused evidence passes this acceptance on port 34194.
Desktop first-run completion still reaches the chosen workspace chat and auto-sends the first task.

P1-1: Native browser dialogs interrupt branded workflows

Evidence:

packages/admin-web/src/pages/Members.tsx:72
Historical fixed: apps/web/src/components/os/apps/BackupApp.tsx:79
Historical fixed: apps/web/src/components/os/apps/AutomationCenterApp.tsx:184
Historical fixed: apps/web/src/components/os/apps/cockpit/ComplianceTemplateModal.tsx:155
Historical fixed: apps/web/src/components/os/apps/SettingsApp.tsx:399, :893, :901, :911, :921, :922, :925

Implementation update 2026-07-08:

Create Workspace custom-template delete now uses an in-app confirmation.
Approvals revoke-all grants now uses the shared in-app ApprovalModal, with focused component coverage, rendered /approvals J3d evidence on port 34159, and latest expanded user-journey evidence passing 22/22 on port 34165.
Artifact permanent delete now uses the shared in-app ApprovalModal, with focused component coverage, rendered /artifacts J3e evidence on port 34161, and latest expanded user-journey evidence passing 22/22 on port 34165.
Memory Center permanent delete, GDPR erase, and allow re-import now use the shared in-app ApprovalModal, with focused component coverage, rendered /memory?tab=memories J3f evidence on port 34164, and latest expanded user-journey evidence passing 22/22 on port 34165.
Wiki Obsidian and Notion exports now use in-app form dialogs, with focused component coverage, rendered /memory?tab=wiki J3g evidence on port 34167, and the path remains included in latest expanded user-journey evidence passing 24/24 on port 34177.
Settings telemetry clear, backup failure, and restore success now use in-app approval/status states, with focused component coverage, rendered /settings?tab=backup J3h evidence on port 34171, and latest expanded user-journey evidence passing 24/24 on port 34177.
Standalone BackupApp restore now uses the shared in-app approval modal, with focused component coverage in p1b-authgate-surfaces.test.tsx.
Automation delete now uses the shared in-app approval modal, with focused component coverage in phase3b-automation-center.test.tsx.
Compliance template delete now uses the shared in-app approval modal, with focused component coverage in compliance-template-trust.test.tsx.
Admin-web member removal now uses an in-app confirmation panel, with focused coverage in admin-pages.test.ts.

Why it matters:

Native dialogs feel outside the product, block focus in unpredictable ways, and are weakest exactly where the product needs trust: delete, restore, revoke, export.

Correction:

Use a shared in-app confirm/result dialog for destructive and recovery flows.
Use inline fields or modal forms instead of prompt().
Use toasts or inline status blocks for backup/restore results.

Acceptance:

No native confirm/alert/prompt remains in shipped UX-critical paths.
Keyboard and screen reader behavior is deliberate.

P1-2: Test output is too noisy

Evidence:

Full Vitest passes but emits repeated React act(...) warnings.
Node emits repeated [DEP0040] punycode warnings.
Some tests intentionally log expected adapter/server errors.

Why it matters:

Noisy green tests train reviewers to ignore output. That lowers confidence during a UX polish arc.

Implementation update 2026-07-10:

The E2E harness removes inherited FORCE_COLOR/NO_COLOR conflicts for both the Playwright worker and web-server child, suppresses the expected mock-embedding banner only in the harness, and keeps local Ollama cost tracking explicitly free. A fresh built-app J1 smoke passed 1/1 on port 34315 with no color, embedding, or pricing warnings. Both the web and public apps/www Vitest configs now propagate the narrow Node DEP0040 suppression to worker processes; the public full local suite passes 8 files / 21 tests without punycode warnings, and the new Files drag lane passes 2/2 cleanly.
Files workspace-rail dead end closed: POST /api/workspaces/:targetWorkspaceId/files/copy accepts a validated sourceWorkspaceId for file-only cross-workspace copies, preserves source data, indexes the target as usual, and returns an actionable 400 for folder drops. The existing API suite passes 33/33; the Files interaction suite passes 2/2 with success and folder-recovery coverage.
Fresh post-fix browser evidence: five-persona state bundles pass 5/5 on port 34316; runtime axe passes 2/2 on port 34317 across the desktop and 390px mobile route set. The changed build completed before both runs.
Fresh broad browser refresh with the test base URL aligned to each fresh server: full-product-audit.spec.ts passes 40/40; full-wiring-audit.spec.ts, user-behavior.spec.ts, and power-user-stress.spec.ts pass 133/133; visual dark/light, boot, light-mode, and polish coverage passes 32/32 with one intentional skip. The earlier 3 desktop-shell failures were harness-only wrong-port navigations, not product failures.
Model setup UX correction: the shared ModelGate now scrolls and focuses the selected provider's key field after a tile click, so the same onboarding and Settings path remains actionable on compact/mobile layouts instead of leaving the field below a long provider grid. ModelGate + onboarding coverage passes 28/28; rebuilt-app five-persona and axe checks pass 5/5 and 2/2 on port 34323.
API-key persistence correction: the Settings save path is now Vault-only for secrets, with provider models/base URLs retained as non-secret metadata; startup migration moves legacy config keys to Vault and scrubs their plaintext values. Settings API coverage passes 44/44, provider/migration coverage passes 18/18, and the server package typecheck passes.
Channels integration evidence from the committed main integration worktree: Telegram, Discord, Slack, WhatsApp, pairing, manager, and loopback chat suites pass 7 files / 79 tests; ChannelsSettings.test.tsx passes 7/7; server typecheck passes. Real bot-token delivery, WhatsApp device pairing, and live NAT/transport behavior remain external verification items.
Expanded accessibility correction: the 41-route desktop/mobile axe matrix is green 2/2 on fresh port 34333 after fixing workspace-card semantics and nested actions, contrast in workspace/platform/billing/usage states, heading order, and keyboard-focusable scroll regions. The harness waits for lazy content and the shell entrance transition before sampling so transient dialog animation does not create false contrast findings; each viewport test has a bounded 150-second budget.
Current five-persona state-bundle rerun: tests/e2e/five-persona-state-bundles.spec.ts passes 5/5 on fresh port 34334 for Solo Founder, Researcher, Engineer / Power User, Team Admin / Security Reviewer, and Mobile Executive. The run keeps the declared account/tier/model/data/offline state fields and the sampled failure, recovery, scale, billing, backup, approvals, launcher, memory, and mobile probes green; it is technical evidence, not subjective 9/10 judge sign-off.
Files workspace-switch correction (2026-07-10): Files freshness is now keyed by the complete workspaceId + directory route, so the workspace rail cannot present the previous workspace's rows as current while a switch is loading. Per-route session cache also preserves resolved empty directories and optimistic file operations for warm remounts. The focused Files/cross-workspace/deep-link/storage slice passes 25/25, including an explicit stale-workspace regression; web typecheck passes.
Remaining judge-nit correction (2026-07-10): the Platform surface's small uppercase top label now uses the established muted eyebrow tier in light mode instead of the dim tier. Platform coverage passes 4/4, and the full web gate after the correction passes 206 files / 1,718 tests.
Motion verification correction (2026-07-10): Home's cache delta pulse used a three-keyframe scale array with a spring transition, which produced an asynchronous Framer Motion exception under jsdom. It now uses the shared timed easing token for keyframes. Home motion/cache coverage passes 13/13; the complete web gate passes 207 files / 1,724 tests with zero unhandled errors. Known React act() and Router future-flag warnings remain test-hygiene debt, not product failures.
Browser Companion honesty correction (2026-07-10): Settings no longer labels Browser AI extensions as fully covered while native toolbar/context-menu evidence is open. The Coverage Compass now marks that row partial with explicit popup/capture coverage and pending native entry points; its focused regression passes 1/1.
Follow-up UX correction (2026-07-10): opacity-diluted uppercase labels in Home, Agents, Marketplace, Artifact Center, Launcher, Login Briefing, Persona Switcher, WaggleDance, and MCP cards now use the guaranteed --honey-text/--text-tertiary/status tokens. The semantic color guard passes with no new offenses, and the affected focused lanes plus the full web suite are green.
Cache-first return correction (2026-07-10): Connectors, MCP Hub, Artifact Center, Approvals, and Launcher now retain resolved session data while their route remounts and silently revalidates. Artifact cache keys include workspace, search, kind, and status, preventing cross-view stale rows; focused cache-return coverage passes 8 files / 71 tests.
Onboarding affordance correction (2026-07-10): inactive Who Are You chips now use a visible line-affordance border, guaranteed tertiary text, and focus-visible ring target instead of relying on color alone. The focused onboarding lane passes 14/14 and the full web gate passes 207 files / 1725 tests.
Model setup recovery correction (2026-07-11): the shared onboarding/Settings ModelGate now surfaces provider-catalog failures with service-aware copy and a Retry action. This closes the first-run dead-end where a failed /api/providers request left users with an empty grid and no way to add a key. Focused ModelGate + onboarding coverage passes 27/27; web typecheck passes. The desktop service/update event bridge was re-audited in the same pass and is already wired and covered by binding tests.
API-key setup rendered verification (2026-07-11): fresh-port Playwright coverage now exercises the actual onboarding and Settings surfaces, not only mocked component behavior. Both paths select Anthropic, focus the password field, submit a format-valid test key through test-key, record the provider payload sent to the canonical /api/settings Vault path, show saved feedback, and confirm onboarding Continue becomes enabled. The lane passes 2/2 on port 34382; live provider acceptance with a real credential remains an external verification concern.
Provider model catalog correction (2026-07-11): the provider route and browser-side registry no longer contain a built-in model inventory. Configured provider APIs are queried for their complete returned model list; new entries are normalized to stable provider/model ids, merged into /api/litellm/models, and retained as an explicitly labelled stale catalog during outages. Provider and Chat catalogs refresh when Waggle regains focus. Selecting or saving an id missing from the running managed config regenerates the catalog and restarts the router once before Chat, scheduled work, or fleet execution receives it; existing ids do not restart. The latest incremental hot-model lane passes 27/27 backend tests and 3/3 web tests, alongside the existing ModelGate/onboarding lane. Real provider credentials plus one paid live completion remain external release evidence only.
Create Workspace mobile disclosure correction (2026-07-11): optional agent assignment is now collapsed by default so workspace name, storage, and creation remain the primary path at 390 x 844. The existing selected template/persona still appears in the summary, and the full picker expands with an explicit control; closing and reopening resets the optional disclosures. Focused shell-overlay contracts pass 15/15, the full web gate passes 207 files / 1726 tests, web typecheck passes, and the rendered J-mobile journey passes 1/1 on fresh port 34380. Broader modal-state screenshots and focus-return coverage remain open.

Correction:

Fix async state updates in affected tests/components.
Suppress only explicitly expected logs inside tests that assert error handling.
Track dependency source of punycode warning or filter it in test setup with a comment if upstream-only.

Acceptance:

Full web test run is green with minimal expected warning noise.

P1-3: Marketplace and local-first test determinism need tightening

Evidence:

tests/e2e/full-wiring-audit.spec.ts:501 failed once because marketplace search returned non-OK, then passed on retry.
tests/e2e/waggle-complete.spec.ts:388 triggered POST /api/marketplace/sync, which performed live external sync and took 34.4s.
The sync logged multiple external source errors even though the harness had WAGGLE_DISABLE_MARKETPLACE_SYNC=1.
Current server fix: POST /api/marketplace/sync now respects WAGGLE_DISABLE_MARKETPLACE_SYNC=1 and returns a no-network skipped response. The focused red/green regression passed, and the combined manual/background sync Vitest slice passed 3/3.
Current E2E status: focused marketplace Playwright verification passed 4/4 on port 34203 after pinning the repo's direct tsx dependency to a matching esbuild generation.

Why it matters:

Waggle's local-first story should not depend on external registries during core verification. Network-heavy sync also makes UX evidence slow and flaky.

Correction:

Separate "local browse/search works" from "explicit external sync works."
In test/audit mode, marketplace sync is explicitly skipped when WAGGLE_DISABLE_MARKETPLACE_SYNC=1.
Surface external sync partial failures in-app with clear source-level status rather than noisy logs only.

Acceptance:

Core marketplace browse/search tests pass without network access in the audit lane.
External sync has its own opt-in test lane and user-visible partial-failure reporting.

P1-4: Initial app payload is too heavy for a seamless desktop feel

Evidence:

Fixed 2026-07-09: current startup JS chunk is 421.96 kB minified / 114.08 kB gzip, down from the original 1,822.76-1,847.38 kB range.
Vite no longer warns that JS chunks exceed 500 kB after minification.
Several persona images are 395 to 653 kB.
shape-selection.ts dynamic/static import warning is fixed; route/app surfaces, closed shell overlays, ChatHost, and cloud analytics are guarded by build-warning-hygiene.test.ts so they stay out of the startup bundle.

Why it matters:

Waggle is a desktop app, but users still judge startup responsiveness. A heavy first payload weakens first impression and can worsen perceived polish on lower-end machines.

Correction:

Lazy-load deep apps and settings subsections that are not needed for first paint.
Fix mixed static/dynamic imports that defeat splitting.
Compress or resize large persona assets.

Acceptance:

Startup JS chunk stays below the Vite 500 kB warning threshold.
Home and core shell render before deep app, overlay, chat, and analytics code is loaded.
Persona/logo media assets are optimized or explicitly accepted in a separate visual/CLS pass.

P1-5: Form, focus, and icon-button accessibility need a focused pass

Evidence:

Static search found many visually labeled inputs without explicit htmlFor/id association, especially in Settings, profile, workspace creation, agent builders, compliance templates, and onboarding.
Some labels wrap controls, which can be valid, but the pattern is inconsistent enough to merit a focused pass.
Representative icon-only controls without accessible names still need a broader unsampled sweep; the sampled Cockpit, WaggleDance, ComplianceDashboard, custom AgentCard, and source-level Mission Control actions now have focused fixes and green coverage.
Modal and confirmation inputs use autoFocus in several places, which can be good when deliberate, but the judge pass should prove focus lands on the intended field and returns to the triggering control when the modal closes.
Runtime axe/DOM smoke found critical unnamed controls/selects on Settings, Profile, Launcher, and Approvals; a serious Files keyboard-scroll issue; workspace semantic/image-alt findings; and Command Center dialog/label-fit warnings.
Shell overlay smoke found Notification Inbox and Create Workspace missing dialog/landmark semantics and Escape close behavior; Create Workspace had 16 visible unnamed icon buttons; Context Rail, Onboarding Tooltips, and tier modal close icons need naming/semantic cleanup.
Current T10 implementation slice fixes Settings Prompt Shape/model/trust/team/KVARK metadata, Profile identity/preferences/brand labels, Chat composer metadata, Vault add-secret/action metadata, Launcher refresh/prompt naming, Approvals refresh/revoke names, Cockpit/WaggleDance/ComplianceDashboard/AgentCard sampled action names, Mission Control source action names/focus rings, All Workspaces/Wiki search metadata/focus affordances, WorkspaceActionsMenu rename/delete confirmation labels and focus rings, warm AskBar metadata/focus ring, workspace TasksTab add-task metadata/focus ring, Timeline event-filter metadata/focus ring, Create Workspace template textarea metadata/focus rings, Workspace Switcher focus-trap/return evidence, Files storage/browser scroll-region keyboard access, and Command Center description/mobile row fit plus search focus ring.

Why it matters:

A seamless UX includes keyboard, screen reader, icon-button naming, and form error ergonomics. Form-heavy admin/productivity software loses quality quickly when labels, focus return, and action names are inconsistent.

Correction:

Audit actual form semantics, not just text appearance.
Add explicit id and htmlFor where labels do not wrap controls.
Verify error messages use aria-describedby or clear adjacent status text.
Add accessible names for icon-only action buttons, especially refresh, pause/resume/stop, delete, close, and retry controls.
Verify modal initial focus, Escape behavior, and focus return for high-traffic dialogs.
Keep tests/e2e/runtime-a11y.spec.ts green after focused source fixes rather than closing T10 from component tests alone.

Acceptance:

Core forms pass label/name checks.
Keyboard focus order is predictable in Settings, onboarding, workspace creation, and agent creation.
Icon-only buttons in high-traffic surfaces have accessible names.
Post-fix runtime axe results confirm the sampled Settings/Profile/Vault/Mission Control/Chat/Agents/WaggleDance/Launcher/Approvals named-control and semantic failures are gone.
Runtime axe has no critical or serious findings on the final judge route set, or each remaining finding is explicitly deferred with a score cap.
Modal focus behavior is deliberate and covered by a keyboard path.

P1-6: Local model pricing and graceful-degradation harness are inconsistent

Evidence:

During the broader sweep, a test named "First chat message accepted without LLM" used the local Ollama provider because it was healthy.
Server log: unknown model ollama/minimax-m2.7:cloud; cost tracker estimated with Sonnet pricing.

Implementation update 2026-07-10:

CostTracker now recognizes any ollama/* model as local/free when it has no explicit catalog entry, reports zero estimated provider cost, and does not emit a misleading cloud-pricing warning. The behavior is covered by packages/agent/tests/cost-tracker.test.ts.

Why it matters:

Usage & Cost is a trust surface. Unknown local model pricing should be explicit to users, and tests that claim no-LLM degradation should not silently use a real model.

Correction:

Add a pricing entry or explicit "local/free/unpriced" behavior for ollama/minimax-m2.7:cloud.
Make no-LLM degradation tests force provider unavailability or rename them to reflect real-provider behavior.

Acceptance:

No unknown-model cost warning in the standard UX audit run.
Tests accurately describe the provider mode they exercise.

P1-7: Route coverage ownership for the full shell (verified fixed)

Evidence:

App.tsx registers roughly 30 app routes/route wrappers plus catch-all recovery.
The visual suite captures only 7 views across dark/light.
Original route-string coverage search found no direct test mentions for /payment-cancelled; /benchmarks and /platform had component tests but no direct route/E2E route-string hits; /launcher, /waggle-dance, /artifacts, /settings/profile, /settings/timeline, /payment-success, /automations, /mcps, and /settings/usage only had 1-4 direct mentions.
Focused T11 command evidence: root npm run test -- apps/web/src/test/p1a-routes.test.ts --run finds no files because root Vitest excludes apps/**; app-local npm run test -w apps/web -- src/test/p1a-routes.test.ts --run passes 29/29; app-local Benchmark/Platform/Payment component tests pass 17/17.
Historical continuation evidence: npm run build passed and an ad hoc Playwright smoke against port 3407 proved /benchmarks and /platform render meaningful shell content and /payment-cancelled redirects to /settings?tab=billing; those historical route artifacts emitted the old T1 CSP/Clerk console errors. Current sampled accountless console-health checks pass 3/3 on port 34196. Screenshots are under output/playwright/route-smoke-3407/.
Historical thin-route continuation evidence: a second npm run build passed and an ad hoc Playwright smoke against port 3411 proved /launcher, /launcher?watch=1, /waggle-dance, /artifacts, /settings/profile, /settings/timeline, /payment-success, /automations, /mcps, /settings/usage, and /files render meaningful shell content. Those historical route artifacts emitted the old T1 CSP/Clerk console errors; current sampled accountless console-health checks pass 3/3 on port 34196. Launcher can still log a detect-in-flight adapter fetch error, and Usage & Cost logs a 403 resource error. Screenshots are under output/playwright/thin-route-smoke-3411/.
Current route evidence: npm run build passed again, a fresh sidecar on 127.0.0.1:3333 returned healthy, and the all-route built-preview smoke on port 3457 proved 33 desktop route navigations plus 11 mobile route spot-checks return 200, produce screenshots, and have no document-level horizontal overflow. It also confirms /payment-cancelled redirects to /settings?tab=billing. Artifacts are under output/playwright/route-evidence-3457/.
Current codified route evidence: J-route-coverage: thin utility routes render or redirect clearly and J-route-coverage: priority thin routes render meaningful shells passed 2/2 on port 34200.
Remaining route-adjacent health blockers are no longer P1-7 route-ownership gaps: Launcher watch-mode runtime noise belongs to T16, Usage & Cost semantics belong to T9, broader accessible-name/focus evidence belongs to T10, and workflow/failure-state depth belongs to T12.
Source inventory found embedded or retired app files that the final judge packet must classify rather than ignore: DashboardApp.tsx, VoiceApp.tsx, MissionControlApp.tsx, BackupApp.tsx, StorageApp.tsx, FilesAppTabs.tsx, and FilesApp.tsx.
Existing tests/vision/personas.spec.ts is valuable, but it focuses on five live persona chat journeys rather than full routed-surface grading.

Why it matters:

The requested goal is whole-system UX, not only the major sidebar routes. Thinly covered surfaces can remain confusing or broken while the core smoke suite stays mostly green.

Correction:

Keep the route coverage manifest as the final judge packet owner.
Keep J-route-coverage in the standard route-existence regression lane.
Expand deeper judge dry runs under T12 so every route's important state is either exercised by a persona journey, a visual capture, or a deliberate deferral.

Acceptance:

Final judge packet maps every registered route/overlay to at least one current evidence source.
/payment-cancelled, /launcher, /launcher?watch=1, /waggle-dance, /artifacts, /settings/profile, /settings/timeline, /payment-success, /automations, /mcps, /settings/usage, /files, /benchmarks, and /platform have codified smoke or visual coverage before the 9/10 gate.
Remaining untested surfaces have written deferral rationale.

P1-8: State and failure-mode coverage is not explicit enough for judge scoring

Evidence:

packages/shared/src/tiers.ts defines four canonical billing tiers plus legacy tier mapping; state coverage must include Trial, Solo, Team, Enterprise, and legacy Pro collapse where relevant.
apps/web/src/lib/dock-tiers.ts defines UI disclosure tiers separate from billing tier; a route can be accessible while navigation depth differs materially.
apps/web/src/providers/ShellContext.tsx, useWorkspaces.ts, useOfflineStatus.ts, and useHasWorkingModel.ts show that shell UX depends on account, tier, workspace, offline, and model readiness state, not only route path.
apps/web/src/lib/adapter.ts spans many API families; a single green route smoke cannot prove empty, error, 403, 404, SSE, or external-service states.
The route manifest names usage scenarios, but the final judge packet also needs to declare which state combination each persona exercised.
Focused continuation evidence: docs/audits/2026-07-08-state-failure-t12-analysis.md records a passing focused state slice (9 files / 79 tests), existing chat SSE network-drop/retry source coverage, native browser dialog line evidence, warning-noise risks, five-persona state-bundle fields, and T12-A through T12-G correction candidates.

Why it matters:

A judge can see a clean default route while still missing the hard UX states: no model, trial expired, no workspace, workspace load error, local marketplace fallback, destructive restore, or unsampled mobile edge states.
Whole-system UX means the right thing happens under realistic conditions, not only that the happy path renders.

Correction:

Add the state/failure matrix as a first-class evidence artifact.
Add T12 as the cross-state judge evidence ticket.
Require every judge scorecard to record account mode, billing tier, disclosure tier, model state, data-volume state, offline/error state, and viewport.
Use the focused T12 supplement as the closure checklist for state-bundle docs, browser state-bundle smoke, failure-state expansion, native dialog deferrals/fixes, warning hygiene, non-main gate decisions, and scale/performance evidence.

Acceptance:

Five judge scorecards cite the state bundle they exercised.
Any skipped state is explicitly deferred before scoring.
No persona receives 9/10 from default-route screenshots alone.

P0-L1: Public launch funnel has release-blocking smoke and recovery gaps

Evidence:

apps/www contains the public homepage, sign-in/sign-up/account pages, Stripe checkout route, Stripe webhook, legal pages, methodology page, and download CTAs.
npm run start -w apps/www -- --hostname 127.0.0.1 --port 3413 starts, but is an invalid Windows smoke host for this middleware setup: Next logs Failed to proxy http://localhost:<port>/... and route requests time out or 500.
npm run start -w apps/www -- --hostname localhost --port 3416 is the valid local production smoke host. With that host, /, /#pricing, /privacy, /terms, /cookies, /eu-ai-act, /sign-in, /sign-up, /docs/methodology, and unauthenticated /account redirect render.
Browser/server evidence still logs Clerk development-key and session-loop warnings under the active env.
Focused continuation evidence: npm run test -w apps/www -- --reporter=dot, npx tsc --noEmit --project apps/www/tsconfig.json, and npm run build:www pass; fresh localhost:3426 route/API smoke confirms the core route shape and signed-out GET checkout redirect.
Playwright interaction evidence against http://localhost:3416 captured desktop-home.png, desktop-pricing.png, desktop-checkout-error.png, mobile-home.png, and mobile-nav-open.png under %TEMP%\waggle-www-ux-localhost-20260708.
Fresh Browser interaction evidence against http://localhost:3491 captured www-t13-home-desktop.png, www-t13-pricing-checkout-error-desktop.png, www-t13-home-mobile.png, www-t13-mobile-menu-open.png, and www-t13-rendered-summary.json under output/playwright/www-t13-3491/.
Fresh Browser sampled states had 0 current-page console warnings/errors, but Browser DOM snapshot failed with the known incrementalAriaSnapshot plugin mismatch.
Pricing annual toggle and mobile nav open both worked in rendered browser evidence.
Signed-out Get Team rendered a small inline Sign in required error instead of using the returned signInUrl to guide the buyer into auth and checkout recovery.
/pricing?checkout=cancelled returns 404.
/methodology returns 404; the real shipped route is /docs/methodology.
Signed-out Team checkout through Pricing.tsx calls POST /api/stripe/checkout; the route returns a signInUrl on 401 but the component only displays "Sign in required."
Checkout cancel uses /pricing?checkout=cancelled, but pricing is a section on /, not a registered Next route.
Historical deployment blocker now locally fixed: deploy-www.yml previously uploaded apps/www/dist even though next.config.mjs has no static export output and the app includes dynamic auth/API routes. The workflow now targets Vercel prebuilt deployment; deployed smoke remains open.
DownloadCTA targets GitHub Releases latest, and GitHub currently reports no releases for the repository.
Fresh external refresh also confirms both waggle-os.ai and www.waggle-os.ai fail DNS resolution from this environment.
OS detection can label iPhone/iPad as macOS and unsupported desktops as Linux, while public platform copy says Windows and macOS.
Legal pages still include Day-0 placeholders, launch-date/address placeholders, and stale "Pro or Teams" privacy copy.

Why it matters:

Download, checkout, legal trust, and deployment are part of real user UX before they ever reach the desktop cockpit.
A five-persona in-app judge score can be honest only if the launch funnel is either verified separately or deliberately scoped out.

Correction:

Track this as T13.
Use docs/audits/2026-07-08-launch-funnel-t13-analysis.md as the focused correction artifact.
Publish a real download/release path or replace it with a controlled availability/status page before launch.
Document and use the stable localhost smoke host, then isolate/validate Clerk env so public routes render without auth-loop spam.
Verify or fix download, mobile nav, sign-in/sign-up/account, Team checkout, checkout cancel recovery, legal trust pages, and deployment target.
If the final judge gate is intentionally installed-app-only, get explicit user approval to defer this launch-funnel gate.

Acceptance:

Launch funnel evidence is attached, or T13 has an approved deferral.
Download CTA leads to valid Windows/macOS artifacts or a deliberate download/status page; mobile and unsupported OS labels are honest.
Local production route smoke uses the correct host binding and has no 500s, timeouts, unexpected 404s, or Clerk redirect-loop spam.
No public legal/trust page contains placeholder launch text or active Pro copy.
Checkout recovery paths do not dead-end.

P1-10: Desktop wrapper, installer, tray, update, and sidecar-startup UX lack installed-app evidence

Evidence:

Tauri config and tests cover build/resource shape, but this packet did not run a packaged app, installer, tray menu, close-to-tray behavior, global shortcut, or update flow.
cargo check --manifest-path app/src-tauri/Cargo.toml passes; app/src-tauri/resources/node.exe --version returns v20.18.1; node scripts/check-sidecar-resources.mjs passes with staged Node runtime, native deps, and sidecar node_modules.
Web-side Tauri binding tests pass from apps/web (focused binding 15/15 and prior binding/adapter bundle 19/19), but the same files are not discoverable from the root Vitest command because root config excludes apps/**.
Native source directly handles tray Open/focus, close-to-tray, Ctrl+Shift+W window visibility, and tray Quit; Settings routes through a tested /settings desktop bridge. This is source evidence only; it has not been proved in a packaged build on a target OS lane.
Rust service events (waggle://service-status, waggle://service-restart-needed) now have current apps/web/src consumers through TauriDesktopEventBridge; waggle://update-available remains mapped for future use while native updater emission stays disabled until signed updater artifacts exist. The prior Pause/About/Quit/Settings tray false-affordance issue is narrowed: Pause and About are hidden, Quit is native, and Settings has a web bridge.
The updater plugin config is intentionally disabled for v1 while Rust still contains update-check emission for future consumers.
packages/server/src/local/service.ts has port-conflict and health/degraded paths, but no current packet evidence shows the user-facing recovery UI for those states.

Why it matters:

Installed desktop UX includes launch, tray, service recovery, and update expectations, not only the React routes after the app is healthy.
A first-run failure in the wrapper can make the whole product feel broken even when apps/web tests pass.

Correction:

Track this as T14.
Verify or fix packaged launch, service readiness/failure recovery, tray actions, close-to-tray, global shortcut, installer/signing expectations, and update visibility.
Split desktop acceptance between source-wired and packaged-proof lanes: Open, Settings, Quit, close-to-tray, and Ctrl+Shift+W need packaged proof; update and watchdog events need visible UI, native handling, or explicit deferral.
Treat the active installed sidecar path as packages/server/src/local/service.ts bundled by scripts/build-sidecar.mjs, not the legacy top-level sidecar/ JSON-RPC path.

Acceptance:

Desktop wrapper evidence is attached, or T14 has an approved deferral.
Tray/update/service events are either consumed by UI, natively handled, or hidden until supported; any emitted route target exists. Remaining T14 proof is packaged watchdog/update presentation, not a missing web consumer in the current source.
Quit actually exits through a tested native or cleanup bridge, or the menu stops presenting it as a supported command.
Sidecar failure/port conflict has a user-facing recovery path.

P1-11: Admin web, CLI launcher, marketplace CLI, and memory MCP utility UX have broken built-entry and shallow UX evidence

Evidence:

packages/admin-web is a separate Vite/React surface with its own sidebar, token/slug connection UX, dashboard, analytics, members, capabilities, jobs, audit, and team settings pages. Component tests now pass through both root and package-local commands without React act(...) warnings. The original built-render smoke covers all 7 desktop pages and all 7 mobile pages under output/playwright/admin-web-t15-57795/; it rendered typed happy paths with no current-port console errors. A package-owned rendered Playwright gate now builds the package and passes 14 tests across all seven pages at 1200 x 800 and 390 x 844, checking hash state, active navigation semantics, browser back/forward traversal, viewport overflow, labelled table scroll regions, rendered control labels, clean app console/pageerror collection, mobile shell keyboard navigation, page-level keyboard traversal from connection fields into dashboard, members, capabilities, jobs, audit, and settings controls/table regions, full-page desktop/mobile visual snapshots, capability governance forms, malformed analytics response recovery, all-page initial API-failure recovery, mutation/destructive-failure recovery for capability policy save, capability override create/remove, capability request decision, member invite, member role change, member removal, and team settings save, plus real local bearer-auth middleware wrong-token/valid-token behavior through protected Fastify routes.
packages/launcher is the public npx waggle acquisition path. Source and built help smokes pass without service banners or .waggle creation. The launcher package gate now owns the real parser/core helpers, invalid-port validation before service setup, occupied-port recovery copy, and --no-open success copy. This packet still has no long-running clean startup/package invocation, browser-open fallback integration, or service-failure copy smoke beyond occupied-port startup failure; no packages/launcher/tsconfig.json exists for a no-emit TypeScript check.
Built @waggle/cli help is now locally fixed: help runs before the REPL dependency graph loads, and clean-home smokes pass for dist/index.js --help, bin/waggle.js --help, extracted packed-bin help, local package-closure installed npx waggle --help, and local package-closure installed npx waggle --local REPL startup/prompt/slash-command//exit with default.mind creation.
Built marketplace CLI previously failed before help because emitted ESM imports were extensionless; this is now locally fixed with NodeNext emitted imports and a built-help runtime guard. The marketplace package manifest also now points at emitted dist entrypoints included by the packed package.
Built legacy waggle-memory-mcp now completes official read-only and write-scope MCP client smokes; write scope saves and recalls a unique memory. A clean local package-closure install also launches the installed server and lists read-only tools; that proof caught and fixed the missing glob runtime dependency declaration in @waggle/core.
Built @waggle/hive-mind-mcp-server now completes official registration/scope and write-scope MCP client smokes; write scope saves and recalls a unique memory.
packages/hive-mind-cli has colocated tests under src/; root Vitest does not discover that lane, but the package now owns it through npm run test --workspace @waggle/hive-mind-cli -- --reporter=dot.
Hive-mind CLI sampled subcommand help is now side-effect free: source init --help and built status --help print focused command help and do not create personal.mind.
Marketplace invalid commands now print Unknown command, show help, exit 1, and do not construct a clean-home marketplace DB.
Package-local test command shapes are now fixed for launcher, @waggle/cli, waggle-memory-mcp, and hive-mind CLI in the T15 lane.

Why it matters:

"Complete UX" includes acquisition, admin, and recovery paths, not only the main desktop cockpit after everything is healthy.
Team admins and engineers are likely to touch these surfaces when installing, debugging, governing, or integrating Waggle.
A source-mode dev command can pass while the command a real user installs or runs via npx is unusable.

Correction:

Track this as T15.
Decide whether admin/CLI/MCP utilities are inside the five-persona scoring scope or explicitly deferred.
If included, keep the admin-web rendered package gate green with its local bearer-auth coverage, and attach focused command evidence for launcher, marketplace CLI, memory MCP, hive-mind MCP, Waggle CLI, and hive-mind CLI happy/error paths.
Keep the legacy waggle-memory-mcp runtime fixed or explicitly deprecate it; marketplace CLI manifest/pack/install shape, launcher packed/installed first-command recovery, packed and local package-closure installed @waggle/cli help plus installed local REPL startup/slash-command/exit and streamed chat/provider plumbing, local package-closure installed hive-mind-cli help, built/installed hive-mind MCP protocol runtime, and legacy memory MCP built/installed protocol runtime are locally fixed, but the utility lane still needs registry-only proof after internal package publication.
Fix or document the remaining package-local test command gaps.
Keep safe help and invalid-command semantics before mutating setup paths; hive-mind CLI and marketplace now have focused guards, while broader utility paths still need sampling.

Acceptance:

Admin web and supported utility CLIs have evidence for help/first-run, invalid input, missing configuration, and common success paths, or T15 has an approved deferral.
Built CLI/MCP entries run from a clean environment, or deprecated surfaces are removed from the judged product scope.
Root/package verification either runs the tests that exist or documents the separate command that owns them.
T15 deep-dive evidence: docs/audits/2026-07-08-admin-cli-utility-t15-analysis.md.

P1-12: AI-tool hook install/verify/uninstall UX has strong route/rendered evidence but lacks packaged desktop proof

Evidence:

T16 route/contract/package-runtime tests pass: 7 shared/agent/server files / 96 tests cover manifests, third-party loader, hook-capable cohort, launcher, backend tool routes, and local packed-package npx lifecycle for all six hook-capable packages; a focused registry-aware adapter slice passes 5 files / 108 tests for launchable third-party adapter launch and prompt-template routing.
Launcher hook UI, prompt, and adapter tests pass: 3 web files / 28 tests.
Root-run hook/shim package tests pass: 60 files / 573 tests with 1 skipped.
Official hook/shim package typechecks pass for 8 packages; the Claude Desktop stub build passes; 6 compiled hook-bin help smokes pass.
BUILTIN_TOOL_MANIFESTS marks claude-desktop as the only non-hook-capable built-in. packages/hive-mind-hooks-claude-desktop/src/index.ts is explicitly a stub with no bin; packages/hive-mind-hooks-codex-desktop ships a bin and re-exports Codex hook behavior.
Hook test output is intentionally noisy: fail-open hook warnings, sidecar-unreachable signal drops, package logs, and negative verify logs appear during passing tests.
Focused in-app Browser smoke renders Launcher mixed states with no current-port console errors: installed, not installed, hooks active, running, Phase 4/unsupported, prompt summary after typing, install success, verify failure, and live output. Evidence lives under output/playwright/launcher-t16-54147/.
The rendered smoke originally proved these UX gaps: install success hid stdout backup-path details, verify failure dropped the actionable stderr path, Claude Desktop lacked explicit launch-only/no-hooks copy, and the prompt textarea lacked name/id/ARIA/autocomplete metadata. Current focused tests protect stdout/stderr result detail visibility, summarize covered install output with Backup and Recovery labels instead of raw stdout:, preserve structured hook failures instead of masking them as generic HTTP 400, show retry/uninstall/reinstall recovery guidance for empty-output Verify failures, summarize Verify [FAIL] output as Check failed with the manual approval detail and no raw [FAIL], label uninstall restore/cleanup rows without implying install state, cap long hook output behind More output, fix observed-output duplicate replay after exit, fix prompt textarea metadata, and label installed Claude Desktop as launch-only with no hook actions. A fresh real Browser route smoke on /launcher renders verify failed (exit 1) plus empty-output recovery guidance instead of HTTP 400; Browser-rendered mocks confirmed the Claude Desktop launch-only layout, the Codex Verify manual-approval panel, and the Codex Uninstall cleanup panel; and codified Playwright coverage confirms standard install changed-file/pointer/backup/recovery labels, all six hook-capable install/verify/uninstall rendered transitions, sidecar-offline Retry, long-output summarization, and non-built-in adapter launch-only/prompt behavior. Packaged desktop lifecycle states remain open.
Hook/shim package-local npm run test --workspace ... scripts now pass for hook core, all six hook-capable packages, and shim core; the scoped non-hook package-local command-shape gaps tracked under T17 are now fixed, with warning hygiene and developer-journey evidence remaining.
Detection can include third-party adapter manifests; launchable adapters now launch through the runtime registry and can receive prompt-template routing. Hook management remains built-in-only until a safe third-party hook policy exists.

Why it matters:

The hook install flow edits real user AI-tool configuration files. A green unit test is not enough for a 9/10 UX claim unless the user can install, verify, understand, and remove hooks safely.
Launcher promises install/verify/uninstall actions; users need clear status, failure copy, and reversibility confidence.

Correction:

Track this as T16.
Decide whether hook-management scenarios are inside the five-persona score or explicitly deferred.
If included, keep the codified rendered Launcher evidence green for all-tool hook-management lifecycle, unsupported claude-desktop, sidecar-offline, long-stderr, and error states.
Keep the hermetic package-pack npx install/verify/uninstall command evidence green for all six supported hook packages, keep the isolated all-six route lifecycle green, and add packaged desktop hook-status evidence or an approved deferral before scoring T16 as complete.
Ensure hook verification output noise does not hide real failures in the standard audit lane.
Keep Launcher hook result copy guarded for Backup/Recovery, Check-failed/manual-approval, uninstall-cleanup, More-output, and rendered lifecycle guidance.

Acceptance:

Hook-management evidence is attached for UI and command paths, or T16 has an approved deferral.
claude-desktop is consistently shown as launchable but not hook-capable; six hook-capable tools expose clear actions and results.
Observed live output does not duplicate terminal replay after process exit; focused Launcher/web regression evidence passes 5 files / 72 tests and includes hook result stdout/stderr, Backup/Recovery result labels for covered install output, Check-failed/manual-approval rows for Verify output, uninstall cleanup labels, More-output summarization, structured failure detail coverage, empty-output recovery copy, Claude Desktop launch-only copy, and detected third-party adapter Launch/prompt routing.
Hook install/uninstall remains reversible and understandable in user-facing copy.
T16 supplement remains linked: docs/audits/2026-07-08-ai-tool-hook-t16-analysis.md.

P1-13: Developer API, background worker, and substrate verification UX has command-shape and suite-stability gaps

Detailed supplement: docs/audits/2026-07-08-developer-substrate-t17-analysis.md.

Evidence:

A remaining-package inventory covered agent, core, hive-mind-core, hive-mind-shim-core, hive-mind-wiki-compiler, optimizer, sdk, server, shared, waggle-dance, weaver, wiki-compiler, and worker.
Direct no-emit TypeScript passes for all 13 remaining workspaces.
npm run test -w @waggle/agent passes 195 files / 3097 tests.
Package-script tests pass for @waggle/core (19 files / 296 tests), @waggle/optimizer (1 file / 21 tests), and @waggle/weaver (3 files / 31 tests).
Root-run substrate/compiler tests pass: hive-mind-core, hive-mind-shim-core, and wiki-compiler together pass 71 files / 875 tests.
Root-run SDK tests pass 5 files / 89 tests, including the filesystem-safe plugin-id regression.
Package-local shared, WaggleDance, worker, and hive-mind wiki compiler scripts now pass 15 files / 154 tests in total (40 + 42 + 46 + 26), with the colocated wiki tests explicitly included in root discovery.
The named @waggle/server release lane passes 185 files, 1 skipped; 2,128 tests, 1 skipped, with one worker. The default wall-clock performance suite is isolated and passes 13/13.
Earlier packet evidence saw one full-suite performance-budget failure (Workspace Listing with 50 workspaces took 841ms against a 500ms budget), while the same server performance benchmark passes in isolation (13 tests). Treat this as a release-lane stability and command-shape risk, not a consistently broken endpoint.
Package-local npm test fails for hive-mind-core and wiki-compiler because Vitest looks for vitest.setup.ts relative to the package cwd; root-run commands pass. hive-mind-shim-core is fixed and passes through its package-local script.
Package-local npm test fails for @waggle/sdk because workspace-local Vitest resolves into packages/sdk/node_modules and cannot find convert-source-map; root-run SDK tests pass.
Standard server verification also exercises marketplace sync paths with many source-sync logs; this overlaps T6/T15 and should not be part of a quiet deterministic release lane.

Why it matters:

Engineers, admins, and release reviewers experience command failures and noisy logs as product quality signals.
"Every part tested" is not credible if package scripts fail by command shape, root runs miss or special-case package tests, or the full server route suite needs reruns to pass.
Server, worker, SDK, WaggleDance, and memory substrate behavior underpins visible UX: chat, workspace listing, marketplace, hooks, background jobs, and memory recall.

Correction:

Track this as T17.
Decide whether developer API/background/substrate evidence is inside the five-persona score or explicitly deferred.
Fix or document package-local test command shape for packages whose tests only pass from the root runner.
Stabilize the server route/performance gate: either make the full suite deterministic under normal audit load or move benchmark assertions to an isolated perf lane with a clear release threshold.
Reduce or document noisy passing output so real failures stand out.

Acceptance:

Developer API/background/substrate evidence is attached, or T17 has an approved deferral from the final score.
Package-local and root verification commands are deterministic and discover the intended tests, or their separate lanes are explicitly documented.
Server route suite and performance benchmark expectations are stable enough to support a final "all tested" claim.

P1-14: Ops, deployment, CI, benchmark, and judging evidence is incomplete for a complete-system claim

Detailed supplement: docs/audits/2026-07-08-ops-deploy-ci-judging-t18-analysis.md.

Evidence:

YAML parse passes for docker-compose.yml, docker-compose.production.yml, render.yaml, litellm-config.yaml, and all 7 workflow files under .github/workflows.
Docker CLI is installed locally, and both development and production Compose files pass docker compose config, but the Docker Desktop Linux engine is not reachable in this environment, so live Compose services and npm run test:infra were not run.
Raw Compose expansion reads ignored local env values and can print secret values; non-interpolated Compose config is safer for shareable evidence.
Tracking/ignore checks show only .env.example files are tracked among the checked env/key files; local .env, env.local, apps/www/.env.local, and AI API KEYS.txt are ignored.
Current render.yaml is explicitly the hosted local-sidecar path: it keeps /data persistence and Stripe sidecar routes and no longer provisions unused Postgres/Redis services. The Postgres/Redis-backed team server remains the Dockerfile/production Compose path.
ci.yml keeps the broad Playwright E2E job advisory with continue-on-error: true, but now has a separate blocking five-journey smoke job; no workflow currently runs the 19 infra-dependent Postgres/Redis suites from vitest.infra-suites.ts; this packet also could not run them locally because the Docker engine is unavailable.
docker-compose.production.yml now fails closed for Postgres and MinIO credentials with required interpolation; deployment tests pass 12/12 and the non-interpolated Compose review preserves placeholders without exposing local secrets.
Benchmark harness TypeScript passes: npx tsc --noEmit --project benchmarks/harness/tsconfig.json.
Benchmark harness package-local tests now pass 29 files / 325 tests through a root-config delegation script.
The focused T18 supplement records the now-fixed local apps/www GitHub Pages artifact mismatch and Render ambiguity, the still-open deployed Vercel/DNS proof gap, and the narrowed LiteLLM evidence boundary: dynamic provider discovery through exact-model completion is hermetically proven, while one paid external-provider smoke remains release evidence.
judging/ contains historical June 2026 artifacts, not current post-fix five-persona scoring evidence.

Why it matters:

A complete UX claim includes the experience of shipping, operating, validating, and judging the system.
Green app tests do not prove that deploy manifests, infra lanes, benchmark harnesses, or judge artifacts are current and trustworthy.
Secret-safe validation matters because ops logs are often copied into issues, PRs, and handoffs.

Correction:

Track this as T18.
Decide whether ops/deploy/CI/benchmark/judging evidence is inside the final score or explicitly deferred.
Document secret-safe Compose validation commands and avoid raw secret-expanded config logs in shareable evidence.
Decide Render's target: hosted local-sidecar demo or team Postgres server, then align provisioned services, env vars, and entrypoint.
Add or explicitly defer blocking smoke/infra CI lanes.
Fix or document benchmark harness package-local test command shape.
Generate fresh five-persona judging artifacts only after approved fixes land.

Acceptance:

Ops/deployment/CI/benchmark/judging evidence is attached, or T18 has an approved deferral from the final score.
Shareable ops validation output is secret-safe.
Current judge artifacts, not historical screenshots, are used for the 9/10 decision.

P1-15: Browser Companion extension UX is a real capture surface with incomplete end-to-end UX evidence

Evidence:

apps/browser-ext/README.md describes a Chrome Manifest V3 Browser Companion that captures the current page or selected text into Waggle memory.
apps/browser-ext contains root-level popup, background, content, manifest, and README files for the extension surface.
packages/server/src/local/routes/browser-ext.ts exposes GET /api/browser-ext/health; packages/server/src/local/cors-config.ts gates extension origins with WAGGLE_BROWSER_EXT_IDS or WAGGLE_DEV_ALLOW_ANY_EXTENSION=1.
The July audit packet had no extension-loaded smoke, popup screenshots, CORS-denied recovery evidence, or memory-frame result verification before the source-inventory supplement.
The T19 continuation added extension JS syntax checks, manifest parsing, packages/server typecheck, a failed-port connectivity check for 127.0.0.1:3333, and a Playwright disconnected-state screenshot at output/playwright/browser-companion-disconnected-state.png.
A live sidecar save-flow run on 127.0.0.1:3333 proved direct POST /api/memory/frames?extract=false saves selection-shaped and page-shaped frames, duplicate detection works, and Memory UI renders imported Browser Companion frames.
A real unpacked Chromium extension run originally proved the background save-memory message failed by default with generic HTTP 500 when the extension origin was not allowlisted; the post-fix extension now bootstraps a token first and maps setup denial to branded recovery copy.
The current secure-default live smoke loaded the extension, selected text in a normal HTTP page, confirmed Chromium MV3 service-worker fetches omit Origin and send sec-fetch-site: none, cleared any stored token, saved through chrome.runtime.sendMessage({ type: 'save-memory' }), stored the token during save, and confirmed /api/memory/frames contains the imported frame.
Focused CORS tests now cover WAGGLE_DEV_ALLOW_ANY_EXTENSION=1 and concrete chrome-extension://<id> origins; the live smoke used the production-shaped WAGGLE_BROWSER_EXT_IDS=<extension id> path.
Historical issue: /api/memory/frames reported the allowlisted extension frame as source: "import", while /api/memory/search returned the same frame as source: "user_stated". Current route coverage and popup-click smoke now confirm imported Browser Companion captures return as source: "import" in Memory search.

Why it matters:

Browser capture is part of the memory/Harvest promise. If it is confusing, disconnected, blocked by CORS, or silently fails, Researcher and Solo-founder trust drops even when the main cockpit works.

Correction:

Track this as T19.
Decide whether the Browser Companion extension is inside the final five-persona score or explicitly deferred.
If included, verify native toolbar-bubble behavior against a normal web page, native context-menu save or an explicit deferral, packaged pairing, and any separately scored global-recall provenance shape in the cockpit; popup keyboard/focus behavior now has live evidence.

Acceptance:

Browser Companion extension save-flow evidence is attached, or T19 has an approved deferral from the final score.
The toolbar popup and context-menu flows produce observable memory results or branded recoverable failures.
CORS-denied, missing-token, and unpaired-extension states provide a clear setup path instead of a generic HTTP 500 or raw MISSING_TOKEN; focused code coverage exists, with live screenshot evidence still optional for judging.
Memory source/provenance labels remain consistent between frame lists, search results, cockpit UI, and any separately scored global-recall result shape.
Popup keyboard/focus basics have live evidence; popup copy and screen-reader announcement behavior are inspected or deferred for the final judge packet.

P2-1: Stale developer comments and legacy terminology can mislead future work

Evidence:

Several comments still refer to Pro gates even when current user-facing strategy changed.
Some route comments are stale relative to implementation.

Why it matters:

This is not directly user-facing, but it increases the chance of reintroducing incorrect UX copy.

Correction:

After user-facing fixes land, clean stale comments in touched files only.

Acceptance:

Comments in touched files describe current behavior.

P2-2: Browser plugin DOM snapshot failed, although screenshot/evaluate worked

Evidence:

Browser setup succeeded.
domSnapshot() failed with TypeError: o.incrementalAriaSnapshot is not a function.
Fallback browser screenshot/evaluate and standalone Playwright testing worked.

Why it matters:

This is a tooling issue, not a product UX issue, but it reduces inspection ergonomics.

Correction:

Continue using Playwright CLI for audit evidence until the Browser plugin version mismatch is fixed.

Acceptance:

UX audit can be repeated without relying on the failing Browser DOM snapshot path.

Proposed Implementation Phases After Approval

Phase 1: Trust and coherence blockers

Keep Clerk/CSP/accountless local mode in regression; sampled accountless console health is currently verified fixed.
Remove/normalize user-facing Pro copy.
Keep mobile Settings layout and first-run onboarding Profile action reachability in regression; both are currently verified fixed in focused 390px journeys.
Keep the verified Ctrl+Shift+N route contract in regression.
Keep Workspace Switcher route traversal in regression.
Keep the verified visual snapshot lane in regression.
Verify: typecheck, full web Vitest, build, full-product audit, visual suite, desktop/mobile screenshots.

Phase 2: Flow polish and accessibility

Replace native confirm/alert/prompt usage in critical flows.
Add in-app result states for backup/restore/export/delete/revoke.
Make marketplace browse/search hermetic in core verification and move live external sync to an explicit lane.
Focused form label and keyboard audit.
Add route coverage manifest and smoke/visual coverage for thin routes.
Add state/failure bundle evidence for the five-persona judge runs.
Verify: targeted tests for destructive flows, accessibility assertions where practical, mobile screenshots.

Phase 3: Performance and polish

Route-level lazy loading for deep apps.
Fix ineffective dynamic imports.
Optimize persona/media assets.
Add/clarify local model pricing semantics for Usage & Cost.
Reduce test warning noise.
Verify: build chunk report, smoke tests, full app tests.

Approval Packet

These tickets are the proposed correction scope. Approval should mean "implement these in order, without creating new product surfaces."

T1: Local auth, Clerk, and CSP console health

Files to inspect/edit:

packages/server/src/local/security-middleware.ts
packages/server/tests/local/security-middleware.test.ts
apps/web/src/lib/clerk.ts
apps/web/src/providers/WaggleClerkProvider.tsx
E2E harness env setup for local/accountless runs, if needed.

Decision needed:

Preferred product behavior is accountless local-first desktop/web by default. In that mode, the app should not mount Clerk or emit Clerk network/CSP errors.
If Clerk is intentionally enabled for a run, CSP must explicitly support the exact Clerk script/connect/frame requirements and the inline startup script must use a nonce/hash-compatible path.

Acceptance:

full-product-audit and phase-ab-verification focused console-health checks have zero critical console errors caused by auth/CSP in the sampled accountless lane; expand the same proof to any final full-run bundle before scoring.
Accountless local mode has no Clerk load, 429, telemetry, or CSP console noise.
Auth still works when a valid Clerk key is intentionally enabled.

T2: Mobile Settings and first-run onboarding responsive layout

Files to inspect/edit:

apps/web/src/components/os/apps/SettingsApp.tsx
apps/web/src/components/os/overlays/OnboardingWizard.tsx
apps/web/src/components/os/overlays/onboarding/WhoAreYouStep.tsx
Existing Settings tests, including apps/web/src/test/pr5-settings-reskin.test.tsx
Add or extend one mobile Playwright assertion for 390px width.

Correction:

Replace the fixed left tab rail at narrow widths with a mobile-appropriate tab selector or wrapped segmented rail.
Prevent provider cards and form rows from creating internal horizontal scroll.
Preserve desktop density.
Keep first-run onboarding Profile primary actions visible, sticky, or clearly signposted at 390 x 844; current focused evidence passes this requirement on port 34194.

Acceptance:

At 390 x 844, Settings is fully usable with no horizontal page overflow, no clipped provider/model/billing/profile controls, and no squeezed two-rail content column; current focused evidence passes this requirement on port 34195.
At 390 x 844, clean-data onboarding Profile keeps Continue visible or clearly reachable without a hidden dead-end feeling; current focused evidence passes this requirement on port 34194.
Keyboard focus remains visible and ordered.
Desktop Settings does not regress.

T3: Pricing and gating copy: Solo/Team/Enterprise only

Files to inspect/edit:

apps/web/src/components/os/apps/MarketplaceApp.tsx
apps/web/src/components/os/apps/mcp/AddCustomMcpForm.tsx
apps/web/src/lib/command-catalog.ts
apps/web/src/components/os/overlays/LoginBriefing.tsx
apps/web/src/components/os/apps/skills/SkillRow.tsx
apps/web/src/components/os/apps/SettingsApp.tsx
apps/web/src/components/os/apps/PaymentSuccessApp.tsx, only for legacy billing clarity.

Correction:

Remove user-facing Pro gating from active product flows.
Use Solo, Teams, and Enterprise language that matches packages/shared/src/tiers.ts and AGENTS.md.
Keep legacy Pro references only where they are explicitly servicing historical billing state.

Acceptance:

rg -n "Pro|PRO|pro_" apps/web/src has no active user-facing Pro upgrade copy outside approved legacy billing handling.
Marketplace, MCP, Settings, skills, command palette, login briefing, and payment success are mutually consistent.

T4: `Ctrl+Shift+N` and Workspace Switcher route contract

Files to inspect/edit:

apps/web/src/hooks/useKeyboardShortcuts.ts
apps/web/src/components/os/AppShell.tsx
apps/web/src/components/os/overlays/WorkspaceSwitcher.tsx
tests/e2e/phase-ab-verification.spec.ts
tests/e2e/power-user-stress.spec.ts
tests/e2e/full-wiring-audit.spec.ts

Recommended decision:

Make Ctrl+Shift+N complete the user's likely action: open chat in the active workspace, else the Home "Start Here" workspace, else the newest workspace. If no workspace exists, then open the Workspace Switcher or creation path.

Correction:

Resolve the mismatch between tests and product behavior.
Ensure route-changing nav closes blocking overlays or makes overlays non-blocking where appropriate.
Preserve focus trap behavior while a modal is intentionally open.

Acceptance:

The two formerly failing Ctrl+Shift+N tests pass; current focused evidence is 2/2 on port 34197.
The full console audit can navigate from Home to Memory without a lingering Workspace Switcher backdrop intercepting clicks; current focused evidence is 1/1 on port 34198.
Modal close and selection behavior is predictable with mouse and keyboard.

T5: Visual baseline triage

Files to inspect/edit:

tests/visual/views.spec.ts
Existing screenshot baseline directories referenced by the visual test runner.
Any directly implicated app view only if the actual screenshot shows a real regression.

Correction:

Classification complete in docs/audits/2026-07-08-visual-t5-classification.md: most original failures were intentional drift; the current baseline/spec worktree passes 14/14 on port 34199.
Preserve the current canonical ASCII-hyphen baseline family used by the runner.
Keep duplicate historical baseline families documented until a separate test-readiness cleanup removes or archives them.

Acceptance:

Visual suite passes after approved baseline updates or real UI fixes; current evidence is 14/14 on port 34199.
A short note records which snapshots were updated versus fixed.
The active baseline family is unambiguous for the current runner.

T6: Marketplace determinism and local-first test lane

Files to inspect/edit:

packages/server/src/local/routes/marketplace.ts
packages/server/src/local/marketplace-background-sync.ts
packages/server/tests/local/marketplace-background-sync.test.ts
E2E tests that call /api/marketplace/sync.

Correction:

Keep WAGGLE_DISABLE_MARKETPLACE_SYNC=1 respected by explicit sync routes in test/local verification, and route live external sync into a separately named live-integration lane.
Keep browse/search useful with local catalog data.

Acceptance:

Standard UX audit does not hit external marketplace sources when the sync-disable env is set; server-side evidence passes.
Marketplace search/sync evidence is stable in the focused browser slice: 4/4 passed on port 34203.

T7: Trust-critical dialogs

Files to inspect/edit:

apps/web/src/components/os/apps/BackupApp.tsx
apps/web/src/components/os/apps/AutomationCenterApp.tsx
apps/web/src/components/os/apps/SettingsApp.tsx
packages/admin-web/src/pages/Members.tsx

Correction:

Replace native confirm, alert, and prompt in destructive or trust-critical flows with existing app modal/toast patterns.
Show what will happen, require explicit confirmation, and show result state after the action.

Acceptance:

No native browser dialogs in backup, restore, export, revoke, delete, or destructive workspace/memory/wiki flows.
Result state is visible in-app.

T8: Performance and payload polish

Files to inspect/edit:

Route/app composition in apps/web/src/components/os/ and apps/web/src/components/os/apps/
apps/web/src/lib/shape-selection.ts
Asset import sites for persona and logo media.

Correction:

Lazy load deep admin/productivity apps where that fits the existing shell.
Fix the dynamic import defeated by static import.
Reduce initial payload from large persona/logo assets.

Acceptance:

Startup JS chunk is materially smaller and below the Vite warning threshold.
Home and core shell render before deep Settings/Marketplace/admin/chat/overlay/analytics code is needed.
Build warnings for defeated dynamic imports and oversized JS chunks stay resolved or explicitly accepted with rationale.

T9: Local model pricing semantics

Files to inspect/edit:

packages/agent/src/cost-tracker.ts
packages/agent/tests/cost-tracker.test.ts
Any UI that labels unknown local model cost estimates.

Correction:

Add explicit behavior for local/unknown Ollama models: either known pricing, free/local label, or "unpriced estimate unavailable."
Make no-LLM tests force provider unavailability if they are testing graceful degradation.

Acceptance:

Standard audit logs do not warn about ollama/minimax-m2.7:cloud.
Cost UI does not imply precise paid-provider cost for an unpriced local model.

T10: Test warning and accessibility hygiene

Files to inspect/edit:

Tests emitting repeated act(...) warnings.
Core form components in Settings, onboarding, workspace creation, agent/persona creation, compliance templates.

Correction:

Remove warning noise where it masks real UX failures.
Add focused label/name checks for high-traffic forms.
Apply the shared overlay contract to shell panels/dialogs: semantics, accessible names, Escape or explicit close, focus behavior, and named icon-only actions.
Keep focused component tests for fixed controls and keep the runtime axe gate green before final judge scoring. Current sampled gate: tests/e2e/runtime-a11y.spec.ts, 2/2 on port 34193.

Acceptance:

Standard verification output is short enough that real failures are visible.
Core forms have accessible names, associated labels, and predictable focus order.
Sampled shell overlays/routes have accessible names/landmarks, close predictably, expose zero unnamed visible icon-only controls, and pass the desktop/mobile runtime axe gate.

T11: Route coverage and final judge harness

Files to inspect/edit:

docs/audits/2026-07-08-ux-route-scenario-manifest.md
apps/web/src/App.tsx
tests/e2e/user-journeys.spec.ts
tests/e2e/full-product-audit.spec.ts
tests/visual/views.spec.ts
tests/vision/personas.spec.ts
tests/vision/README.md
docs/audits/2026-07-08-route-evidence-t11-analysis.md

Correction:

Maintain the route coverage manifest that maps every registered route and major overlay to a smoke, visual, or persona-journey check.
Keep the codified route owners for /benchmarks, /platform, /payment-cancelled, Launcher watch mode, Usage & Cost, and selected thin judge routes in tests/e2e/user-journeys.spec.ts.
Use the app-local test command shape for app tests: npm run test -w apps/web -- ...; the root Vitest command excludes apps/**.
Extend the final judge process so the five persona scores are backed by route evidence, not only chat transcript quality.

Acceptance:

Every registered route has an evidence owner before final judge scoring; current focused evidence is J-route-coverage 2/2 on port 34200.
The five-persona judge packet includes screenshots, console status, route coverage, and 10-point scorecard results.
No route is silently excluded from the final 9/10 claim.

T12: State and failure bundle evidence

Files to inspect/edit:

docs/audits/2026-07-08-ux-state-failure-scenario-matrix.md
tests/e2e/user-journeys.spec.ts
tests/vision/personas.spec.ts
New focused failure-injection tests if needed.

Correction:

Record the account mode, billing tier, UI disclosure tier, model state, data state, offline/error state, and viewport for each judge run.
Do not award 9/10 from default-state screenshots alone.
Include first-run onboarding states: fresh clean data, incomplete wizard, high-volume import detected, model ready/no-model, first-task auto-send, mobile Profile, and post-completion reopen.
Track any remaining first-run work as broader state/recovery evidence. The focused polish items from docs/audits/2026-07-08-first-run-onboarding-t1-t2-t12-analysis.md are fixed for high-volume import deliberate review, non-contradictory model-ready copy, and immediate first-task composer clear/disable.
Include shell overlay states from docs/audits/2026-07-08-shell-overlays-t10-t12-analysis.md: Notification Inbox empty/unread, Create Workspace desktop/mobile, Context Rail loading/empty/expanded, Onboarding Tooltips suppressed/visible, and tier modal close paths.

Acceptance:

Every judge scorecard cites its state bundle.
Any skipped state bundle has an approved deferral.

T13: Launch funnel UX gate

Files to inspect/edit:

apps/www
apps/www/app/_components/Pricing.tsx
apps/www/app/api/stripe/checkout/route.ts
apps/www/app/(legal)/**
.github/workflows/deploy-www.yml

Correction:

Verify or fix download, mobile nav, sign-in/sign-up/account, real deployed Team checkout, legal trust pages, and deploy target.
Keep this out of Phase 1 unless directly required by a Phase 1 verification failure.
Current evidence: focused T13 supplement records passing www test/typecheck/build, fresh localhost route/API smoke, rendered Browser checkout-recovery evidence, and local checkout/legal/download/deploy-workflow fixes. The canonical domain still does not resolve, real signed installer/release artifacts are not published, deployed Vercel/DNS smoke is missing, deployed Clerk/Stripe success/cancel evidence is missing, and formal legal sign-off is not proved.

Acceptance:

Launch funnel evidence is attached, or T13 has an approved deferral from the final score.

T14: Desktop wrapper and release UX gate

Files to inspect/edit:

app/src-tauri/**
app/tests/**
app/scripts/**
.github/workflows/release.yml
packages/server/src/local/service.ts
scripts/build-sidecar.mjs

Correction:

Verify or fix packaged startup, sidecar readiness/recovery, tray actions, close-to-tray, global shortcut, installer/signing expectations, and update visibility.
Keep this out of Phase 1 unless directly required by a Phase 1 verification failure.
Current evidence: desktop TypeScript, static app helper tests, static Tauri config/update/tray tests, sidecar resource preflight, Rust cargo check, web-side Tauri binding tests from apps/web, web typecheck, release workflow ordering, and service-level startup/chat/workspace E2E pass. Native source directly handles tray Open/focus, close-to-tray, Ctrl+Shift+W, and Quit; Settings routes through a tested /settings bridge; Pause and About are hidden. Packaged Tauri interaction evidence and visible update/service recovery UX remain missing.
T14 deep-dive evidence: docs/audits/2026-07-08-desktop-wrapper-t14-analysis.md.

Acceptance:

Desktop wrapper evidence is attached, or T14 has an approved deferral from the final score.

T15: Admin and CLI utility UX gate

Files to inspect/edit:

packages/admin-web/**
packages/cli/**
packages/launcher/**
packages/marketplace/**
packages/memory-mcp/**
packages/hive-mind-mcp-server/**
packages/hive-mind-cli/**
vitest.config.ts

Correction:

Keep admin-web UX beyond the rendered package proof green as states evolve; verify or fix launcher browser-open fallback and service-crash recovery, marketplace CLI help/error/JSON states, memory MCP/hive-mind MCP stdio and missing-config behavior, Waggle CLI command/auth flows, and hive-mind CLI broader package/publish behavior.
Keep the built legacy waggle-memory-mcp read/write entry guarded or explicitly deprecate it; marketplace package manifest/pack/install shape, launcher packed/installed first-command recovery, @waggle/cli built/packed/local package-closure installed npx help plus local REPL startup/slash-command/exit and streamed chat/provider plumbing, local package-closure installed hive-mind-cli help, installed legacy memory MCP read-only startup, and installed hive-mind MCP read-only startup are locally fixed but still need registry-only proof after internal package publication.
Maintain the safe help/invalid-command behavior now guarded for hive-mind CLI and marketplace CLI; expand evidence to clean package/publish runs.
Keep this out of Phase 1 unless directly required by a Phase 1 verification failure.
Current evidence: docs/audits/2026-07-08-admin-cli-utility-t15-analysis.md records passing targeted source tests, no-emit TypeScript, builds, focused marketplace/launcher/@waggle/cli/hive-mind CLI fixes, focused legacy memory MCP read/write protocol proof plus installed read-only startup, passing built hive-mind MCP write-scope proof plus installed read-only startup, and admin-web evidence proving package-local tests, warning cleanup, 390px shell layout, hash navigation, active semantics, browser back/forward traversal, labelled table scroll regions, rendered control labels, mobile shell keyboard reachability, page-level keyboard traversal from connection fields into the covered admin pages, desktop/mobile visual snapshots, capability governance form coverage, malformed analytics response recovery, all-page initial API-failure recovery, rendered mutation/destructive-failure recovery for capability policy save, capability override create/remove, capability request decision, member invite, member role change, member removal, and team settings save, plus real local bearer-auth middleware wrong-token/valid-token behavior. Launcher evidence now covers invalid-port validation before service setup, occupied-port recovery copy, --no-open success copy, packed first-command help, clean installed occupied-port startup recovery, and clean installed long-running /health startup; marketplace evidence now covers package manifest/packed-file alignment plus installed help/invalid-command; @waggle/cli evidence now covers packed bin help plus local package-closure installed npx help, installed local REPL startup/slash-command/exit, and installed streamed chat/provider plumbing against a mock LiteLLM-compatible endpoint; hive-mind CLI evidence now covers local package-closure installed help and a fixed missing @waggle/shared runtime dependency; legacy memory MCP evidence now covers a fixed missing glob runtime dependency in @waggle/core. Remaining utility evidence is registry-only proof after internal package publication.

Acceptance:

Utility/admin evidence is attached, broken built entries are fixed/deprecated, or T15 has an approved deferral from the final score.

T16: AI-tool hook lifecycle UX gate

Files to inspect/edit:

apps/web/src/components/os/apps/LauncherApp.tsx
packages/shared/src/tool-detection.ts
packages/shared/tests/tool-manifests.test.ts
packages/agent/src/tool-launcher.ts
packages/agent/tests/tool-launcher.test.ts
packages/agent/tests/phase4-hooks-cohort.test.ts
packages/hive-mind-hooks-*/**

Correction:

Verify or fix hook install/verify/uninstall UX for claude-code, codex, codex-desktop, cursor, hermes, and openclaw; verify unsupported-but-launchable copy for claude-desktop.
Keep this out of Phase 1 unless directly required by a Phase 1 verification failure.
Current evidence: shared/agent/server T16 route-contract plus package-runtime tests pass 7 files / 96 tests; the package-runtime lane builds and packs the local hook package closure, installs it into a clean temp project, then runs npx --yes @waggle/hive-mind-hooks-<id> install/verify/uninstall for all six hook-capable packages; the focused registry-aware adapter launch slice passes 5 files / 108 tests; Launcher/prompt/adapter regressions pass 5 files / 72 tests; root-run hook/shim package tests pass 60 files / 573 tests with 1 skipped; hook/shim package-local scripts pass; official package typechecks pass 8/8; Claude Desktop stub build passes; compiled hook-bin help smokes pass 6/6; focused no-reconnect observed-output and hook-result stdout/stderr/Backup-Recovery/Check-failed/uninstall-cleanup/More-output/structured-failure/empty-output/launch-only/third-party-launch coverage passes; focused rendered Launcher Browser smoke artifacts live under output/playwright/launcher-t16-54147/; a fresh real Browser route smoke rendered /launcher and Verify now shows verify failed (exit 1) plus retry/uninstall/reinstall guidance instead of HTTP 400; Browser-rendered mocks now show Claude Desktop launch-only copy, Codex Verify manual-approval rows without raw [FAIL], and Codex Uninstall cleanup labels without Install pointer; Playwright rendered coverage proves standard install changed-file/pointer/backup/recovery labels, all six hook-capable install/verify/uninstall transitions, sidecar-offline Retry, long-stderr summarization, non-built-in adapter launch-only/prompt behavior, and real OpenClaw observed launch/output/exit/process cleanup; gated route coverage proves real /api/tools/hooks install/verify/uninstall for all six hook-capable tools against an isolated profile after fixing Windows hook npx resolution; Windows npm shims now resolve to Node module targets when possible, with a quoted fallback for unknown .cmd/.bat files; Tailwind motion-token ambiguity warnings and the shape-selection.ts dynamic/static import warning are fixed and guarded. Packaged desktop hook-status evidence and remaining warning-noise gaps remain. Focused supplement: docs/audits/2026-07-08-ai-tool-hook-t16-analysis.md.
2026-07-09 T16 update: Codex WindowsApps launchability/recovery is now fixed at the UX contract layer. The restricted WindowsApps alias is detected as installed but not launchable, and Launcher shows recovery copy instead of a Launch button. T16 remains open for packaged desktop hook-status evidence and warning hygiene.

Acceptance:

Hook lifecycle evidence is attached, or T16 has an approved deferral from the final score.

T17: Developer API, background worker, and substrate verification UX gate

Detailed supplement: docs/audits/2026-07-08-developer-substrate-t17-analysis.md.

Files to inspect/edit:

packages/sdk/**
packages/server/**
packages/worker/**
packages/waggle-dance/**
packages/hive-mind-core/**
packages/hive-mind-shim-core/**
packages/hive-mind-wiki-compiler/**
packages/wiki-compiler/**
packages/*/package.json
vitest.config.ts

Correction:

Verify or fix SDK, server, worker, WaggleDance, substrate, wiki compiler, optimizer, shared, and agent package verification lanes.
Keep this out of Phase 1 unless directly required by a Phase 1 verification failure.
Current evidence: all 13 remaining workspaces pass no-emit TypeScript; agent tests pass 195 files / 3097 tests; root-run package suites pass for SDK/substrate/shared/worker/WaggleDance/compiler lanes; the dedicated server performance lane passes 13/13; the named server release lane passes 185 files / 2128 tests with one worker; package-local SDK/substrate test scripts pass; Playwright webServer startup is fixed by pinning tsx@4.21.0, and the focused marketplace browser slice now passes 4/4 on port 34203.

Acceptance:

Developer API/background/substrate verification evidence is attached, or T17 has an approved deferral from the final score. The previously identified agent-group dead-end is now corrected: local group runs execute through the sidecar job store with worker progress/cancel/status, and cloud group runs queue the worker-supported group job type; focused route/orchestrator coverage passes 19/19.

T18: Ops, deployment, CI, benchmark, and judging evidence gate

Detailed supplement: docs/audits/2026-07-08-ops-deploy-ci-judging-t18-analysis.md.

Files to inspect/edit:

.github/workflows/**
Dockerfile
docker-compose.yml
docker-compose.production.yml
render.yaml
litellm-config.yaml
ops/**
benchmarks/**
judging/**
vitest.infra.config.ts
vitest.infra-suites.ts

Correction:

Verify or fix Docker/Compose, Render, LiteLLM, CI, infra-test, benchmark, and judging evidence lanes.
Keep this out of Phase 1 unless directly required by a Phase 1 verification failure.
Current evidence: YAML parse passes, Compose config validates, env/key files checked here are ignored except examples, benchmark harness TypeScript passes, package-local and root-run benchmark tests pass 29 files / 325 tests, the five-journey CI browser smoke is blocking while the broad E2E job remains advisory, no CI infra lane exists, local Docker engine is unavailable for npm run test:infra, Render sidecar mode is explicit, and current judging/ artifacts are historical.

Acceptance:

Ops/deployment/CI/benchmark/judging evidence is attached, or T18 has an approved deferral from the final score.

Five-Persona Judge Gate

Existing useful harness:

tests/vision/personas.spec.ts already exercises five persona-style journeys with real local LLM state, screenshots, and artifact output.
That harness is useful, but it is not yet a complete 9/10 scoring judge. It should be used as a base and extended with explicit scorecards or a separate review pass.
The detailed scorecard packet lives in docs/audits/2026-07-08-five-persona-judge-scorecards.md.

Judge personas:

Solo founder: first run, accountless mode, Home Start Here, workspace creation, first chat, return later to a remembered next move.
Researcher/analyst: import or browse memory, search, provenance, wiki/timeline, verify no cross-workspace leakage.
Engineer/power user: command palette, Ctrl+Shift+N, Launcher/MCP, file/tool actions, logs, keyboard-heavy navigation.
Nontechnical team admin: Settings, billing language, vault, backup, approvals, workspace/team management, clear recovery from errors.
Mobile executive: Home, Settings, account/billing, memory glance, theme/profile controls at 390px width.

Scorecard, 10 points total:

Functional completion: 2 points.
Flow, information architecture, and discoverability: 2 points.
Trust, error handling, and recovery: 2 points.
Visual quality, accessibility, and responsive behavior: 2 points.
Performance and polish: 1 point.
Memory, personalization, and domain fit: 1 point.

Pass condition:

Each of the five personas scores at least 9/10.
No persona has any scorecard dimension below 8/10 when normalized.
No open P0 findings remain.
Any P1 findings left open have explicit deferral rationale and do not affect the judged journey.
T13/T14/T15/T16/T17/T18/T19 are fixed or explicitly deferred before claiming complete product UX.

Verification Gate For Phase 1

Run after the approved Phase 1 fixes:

npm run typecheck:web
npm run ux:contrast
npm run ux:color-guard
cd apps/web
npx vitest run
cd ../..
npm run build

Then run the focused browser lane on a fresh port:

$env:WAGGLE_E2E_PORT='3391'
$env:WAGGLE_E2E_BASE_URL='http://127.0.0.1:3391'
$env:WAGGLE_E2E_SKIP_LITELLM='1'
node node_modules/playwright/cli.js test tests/e2e/full-product-audit.spec.ts tests/e2e/full-wiring-audit.spec.ts tests/e2e/phase-ab-verification.spec.ts tests/e2e/power-user-stress.spec.ts tests/e2e/user-behavior.spec.ts tests/visual/views.spec.ts --project=chromium --reporter=list

Manual/visual checks:

Mobile Home, Settings general/models/billing/profile, Memory, workspace chat, Command Center, and Workspace Switcher at 390 x 844.
Desktop Home, Settings, Marketplace/MCP, Memory, Chat, Launcher.
Route coverage manifest updated with evidence owner for every registered route.
Five-persona judge dry run before asking for final 9/10 approval.
Judge runbook evidence folders filled or linked before accepting final scores.

Recommended Approval Decision

Approve Phase 1 first. It has the smallest scope that can move the product from "mostly functional" to "credible enough to judge." Phase 1 should be capped to the directly implicated files and should not become a broad redesign.

Decision brief: docs/audits/2026-07-08-ux-approval-brief.md.

Implementation plan for that approval: docs/superpowers/plans/2026-07-08-ux-phase-1-corrections.md.

Track closure in: docs/audits/2026-07-08-ux-correction-register.md.

Do not start new surfaces. Improve the existing Home, Settings, Marketplace/MCP, auth/accountless, and verification surfaces first.

159 KiB Raw Blame History

Complete UX and Usage Audit - 2026-07-08

Phase 1 Implementation Evidence

Executive Verdict

Evidence Collected

Judge Personas

Route and Surface Coverage Matrix

Usage Scenarios Covered

First Run and Account

Home and Daily Continuity

Workspace, Chat, Files, and Artifacts

Memory, Wiki, Trust, and Provenance

Agents, Personas, and Automations

Skills, Connectors, MCP, Marketplace, and Launcher

Team, Approvals, Billing, Vault, Backup, and Compliance

Settings and Narrow Viewports

Findings and Correction Backlog

P0-1: Accountless Clerk/CSP console health (verified fixed)

P0-2: Mobile Settings structural usability (verified fixed)

P0-3: Cockpit pricing copy uses Solo/Team/Enterprise (verified fixed)

P0-4: Visual regression suite for tracked views (verified fixed)

P0-5: Ctrl+Shift+N route contract (verified fixed)

P0-6: Workspace Switcher route traversal (verified fixed)

P0-7: Mobile first-run onboarding primary action reachability (verified fixed)

P1-1: Native browser dialogs interrupt branded workflows

P1-2: Test output is too noisy

P1-3: Marketplace and local-first test determinism need tightening

P1-4: Initial app payload is too heavy for a seamless desktop feel

P1-5: Form, focus, and icon-button accessibility need a focused pass

P1-6: Local model pricing and graceful-degradation harness are inconsistent

P1-7: Route coverage ownership for the full shell (verified fixed)

P1-8: State and failure-mode coverage is not explicit enough for judge scoring

P0-L1: Public launch funnel has release-blocking smoke and recovery gaps

P1-10: Desktop wrapper, installer, tray, update, and sidecar-startup UX lack installed-app evidence

P1-11: Admin web, CLI launcher, marketplace CLI, and memory MCP utility UX have broken built-entry and shallow UX evidence

P1-12: AI-tool hook install/verify/uninstall UX has strong route/rendered evidence but lacks packaged desktop proof

P1-13: Developer API, background worker, and substrate verification UX has command-shape and suite-stability gaps

P1-14: Ops, deployment, CI, benchmark, and judging evidence is incomplete for a complete-system claim

P1-15: Browser Companion extension UX is a real capture surface with incomplete end-to-end UX evidence

P2-1: Stale developer comments and legacy terminology can mislead future work

P2-2: Browser plugin DOM snapshot failed, although screenshot/evaluate worked

Proposed Implementation Phases After Approval

Approval Packet

T1: Local auth, Clerk, and CSP console health

T2: Mobile Settings and first-run onboarding responsive layout

T3: Pricing and gating copy: Solo/Team/Enterprise only

T4: Ctrl+Shift+N and Workspace Switcher route contract

T5: Visual baseline triage

T6: Marketplace determinism and local-first test lane

T7: Trust-critical dialogs

T8: Performance and payload polish

T9: Local model pricing semantics

T10: Test warning and accessibility hygiene

T11: Route coverage and final judge harness

T12: State and failure bundle evidence

T13: Launch funnel UX gate

T14: Desktop wrapper and release UX gate

T15: Admin and CLI utility UX gate

T16: AI-tool hook lifecycle UX gate

T17: Developer API, background worker, and substrate verification UX gate

T18: Ops, deployment, CI, benchmark, and judging evidence gate

Five-Persona Judge Gate

Verification Gate For Phase 1

Recommended Approval Decision

159 KiB

Raw Blame History

P0-5: `Ctrl+Shift+N` route contract (verified fixed)

T4: `Ctrl+Shift+N` and Workspace Switcher route contract