waggle-os/docs/audits/2026-07-08-state-failure-t12-analysis.md

# Focused T12 State and Failure Bundle Analysis

Status: implementation follow-up in progress; this file now records guarded scorecard contracts and current browser evidence.

Purpose: the state/failure matrix defines the dimensions; this supplement records the current command/source evidence and converts T12 into the concrete bundle contract each judge persona must satisfy before a 9/10 score is credible.

Guideline source refreshed during this pass: Vercel Web Interface Guidelines, `https://raw.githubusercontent.com/vercel-labs/web-interface-guidelines/main/command.md`. The rules most relevant to T12 are destructive actions using confirmation/undo instead of immediate action, URL-state clarity for stateful UI, inline error/focus behavior, visible focus, long-content handling, and large-list performance.

## Current Command Evidence

| Evidence | Result | What it proves | What it does not prove |
|---|---:|---|---|
| `npm run test -w apps/web -- src/test/p1b-authgate-surfaces.test.tsx src/test/p4-onboarding-status.test.ts src/test/wave-t-onboarding-boot-gate.test.ts src/test/pr7a-billing.test.tsx src/test/phase5b-backup.test.tsx src/test/p7-b4-files-error.test.tsx src/test/phase4b-connector-hub.test.tsx src/test/phase4b-mcp-hub.test.tsx src/test/p7-b1-approvals-error.test.tsx --run` | Pass, 9 files / 79 tests | Auth-gate surfaces, onboarding status/boot gates, billing/payment states, backup status classification, files errors, connector hub, MCP hub, and approvals error cases have healthy focused component/unit coverage. | It is not a rendered route, viewport, console, or five-persona state-bundle run. |
| `tests/e2e/failure-injection/network-drop.spec.ts` source inspection | Existing coverage found | Chat SSE hard drop, truncated stream after one token, and retry after dropped stream are represented as browser failure-injection scenarios. | The spec was not rerun in this focused T12 pass, and it covers chat stream failure rather than sidecar/offline/Stripe/marketplace failure broadly. |
| Fresh built-app route smoke, port 3407 | Mixed | `/payment-cancelled` redirects to `/settings?tab=billing`, which is useful Team Admin payment-recovery state evidence; `/benchmarks` and `/platform` also render meaningful shell content. | The smoke is ad hoc, not codified; all three routes emit the existing T1 CSP/Clerk console errors, and payment recovery copy still needs judge inspection. |
| Fresh thin-route smoke, port 3411 | Mixed | Engineer, Researcher, Team Admin, and Mobile Executive state bundles now have ad hoc rendered evidence for Launcher/watch, WaggleDance, Artifacts, Profile, Timeline, Payment Success fallback, Automations, MCP Hub, Usage & Cost, and Files. | The smoke is ad hoc, not codified; all routes emit T1 CSP/Clerk errors, Launcher can log a detect-in-flight adapter error, Usage & Cost logs a 403 resource error, and primary workflow states are still untested. |
| `J-route-coverage` Playwright tests, port 34139 | Pass, 2 tests | Codifies route-level state inputs for Engineer (`/launcher`, `/launcher?watch=1`, `/mcps`, `/files`), Researcher (`/artifacts`, `/settings/timeline`), Team Admin (`/payment-success`, `/payment-cancelled`), and Mobile Executive/Profile (`/settings/profile`, `/settings/usage`). | This is rendered shell/recovery-copy proof only; it does not exercise destructive actions, real checkout provider return, Launcher hook lifecycle, MCP install/verify, file upload, or failure-state bundles. |
| Fresh Mobile Executive smoke, port 3419 | Mixed | A 390 x 844 rendered lane now has screenshots for Home, Settings general/models/billing/profile, Memory, workspace chat, Command Center, and Workspace Switcher. It proves the routes render and that Workspace Switcher can close with Escape on mobile. | The smoke is ad hoc, not codified; every route still emits T1 CSP/Clerk errors; Settings has visible clipped/squeezed controls despite clean document scroll width; Memory/chat tab strips overflow; Command Center remained visible after Escape and logged a missing dialog description warning. |
| `npx playwright test tests/e2e/five-persona-state-bundles.spec.ts --project=chromium --reporter=list`, port 34245 | Pass, 5 tests | Codifies rendered state bundles plus one failure probe for all five judge personas and writes `state-bundle.md`/`.json` plus screenshots under `output/playwright/five-persona-state-bundles/`. Current capture has 0 critical console errors, 0 page errors, 0 critical network failures, 0 visible overflow, and overlay close proof where required. | It is still a no-LLM/accountless smoke; it does not prove authenticated Teams, real checkout, packaged desktop, hook lifecycle, file upload, every destructive/failure recovery path, or final judge scoring. |
| `npx playwright test tests/e2e/five-persona-state-bundles.spec.ts --project=chromium --grep "mobile-executive" --reporter=list`, port 34244 | Pass, 1 test | Rebuilt after the workspace tab and chat agent-strip mobile fixes; regenerated Mobile Executive evidence now reports 0 visible horizontal overflow items on Home, Settings, Memory, and mobile chat backend-offline failure at 390 x 844. | It only rechecks the Mobile Executive bundle; broader authenticated/mobile Team paths still need targeted evidence. |
| `npx playwright test tests/e2e/five-persona-state-bundles.spec.ts --project=chromium --reporter=list --retries=0`, port 34270 with matched `WAGGLE_E2E_BASE_URL` | Pass, 5 tests | Regenerates all five persona bundles with route, failure, and overlay evidence. Mobile Executive captures Notification Inbox plus Command Center screenshots; Engineer captures Marketplace unavailable; Team Admin captures backup restore failure. The run enforces 0 critical console/page/network failures and 0 visible overflow across routes, failures, and overlays. | It is still an accountless/no-LLM bundle; authenticated Teams, real checkout, packaged desktop, hook lifecycle, file upload, Stripe/model/health failure states, and final judge scoring remain outside this proof. |
| `npx playwright test tests/e2e/five-persona-state-bundles.spec.ts --project=chromium --grep "engineer-power-user\|team-admin-security-reviewer" --reporter=list --retries=0`, port 34269 with matched `WAGGLE_E2E_BASE_URL` | Pass, 2 tests | Regenerates Engineer and Team Admin bundles with `marketplace-unavailable` and `backup-restore-failure` probes. Both bundles report 0 visible overflow, 0 critical console errors, 0 page errors, and 0 critical network failures, with screenshots under `output/playwright/five-persona-state-bundles/`. | It is still an accountless/no-LLM bundle; Stripe unavailable/cancelled, local model unavailable, health degradation, authenticated Teams, packaged desktop, hook lifecycle, file upload, and final judge scoring remain outside this proof. |
| `npx playwright test tests/e2e/five-persona-state-bundles.spec.ts --project=chromium --grep "engineer-power-user\|team-admin-security-reviewer\|mobile-executive" --reporter=list --retries=0`, port 34272 with matched `WAGGLE_E2E_BASE_URL` | Pass, 3 tests | Focused verification for the newly expanded T12-C probes: Engineer now captures Cockpit `mission-control-health-degraded`, Team Admin captures `billing-checkout-unavailable`, and Mobile Executive captures `local-model-runtime-unavailable`. The run reports 0 critical console/page/network failures and 0 visible overflow for the touched bundles. | It is still an accountless/no-LLM bundle; authenticated Teams, real checkout provider success/cancel, packaged desktop, hook lifecycle, file upload, and final judge scoring remain outside this proof. |
| `npx playwright test tests/e2e/five-persona-state-bundles.spec.ts --project=chromium --reporter=list --retries=0`, port 34273 with matched `WAGGLE_E2E_BASE_URL` | Pass, 5 tests | Regenerates all five persona bundles after the expanded failure probes. The current bundle includes chat backend offline, Memory API unavailable, Launcher sidecar offline, Cockpit health degraded, Marketplace unavailable, billing checkout unavailable, backup create failure, backup restore failure, local model runtime unavailable, mobile chat backend offline, Notification Inbox, Command Center, and Workspace Switcher evidence, with 0 critical console/page/network failures and 0 visible overflow. | It is still an accountless/no-LLM bundle; authenticated Teams, real checkout provider success/cancel, packaged desktop, hook lifecycle, file upload, scale/performance states, and final judge scoring remain outside this proof. |
| `npx playwright test tests/e2e/five-persona-state-bundles.spec.ts --project=chromium --grep "engineer-power-user" --reporter=list --retries=0`, port 34274 with matched `WAGGLE_E2E_BASE_URL` | Pass, 1 test | Focused verification for the Files upload failure lane. The Engineer bundle now captures `files-upload-failure`: a failed `/api/workspaces/:id/files/upload` shows branded `Upload failed` recovery copy and does not render the failed filename as a successful file row. The bundle reports 0 critical console/page/network failures and 0 visible overflow. | It is still an accountless/no-LLM focused slice; authenticated Teams, real checkout provider success/cancel, packaged desktop, hook lifecycle, scale/performance states, and final judge scoring remain outside this proof. |
| `npx playwright test tests/e2e/five-persona-state-bundles.spec.ts --project=chromium --reporter=list --retries=0`, port 34275 with matched `WAGGLE_E2E_BASE_URL` | Pass, 5 tests | Regenerates the full five-persona bundle after the Files upload-failure fix. The current full bundle includes all previous sampled probes plus `files-upload-failure`, and reports 0 critical console/page/network failures and 0 visible overflow. | It is still an accountless/no-LLM bundle; authenticated Teams, real checkout provider success/cancel, packaged desktop, hook lifecycle, scale/performance states, and final judge scoring remain outside this proof. |
| `npm run test -- packages/server/tests/local/files-upload-multipart.test.ts --reporter=dot` | Pass, 1 test | Regresses the real browser multipart boundary that previously returned Fastify `415 FST_ERR_CTP_INVALID_MEDIA_TYPE`; upload now returns `201`, the file entry shape is `/successful-upload.md`, and `/files/list` returns the uploaded file. | This is server-route proof only; it does not prove the rendered Files UI without the Playwright bundle. |
| `npx playwright test tests/e2e/five-persona-state-bundles.spec.ts --project=chromium --grep "engineer-power-user" --reporter=list --retries=0`, port 34278 with matched `WAGGLE_E2E_BASE_URL` | Pass, 1 test | Focused verification for the successful Files upload workflow. The Engineer bundle now captures `files-upload-success`: the upload response is `ok`, returns `{ name: 'successful-upload.md', path: '/successful-upload.md' }`, and the Files UI renders the uploaded filename with 0 visible overflow. | It is still an accountless/no-LLM focused slice; authenticated Teams, real checkout provider success/cancel, packaged desktop, hook lifecycle, scale/performance states, and final judge scoring remain outside this proof. |
| `npx playwright test tests/e2e/five-persona-state-bundles.spec.ts --project=chromium --reporter=list --retries=0`, port 34279 with matched `WAGGLE_E2E_BASE_URL` | Pass, 5 tests | Regenerates the full five-persona bundle after the multipart upload fix. The current Engineer bundle includes `files-upload-failure` and `files-upload-success`; the full run reports 0 critical console/page/network failures and 0 visible overflow. | It is still an accountless/no-LLM bundle; authenticated Teams, real checkout provider success/cancel, packaged desktop, hook lifecycle, scale/performance states, and final judge scoring remain outside this proof. |
| `npx playwright test tests/e2e/five-persona-state-bundles.spec.ts --project=chromium --grep "engineer-power-user" --reporter=list --retries=0`, port 34280 with matched `WAGGLE_E2E_BASE_URL` | Pass, 1 test | Focused verification for a sampled Files large-list state. The Engineer bundle now captures `files-large-list`: `/api/workspaces/:id/files/list` is mocked with 240 files, the Files UI renders `bulk-file-000.md`, shows `240 items`, and reports 0 visible overflow. | It is still one large-list slice; broader memory/event/marketplace/agent/file scale and slow-data states remain outside this proof. |
| `npx playwright test tests/e2e/five-persona-state-bundles.spec.ts --project=chromium --reporter=list --retries=0`, port 34281 with matched `WAGGLE_E2E_BASE_URL` | Pass, 5 tests | Regenerates the full five-persona bundle after the Files large-list probe. The current Engineer bundle includes `files-upload-failure`, `files-upload-success`, and `files-large-list`; the full run reports 0 critical console/page/network failures and 0 visible overflow. | It is still an accountless/no-LLM bundle; authenticated Teams, real checkout provider success/cancel, packaged desktop, hook lifecycle, broader scale/performance states, and final judge scoring remain outside this proof. |
| `npx playwright test tests/e2e/five-persona-state-bundles.spec.ts --project=chromium --grep "engineer-power-user" --reporter=list --retries=0`, port 34282 with matched `WAGGLE_E2E_BASE_URL` | Pass, 1 test | Focused verification for a sampled Marketplace large-catalog state. The Engineer bundle now captures `marketplace-large-catalog`: `/api/marketplace`, `/api/connectors`, and `/api/mcps` are mocked as a 240-entry catalog, and the Marketplace UI renders `Bulk Skill 000` plus grouped counts for `Skills 120`, `Connectors 60`, and `MCPs 60` with 0 visible overflow. | It is still one marketplace scale slice; broader memory/event/agent scale, slow-data states, and deeper performance evidence remain outside this proof. |
| `npx playwright test tests/e2e/five-persona-state-bundles.spec.ts --project=chromium --reporter=list --retries=0`, port 34283 with matched `WAGGLE_E2E_BASE_URL` | Pass, 5 tests | Regenerates the full five-persona bundle after the Marketplace large-catalog probe. The current Engineer bundle reports 7 failure/scale probes, including `marketplace-unavailable`, `marketplace-large-catalog`, `files-upload-failure`, `files-upload-success`, and `files-large-list`; the full run reports 0 critical console/page/network failures and 0 visible overflow. | It is still an accountless/no-LLM bundle; authenticated Teams, real checkout provider success/cancel, packaged desktop, hook lifecycle, remaining broader scale/performance states, and final judge scoring remain outside this proof. |
| `npx playwright test tests/e2e/five-persona-state-bundles.spec.ts --project=chromium --grep "researcher" --reporter=list --retries=0`, port 34284 with matched `WAGGLE_E2E_BASE_URL` | Pass, 1 test | Focused verification for a sampled Memory large-list state. The Researcher bundle now captures `memory-large-list`: bare `/api/memory?...limit=200` is mocked with 200 unique memories, and the Memory Center list renders `200 memories` plus `Bulk Memory 000` with 0 visible overflow and 0 critical console/page/network failures. | It is still one memory scale slice; broader event/agent scale, slow-data states, and deeper performance evidence remain outside this proof. |
| `npx playwright test tests/e2e/five-persona-state-bundles.spec.ts --project=chromium --reporter=list --retries=0`, port 34285 with matched `WAGGLE_E2E_BASE_URL` | Pass, 5 tests | Regenerates the full five-persona bundle after the Memory large-list probe. The current Researcher bundle reports 2 failure/scale probes, including `memory-list-unavailable` and `memory-large-list`; the full run reports 0 critical console/page/network failures and 0 visible overflow. | It is still an accountless/no-LLM bundle; authenticated Teams, real checkout provider success/cancel, packaged desktop, hook lifecycle, remaining broader scale/performance states, and final judge scoring remain outside this proof. |
| `npx playwright test tests/e2e/five-persona-state-bundles.spec.ts --project=chromium --grep "engineer-power-user" --reporter=list --retries=0`, port 34286 with matched `WAGGLE_E2E_BASE_URL` | Pass, 1 test | Focused verification for a sampled Agents large-list state. The Engineer bundle now captures `agents-large-list`: `/api/agents` is mocked with 180 live agents, and the Agents UI renders `180 agents` plus `Bulk Agent 000` with 0 visible overflow and 0 critical console/page/network failures. | It is still one agent scale slice; broader event scale, slow-data states, and deeper performance evidence remain outside this proof. |
| `npx playwright test tests/e2e/five-persona-state-bundles.spec.ts --project=chromium --reporter=list --retries=0`, port 34287 with matched `WAGGLE_E2E_BASE_URL` | Pass, 5 tests | Regenerates the full five-persona bundle after the Agents large-list probe. The current Engineer bundle reports 8 failure/scale probes, including `agents-large-list`, `marketplace-unavailable`, `marketplace-large-catalog`, `files-upload-failure`, `files-upload-success`, and `files-large-list`; the full run reports 0 critical console/page/network failures and 0 visible overflow. | It is still an accountless/no-LLM bundle; authenticated Teams, real checkout provider success/cancel, packaged desktop, hook lifecycle, remaining event scale/slow-data states, and final judge scoring remain outside this proof. |
| `npx playwright test tests/e2e/five-persona-state-bundles.spec.ts --project=chromium --grep "researcher" --reporter=list --retries=0`, port 34289 with matched `WAGGLE_E2E_BASE_URL` | Pass, 1 test | Focused verification for a sampled Timeline/Event large-list state. The Researcher bundle now captures `timeline-large-events`: `/api/events?...limit=500` is mocked with 360 timeline events, and the Timeline UI renders `360 events` plus `Used bulk_tool_000` with 0 visible overflow and 0 critical console/page/network failures. | It is still one event scale slice; slow-data states and deeper performance evidence remain outside this proof. |
| `npx playwright test tests/e2e/five-persona-state-bundles.spec.ts --project=chromium --reporter=list --retries=0`, port 34290 with matched `WAGGLE_E2E_BASE_URL` | Pass, 5 tests | Regenerates the full five-persona bundle after the Timeline/Event large-list probe. The current Researcher bundle reports `memory-list-unavailable`, `memory-large-list`, and `timeline-large-events`; the full run reports 0 critical console/page/network failures and 0 visible overflow. | It is still an accountless/no-LLM bundle; authenticated Teams, real checkout provider success/cancel, packaged desktop, hook lifecycle, slow-data states, deeper performance evidence, and final judge scoring remain outside this proof. |
| `npx playwright test tests/e2e/five-persona-state-bundles.spec.ts --project=chromium --grep "researcher" --reporter=list --retries=0`, port 34291 with matched `WAGGLE_E2E_BASE_URL` | Pass, 1 test | Focused verification for a sampled slow-data state. The Researcher bundle now captures `memory-slow-list`: `/api/memory?...limit=200` is deliberately delayed by 2 seconds, the Memory UI exposes an aria-busy `Loading memories` status while waiting, then renders `40 memories` plus `Bulk Memory 000` with 0 visible overflow and 0 critical console/page/network failures. | It is still one slow-data slice; deeper app-payload/performance evidence remains outside this proof. |
| `npx playwright test tests/e2e/five-persona-state-bundles.spec.ts --project=chromium --reporter=list --retries=0`, port 34292 with matched `WAGGLE_E2E_BASE_URL` | Pass, 5 tests | Regenerates the full five-persona bundle after the slow Memory probe. The current Researcher bundle reports `memory-list-unavailable`, `memory-slow-list`, `memory-large-list`, and `timeline-large-events`; the full run reports 0 critical console/page/network failures and 0 visible overflow. | It is still an accountless/no-LLM bundle; authenticated Teams, real checkout provider success/cancel, packaged desktop, hook lifecycle, broader slow-data/performance states, and final judge scoring remain outside this proof. |
| `node node_modules/playwright/cli.js test tests/e2e/five-persona-state-bundles.spec.ts --project=chromium --grep researcher --reporter=list` | Pass, 1 test | Focused verification for a sampled Researcher export-failure state. The Researcher bundle now captures `wiki-export-obsidian-failure`: `/api/wiki/pages` returns a compiled page, `/api/wiki/export/obsidian` returns `500`, and the Wiki UI renders `Export Failed` plus `Disk permission denied for C:/Research Vault`, with 0 visible overflow and 0 critical console/page/network failures. | It is still one export-failure slice; Notion failure, export permission variants, authenticated memory governance, and final judge scoring remain outside this proof. |
| `node node_modules/playwright/cli.js test tests/e2e/five-persona-state-bundles.spec.ts --project=chromium --reporter=list` | Pass, 5 tests | Regenerates the full five-persona bundle after adding the Researcher export-failure probe. The Researcher bundle now reports 5 failure/workflow/scale/slow-data probes, including `wiki-export-obsidian-failure`; the full run reports 0 critical console/page/network failures and 0 visible overflow. | It is still an accountless/no-LLM bundle; authenticated Teams, real checkout provider success/cancel, packaged desktop, hook lifecycle, broader performance states, and final judge scoring remain outside this proof. |
| `npx playwright test tests/e2e/five-persona-state-bundles.spec.ts --project=chromium --grep "engineer-power-user" --reporter=list --retries=0`, port 34294 with matched `WAGGLE_E2E_BASE_URL` | Pass, 1 test | Focused verification for a sampled Agents slow-data state. The Engineer bundle now captures `agents-slow-list`: `/api/agents` is deliberately delayed by 2 seconds, the Agents UI exposes an aria-busy `Loading agents` status while waiting, then renders `40 agents` plus `Bulk Agent 000` with 0 visible overflow and 0 critical console/page/network failures. | It is still one agent slow-data slice; authenticated Teams, real checkout provider success/cancel, packaged desktop, hook lifecycle, broader performance states, and final judge scoring remain outside this proof. |
| `npx playwright test tests/e2e/five-persona-state-bundles.spec.ts --project=chromium --reporter=list --retries=0`, port 34295 with matched `WAGGLE_E2E_BASE_URL` | Pass, 5 tests | Regenerates the full five-persona bundle after the Agents slow-list probe. The current Engineer bundle reports 9 failure/workflow/scale/slow-data probes, including `agents-slow-list`, `agents-large-list`, `marketplace-unavailable`, `marketplace-large-catalog`, `files-upload-failure`, `files-upload-success`, and `files-large-list`; the full run reports 0 critical console/page/network failures and 0 visible overflow. | It is still an accountless/no-LLM bundle; authenticated Teams, real checkout provider success/cancel, packaged desktop, hook lifecycle, broader performance states, and final judge scoring remain outside this proof. |
| `npx playwright test tests/e2e/five-persona-state-bundles.spec.ts --project=chromium --grep "team-admin-security-reviewer" --reporter=list --retries=0`, port 34297 with matched `WAGGLE_E2E_BASE_URL` | Pass, 1 test | Focused verification for Team Admin checkout return states after the cancelled-checkout banner fix. The bundle now captures `billing-checkout-success-return` with mocked `/api/stripe/sync` returning `TEAMS`, and `billing-checkout-cancel-return` redirecting to Billing with `Checkout was cancelled` plus `No charge was made` copy. | It is local/accountless rendered return proof; real deployed Stripe/Clerk checkout success/cancel, authenticated Teams, packaged desktop, hook lifecycle, and final judge scoring remain outside this proof. |
| `npx playwright test tests/e2e/five-persona-state-bundles.spec.ts --project=chromium --reporter=list --retries=0`, port 34298 with matched `WAGGLE_E2E_BASE_URL` | Pass, 5 tests | Regenerates the full five-persona bundle after the Team Admin checkout return probes. The Team Admin bundle reports 5 failure/workflow probes, including `billing-checkout-unavailable`, `billing-checkout-success-return`, `billing-checkout-cancel-return`, `backup-create-failure`, and `backup-restore-failure`; the full run reports 0 critical console/page/network failures and 0 visible overflow. | It is still an accountless/no-LLM bundle; real authenticated Teams server, real deployed/provider checkout success/cancel, packaged desktop, hook lifecycle, broader performance states, and final judge scoring remain outside this proof. |
| `npx playwright test tests/e2e/user-journeys.spec.ts --project=chromium --grep "J-route-coverage: thin utility routes render or redirect clearly" --reporter=list --retries=0`, port 34299 with matched `WAGGLE_E2E_BASE_URL` | Pass, 1 test | Confirms the broader thin-route route-smoke still accepts the `/payment-cancelled` redirect into Billing after the new `checkout=cancelled` marker. | Route-smoke only; detailed success/cancel copy is owned by the five-persona Team Admin bundle above. |
| `npx playwright test tests/e2e/five-persona-state-bundles.spec.ts --project=chromium --grep "team-admin-security-reviewer" --reporter=list --retries=0`, port 34300 with matched `WAGGLE_E2E_BASE_URL` | Pass, 1 test | Focused verification for local mocked Teams-state evidence. The Team Admin bundle now captures `billing-team-active-state` with `/api/tier` mocked to `TEAMS` and visible `Waggle Team`, `$49/mo per seat`, and `Manage subscription` copy, plus `team-settings-unlocked-state` with visible `Team Server URL`, `Auth Token`, and trust warning copy. | It is local rendered Teams-tier state proof, not real authenticated Team server membership, deployed checkout, packaged desktop, hook lifecycle, or final judge scoring. |
| `npx playwright test tests/e2e/five-persona-state-bundles.spec.ts --project=chromium --reporter=list --retries=0`, port 34301 with matched `WAGGLE_E2E_BASE_URL` | Pass, 5 tests | Regenerates the full five-persona bundle after adding the Team active billing and Team settings unlocked probes. The Team Admin bundle reports 7 failure/workflow probes, including `billing-team-active-state`, `team-settings-unlocked-state`, checkout unavailable/success/cancel, backup creation failure, and backup restore failure; the full run reports 0 critical console/page/network failures and 0 visible overflow. | It is still an accountless/no-LLM bundle with mocked Teams tier; real authenticated Team server, real deployed/provider checkout success/cancel, packaged desktop, hook lifecycle, broader performance states, and final judge scoring remain outside this proof. |
| `node node_modules/playwright/cli.js test tests/e2e/five-persona-state-bundles.spec.ts --project=chromium --grep team-admin-security-reviewer --reporter=list` | Pass, 1 test | Focused verification for sampled Team Admin approval-grant revocation. The bundle now captures `approvals-revoke-all-grants`: `/approvals` is mocked with one saved grant, the in-app approval modal confirms the destructive revoke-all action, and the final route renders `No saved grants` plus revoked-state toast copy with 0 critical console/page/network failures and 0 visible overflow. | It is local accountless rendered proof for the sampled revoke-all path; broader authenticated approval audit history, focus/keyboard behavior, and less common approval policy states remain outside this proof. |
| `node node_modules/playwright/cli.js test tests/e2e/five-persona-state-bundles.spec.ts --project=chromium --reporter=list` | Pass, 5 tests | Regenerates the full five-persona bundle after adding the Team Admin approval-grant revoke-all probe. The Team Admin bundle now reports 8 failure/workflow probes, including `approvals-revoke-all-grants`, Teams state, checkout unavailable/success/cancel, backup creation failure, and backup restore failure; the full run reports 0 critical console/page/network failures and 0 visible overflow. | It is still an accountless/no-LLM bundle with mocked Teams tier; real authenticated Team server, real deployed/provider checkout success/cancel, packaged desktop, hook lifecycle, broader performance states, and final judge scoring remain outside this proof. |
| Native dialog fixed-string scan for `confirm(`, `alert(`, and `prompt(` | Current high-confidence production scan clean | T7/T12 no longer has known production browser-native dialog calls after the focused fixes. | Remaining broad scan hits are markdown sanitizer test payloads only; T7/T12 still needs persona-state, failure-state, and accessibility evidence. |

Observed warning noise in the focused state slice:

- Node `[DEP0040] punycode` deprecation warnings.
- Expected but noisy `[useWorkspaces] fetch failed: Unauthorized` logs in auth-gate tests.
- Repeated React `act(...)` warnings from provider/test setup.
- React Router v7 future-flag warnings.

Interpretation: the tested state slices are encouraging, but the output is still too noisy for a final judge lane. T10 should reduce or isolate expected warning noise so new state failures stand out.

## Native Dialog Evidence

High-confidence browser-native dialog hits that affect T7/T12:

| Surface | Evidence | Judge risk |
|---|---|---|
| Workspace/template management | `apps/web/src/components/os/overlays/CreateWorkspaceDialog.tsx:877`, `:925` | Solo founder or admin sees a browser confirm inside a branded creation/manage flow. |
| Approvals | Historical: `apps/web/src/components/os/apps/ApprovalsApp.tsx:158`. Post-fix: shared in-app `ApprovalModal` plus focused component/rendered evidence. | Fixed for the sampled Team-admin revoke-all path; broader approval states still need persona-bundle evidence. |
| Backup app | Historical: `apps/web/src/components/os/apps/BackupApp.tsx:79`. Post-fix: shared in-app `ApprovalModal` plus focused component evidence. | Fixed for the standalone restore-selection path; broader backup/persona-bundle evidence remains. |
| Automation Center | Historical: `apps/web/src/components/os/apps/AutomationCenterApp.tsx:184`. Post-fix: shared in-app `ApprovalModal` plus focused component evidence. | Fixed for sampled automation delete path; broader automation persona-bundle evidence remains. |
| Artifact Center | Historical: `apps/web/src/components/os/apps/ArtifactCenterApp.tsx:165`. Post-fix: shared in-app `ApprovalModal` plus focused component/rendered evidence. | Fixed for the sampled Researcher/engineer permanent-delete path; archive and broader artifact states still need persona-bundle evidence. |
| Compliance templates | Historical: `apps/web/src/components/os/apps/cockpit/ComplianceTemplateModal.tsx:155`. Post-fix: shared in-app `ApprovalModal` plus focused component evidence. | Fixed for sampled compliance-template delete path; broader compliance persona-bundle evidence remains. |
| Memory Center | Historical: `apps/web/src/components/os/apps/memory/MemoryCenterTab.tsx:301`, `:311`, `:356`. Post-fix: shared in-app `ApprovalModal` plus focused component/rendered evidence for delete, erase, and allow re-import. | Fixed for sampled Researcher permanent-delete, GDPR erasure, and suppression-lift paths; broader persona-bundle evidence remains. |
| Settings telemetry/backup | Historical: `apps/web/src/components/os/apps/SettingsApp.tsx:399`, `:893`, `:901`, `:911`, `:921`, `:922`, `:925`. Post-fix: shared in-app `ApprovalModal` plus inline backup status evidence. | Fixed for sampled Team-admin telemetry clear, backup failure, and restore-success paths; broader backup/persona-bundle evidence remains. |
| Wiki export | Historical: `apps/web/src/components/os/apps/memory/WikiTab.tsx:188`, `:214`. Post-fix: in-app Obsidian/Notion export form dialogs plus focused component/rendered evidence; the Researcher bundle now includes an Obsidian export-failure state. | Fixed for sampled Researcher export destination paths and one rendered Obsidian export-failure path; broader Notion/permission variants remain. |

Earlier false positives in `MemoryTrustManage.tsx` were local memory-review callbacks and are no longer present in the production scan.

## State Bundle Contract

Each judge run must write a `state-bundle.md` before scoring. A persona cannot score 9/10 from default-state screenshots alone.

| Persona | Required bundle fields | Minimum state evidence |
|---|---|---|
| Solo founder | Accountless local, Solo billing, simple disclosure, no-model recovery plus first working/skipped chat lane, fresh/no-workspace then one workspace, accountless Clerk/CSP error lane, desktop plus mobile Home spot-check | `/auth`, onboarding/skip, `/home`, workspace create/list, first chat, return Home, console summary. |
| Researcher | Declared auth mode, Solo or declared Team, power disclosure, memory-focused model state, populated memory plus no-results, missing-source/export/delete failure state, desktop plus optional mobile Memory | Memory overview, search hit, no-results, provenance/trust detail, wiki/timeline, branded confirmation/result, chat explanation, T19 decision if browser capture is included. |
| Engineer / power user | Accountless local, Solo with Teams-only deferrals, power/admin disclosure, local/no-LLM plus one working-provider lane if chat is scored, workspace plus detected/undetected tools and MCP catalog, marketplace/tool unavailable states, desktop keyboard path | Command Center, `Ctrl+Shift+N`, Launcher, MCP Hub, Files, Events, console summary, T15/T16/T17/T18 decisions if utilities/hooks/developer lanes are in scope. |
| Team admin / security reviewer | Authenticated or declared accountless limitation, Teams plus Solo gating comparison and legacy Pro-to-Solo state, professional/admin disclosure, vault/approval/backup/team data, backup or restore failure copy, desktop plus mobile Settings/Profile spot-check | Billing, active mocked Team billing state, unlocked Team settings state, Vault secret hidden, Approvals, backup/restore, Team governance, payment success, payment cancelled, branded confirmation/result, T13/T14/T15 decisions. |
| Mobile executive | Accountless local, Solo unless Team view sampled, simple disclosure with power comparison only if needed, no-model or verified-model banner fitting mobile, one workspace plus some memory, overlay close/error state, 390 x 844 primary viewport | Mobile Home, Settings general/billing/models/profile, Memory, chat, overlay open/closed, horizontal overflow check, focus/touch notes. |

## Current Strengths

- The focused 79-test state slice passed across auth gate, onboarding, billing, backup, files, connector, MCP, and approvals error cases.
- Current follow-up: `p7-b1-approvals-error.test.tsx` now covers Approvals revoke-all without native `confirm()`, rendered `J3d` passes on port `34159`, and the expanded user-journey suite passed 20/20 on port `34160` before the Artifact `J3e` addition.
- Current follow-up: `artifact-center-trust.test.tsx` now covers Artifact permanent delete without native `confirm()`, rendered `J3e` passes on port `34161`, and the expanded user-journey suite passes 21/21 on port `34162`.
- Current follow-up: `memory-center-trust.test.tsx` now covers Memory Center delete, GDPR erase, and allow re-import without native `confirm()`, rendered `J3f` passes on port `34164`, and the expanded user-journey suite passes 22/22 on port `34165`.
- Current follow-up: `wiki-export-trust.test.tsx` now covers Wiki Obsidian and Notion export destinations without native `prompt()`, rendered `J3g` passes on port `34167`, and the path remains included in the latest 24/24 user-journey suite on port `34173`.
- Current follow-up: `settings-trust.test.tsx` now covers Settings telemetry clear, backup failure, and restore success without native dialogs, rendered `J3h` passes on port `34171`, and the expanded user-journey suite passes 24/24 on port `34173`.
- Current follow-up: `p1b-authgate-surfaces.test.tsx` now covers standalone `BackupApp` restore without native `confirm()`.
- Current follow-up: `phase3b-automation-center.test.tsx` now covers Automation delete without native `confirm()`.
- Current follow-up: `compliance-template-trust.test.tsx` now covers compliance template delete without native `confirm()`.
- Current follow-up: `admin-pages.test.ts` now covers admin-web member removal without native `confirm()`.
- Current follow-up: the five-persona scorecards now require explicit per-persona T12 state bundles and are guarded by `tests/five-persona-state-bundle-contract.test.ts`.
- Current follow-up: `tests/e2e/five-persona-state-bundles.spec.ts` now writes rendered state-bundle evidence and screenshots for all five judge personas under `output/playwright/five-persona-state-bundles/`.
- Current follow-up: `MemoryCenterApp` lets the Memory view tablist wrap on mobile, guarded by `p3-memory-center-app.test.tsx`; the regenerated Mobile Executive bundle reports 0 Memory-route overflow items at 390 x 844.
- Current follow-up: five-persona browser bundles now include sampled failure/workflow/scale/slow-data probes for chat backend offline, Memory API unavailable, Memory slow-list handling, Memory large-list handling, Timeline/Event large-list handling, Launcher sidecar offline, Cockpit health degraded, Agents slow-list handling, Agents large-list handling, Marketplace unavailable, Marketplace large-catalog handling, Files upload failure, Files upload success, Files large-list handling, billing checkout unavailable, Team active billing state, Team settings unlocked state, billing checkout success return, billing checkout cancel return, backup creation failure, backup restore failure, approval grant revoke-all, local model runtime unavailable, and mobile chat backend offline. The same E2E gate now fails on critical console errors, critical network failures, page errors, and visible horizontal overflow.
- Current follow-up: `WorkspaceDesktopApp` wraps workspace tabs on mobile, and `ChatApp` wraps the composer agent strip with a shorter mobile model chip; the regenerated Mobile Executive chat failure bundle reports 0 visible overflow at 390 x 844.
- Current follow-up: the five-persona bundle now records explicit overlay evidence and screenshots. Mobile Executive captures Notification Inbox and Command Center; `NotificationInbox` no longer uses horizontal entrance motion that can transiently overflow the 390 px viewport.
- Chat stream failure-injection specs exist for network drop and retry behavior.
- The existing state matrix correctly separates billing tier from UI disclosure tier, which is essential for Solo-vs-Team scoring.
- The judge runbook already requires state-bundle files, console capture, route evidence, screenshots, and deferral records.

## Current Gaps

- Scorecards now require declared state bundles and current rendered evidence folders tie route, account mode, billing tier, disclosure tier, model state, data state, offline/error state, viewport, screenshots, and console status into one five-persona no-LLM/accountless bundle.
- Component/unit state tests are not enough to prove shell routing, viewport fit, focus behavior, console health, or user-visible recovery copy.
- Current high-confidence production native-dialog scan is clean after the focused fixes; T7/T12 now needs broader persona-state evidence, keyboard/screen-reader validation, and failure-state proof rather than more known browser-native dialog replacement.
- Offline/API failure and sampled scale coverage is broader but still incomplete. Chat backend offline, Memory API unavailable, Memory slow-list handling, Memory large-list handling, Timeline/Event large-list handling, Launcher sidecar offline, Cockpit health degraded, Agents slow-list handling, Agents large-list handling, Marketplace unavailable, Marketplace large-catalog handling, Files upload failure, Files upload success, Files large-list handling, billing checkout unavailable, Team active billing state, Team settings unlocked state, billing checkout success return, billing checkout cancel return, backup creation failure, backup restore failure, approval grant revoke-all, local model runtime unavailable, and mobile chat backend offline now have bundled browser evidence; real deployed/provider checkout success/cancel, real authenticated Teams server, packaged desktop/launch, hook lifecycle, broader slow-data/performance states, and deeper performance evidence still need user-visible evidence or approved deferral.
- Test output is noisy enough that a final judge lane could hide new regressions.
- Mobile state evidence is now concrete for Home, Profile, Settings, Memory, mobile chat backend-offline recovery, Notification Inbox, and Command Center at 390 x 844, with Memory, workspace tabs, chat agent-strip, notification animation, and Command Center overflow/close evidence fixed; authenticated mobile Team paths still need targeted evidence.
- Scale/performance state evidence now has five rendered large-list slices, Files, Marketplace, Memory, Agents, and Timeline/Events, plus two sampled slow-data slices for delayed Memory and Agents loading. Broader slow-data states and deeper app-payload/performance gates remain weak.

## Correction Candidates

| ID | Correction | Phase recommendation | Closure evidence |
|---|---|---|---|
| T12-A | Add a required state-bundle row to every judge scorecard and evidence folder. | Scorecard contract guarded; browser evidence captured for the default accountless/no-LLM lane. | Each scorecard now declares account, billing, disclosure, model, data, offline/error, viewport, and non-main gate decisions, guarded by `tests/five-persona-state-bundle-contract.test.ts`; `output/playwright/five-persona-state-bundles/*/state-bundle.md` now exists for all five personas. Remaining closure requires real authenticated Teams server/failure-state bundles or explicit deferrals. |
| T12-B | Add a lightweight browser state-bundle smoke after Phase 1 for the five personas. | Codified and passing for default accountless/no-LLM lane plus sampled failure and overlay probes. | `tests/e2e/five-persona-state-bundles.spec.ts` captures route sequence, screenshot paths, console summary, visible element-bounds results, overlay evidence, failure probes, and state fields for all five personas. Remaining closure requires real authenticated Teams server, packaged desktop/launch, and broader destructive/failure evidence or approved deferrals. |
| T12-C | Expand failure evidence beyond chat SSE to sidecar/offline, Stripe cancel/unavailable, marketplace unavailable, model unavailable, and backup/restore failure states. | Partially fixed for sampled persona-critical failures. | Current browser bundle proves chat backend offline, Memory API unavailable, Memory slow-list handling, Memory large-list handling, Wiki export failure, Timeline/Event large-list handling, Launcher sidecar offline, Cockpit health degraded, Agents slow-list handling, Agents large-list handling, Marketplace unavailable, Marketplace large-catalog handling, Files upload failure, Files upload success, Files large-list handling, billing checkout unavailable, Team active billing state, Team settings unlocked state, billing checkout success return, billing checkout cancel return, backup creation failure, backup restore failure, approval grant revoke-all, local model runtime unavailable, and mobile chat backend offline with branded recovery copy, successful workflow proof, sampled slow-data proof, sampled Teams-tier proof, sampled scale proof, or sampled destructive-action proof, 0 critical console/page/network failures, and 0 visible overflow. Remaining closure: real deployed/provider checkout success/cancel, real authenticated Teams server, packaged desktop/launch, hook lifecycle, broader slow-data/performance states, deeper performance evidence, and approved deferrals where applicable. |
| T12-D | Replace or explicitly defer native dialogs on persona-critical destructive flows. | Phase 2, shared with T7 | Current high-confidence production native-dialog scan is clean; Approvals revoke-all now has component, rendered user-journey, and five-persona Team Admin bundle evidence; Artifact permanent delete, Memory Center delete/erase/re-import, Wiki export destinations, Settings telemetry/backup/restore, standalone BackupApp restore, Automation delete, compliance template delete, admin member removal, and Create Workspace template delete now have in-app confirmation evidence. Remaining closure needs focus/keyboard behavior evidence, authenticated/audit-history proof, and less common destructive paths. |
| T12-E | Reduce expected warning noise in focused state tests. | Phase 2, shared with T10 | The state slice output is short enough that real failures are visible; known unavoidable warnings are isolated or documented. |
| T12-F | Attach non-main gate decisions to persona bundles. | Phase 2/launch | T13/T14/T15/T16/T17/T18/T19 are evidenced or explicitly deferred per affected persona. |
| T12-G | Add scale/performance state evidence for large lists and slow/failed data. | Partially fixed for sampled Files, Marketplace, Memory, Agents, Timeline/Event large-list states, plus slow Memory and Agents states. | `files-large-list` now renders 240 mocked files in the Engineer bundle, proves `bulk-file-000.md` plus `240 items`, and reports 0 critical console/page/network failures and 0 visible overflow. `marketplace-large-catalog` renders a 240-entry mocked Marketplace catalog, proves `Bulk Skill 000` plus grouped `Skills 120`, `Connectors 60`, and `MCPs 60`, and reports 0 critical console/page/network failures and 0 visible overflow. `memory-slow-list` delays Memory loading by 2 seconds, proves the aria-busy `Loading memories` live status before rendering `40 memories` plus `Bulk Memory 000`, and reports 0 critical console/page/network failures and 0 visible overflow. `memory-large-list` renders a 200-entry mocked Memory Center list in the Researcher bundle, proves `200 memories` plus `Bulk Memory 000`, and reports 0 critical console/page/network failures and 0 visible overflow. `agents-slow-list` delays Agents loading by 2 seconds, proves the aria-busy `Loading agents` live status before rendering `40 agents` plus `Bulk Agent 000`, and reports 0 critical console/page/network failures and 0 visible overflow. `agents-large-list` renders a 180-agent mocked roster in the Engineer bundle, proves `180 agents` plus `Bulk Agent 000`, and reports 0 critical console/page/network failures and 0 visible overflow. `timeline-large-events` renders 360 mocked Timeline events in the Researcher bundle, proves `360 events` plus `Used bulk_tool_000`, and reports 0 critical console/page/network failures and 0 visible overflow. Remaining closure needs broader slow-data/performance evidence or approved deferrals. |

## Approval Recommendation

Keep T12 out of Phase 1 implementation except for the state fields needed when Phase 1 adds T11 route evidence. After Phase 1, run a judge dry run only as advisory unless T12-A through the persona-critical portions of T12-F have evidence or approved deferrals.