This commit is contained in:
@@ -0,0 +1,220 @@
|
||||
# Phase 1 — Network Exposure & Auth Boundary — Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Close the critical LAN auth-bypass (R1-001) and its surrounding network-exposure cluster (R2-003, R2-006, R2-004, R6-005) on the shippable sidecar, then restore the two non-functional quality gates (lint, tauri-tsc).
|
||||
|
||||
**Architecture:** Defense-in-depth on the HTTP boundary: (1) bind loopback by default, opt into `0.0.0.0` only via `WAGGLE_HOST` (already the env the deploys *should* set); (2) stop the unauthenticated `/health` from leaking the bearer token; (3) exact-match CORS; (4) a shared same-origin guard reused across the sensitive local-only endpoints; (5) Host-header allowlist to defeat DNS-rebinding against the localhost-trust exemption.
|
||||
|
||||
**Tech Stack:** Fastify 5 sidecar (`packages/server`), Vitest, React adapter (`apps/web/src/lib/adapter.ts`), Docker/Render deploy configs.
|
||||
|
||||
**Verify gate (run after every task):**
|
||||
- `npx tsc --noEmit -p packages/server/tsconfig.json` → 0 errors
|
||||
- `npx vitest run packages/server/tests/local/network-auth.test.ts packages/server/tests/local/security-middleware.test.ts` → green
|
||||
- Adapter task also: `cd apps/web && npx tsc --noEmit` (or root `npm run build`)
|
||||
|
||||
---
|
||||
|
||||
## File Map
|
||||
|
||||
| File | Action | Responsibility |
|
||||
|---|---|---|
|
||||
| `packages/server/src/local/origin-guard.ts` | **Create** | `isLocalOrigin()` + `isLocalRequest()` — shared same-origin gate (URL-parse, no prefix bypass) |
|
||||
| `packages/server/src/local/net-config.ts` | **Create** | `resolveBindHost(env)` — loopback default, `WAGGLE_HOST` override |
|
||||
| `packages/server/src/local/routes/vault.ts` | Modify ~169-186 | Use shared `isLocalRequest` (behavior-preserving) |
|
||||
| `packages/server/src/local/service.ts` | Modify :202 | Bind via `resolveBindHost` |
|
||||
| `packages/server/src/local/index.ts` | Modify :278, :1904, :1920-1937, :2329 | bind default, CORS exact-match, gate `/api/debug/logs` + drop providerKeys, drop `wsToken` from `/health` |
|
||||
| `packages/server/src/local/security-middleware.ts` | Modify ~236-294 | Host-header allowlist (R2-004) |
|
||||
| `packages/server/src/local/routes/browse.ts` | Modify :20-94 | Gate `/api/browse/*` to local request (R6-005) |
|
||||
| `apps/web/src/lib/adapter.ts` | Modify :106-117 | Stop reading `wsToken` from `/health`; `connect()` returns health |
|
||||
| `apps/web/src/lib/adapter.ts` (SystemHealth type) | Modify | Drop `wsToken` field |
|
||||
| `Dockerfile`, `render.yaml`, `docker-compose.production.yml` | Modify | `WAGGLE_HOST=0.0.0.0` (preserve cloud reachability) |
|
||||
| `eslint.config.js` (root) | **Create** | Restore functional repo lint gate |
|
||||
| `app/tsconfig.json` | Modify | Point tauri-tsc gate at real TS / stop failing on empty `src/` |
|
||||
| `packages/server/tests/local/network-auth.test.ts` | **Create** | All Phase-1 regression tests |
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Shared same-origin guard (`origin-guard.ts`)
|
||||
|
||||
**Files:** Create `packages/server/src/local/origin-guard.ts`; Test `packages/server/tests/local/network-auth.test.ts`
|
||||
|
||||
- [ ] **Step 1 — failing test** (in `network-auth.test.ts`):
|
||||
```ts
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import { isLocalOrigin } from '../../src/local/origin-guard.js';
|
||||
|
||||
describe('isLocalOrigin', () => {
|
||||
it('allows local + tauri origins', () => {
|
||||
expect(isLocalOrigin('http://127.0.0.1:1420')).toBe(true);
|
||||
expect(isLocalOrigin('http://localhost:3333')).toBe(true);
|
||||
expect(isLocalOrigin('tauri://localhost')).toBe(true);
|
||||
expect(isLocalOrigin('https://tauri.localhost')).toBe(true);
|
||||
});
|
||||
it('rejects external + prefix-bypass origins', () => {
|
||||
expect(isLocalOrigin('https://evil.example.com')).toBe(false);
|
||||
expect(isLocalOrigin('http://localhost.evil.com')).toBe(false);
|
||||
expect(isLocalOrigin('not-a-url')).toBe(false);
|
||||
});
|
||||
});
|
||||
```
|
||||
- [ ] **Step 2 — run, expect fail** (module missing): `npx vitest run packages/server/tests/local/network-auth.test.ts`
|
||||
- [ ] **Step 3 — implement** `origin-guard.ts`:
|
||||
```ts
|
||||
import type { FastifyRequest } from 'fastify';
|
||||
|
||||
const LOCAL_HOSTS = new Set(['127.0.0.1', 'localhost', '::1']);
|
||||
|
||||
/** True if the Origin/Referer string denotes the local Waggle app.
|
||||
* URL-parsed to prevent prefix-bypass (http://localhost.evil.com). */
|
||||
export function isLocalOrigin(raw: string): boolean {
|
||||
try {
|
||||
const u = new URL(raw);
|
||||
if (u.protocol === 'tauri:') return true;
|
||||
if (u.protocol === 'https:' && u.hostname === 'tauri.localhost') return true;
|
||||
if ((u.protocol === 'http:' || u.protocol === 'https:') && LOCAL_HOSTS.has(u.hostname)) return true;
|
||||
return false;
|
||||
} catch { return false; }
|
||||
}
|
||||
|
||||
/** Same-origin gate for sensitive local-only endpoints. A request with no
|
||||
* origin/referer is treated as local (same-host curl / server inject); the
|
||||
* 127.0.0.1 bind is the primary control, this is defense in depth. */
|
||||
export function isLocalRequest(request: FastifyRequest): boolean {
|
||||
const origin = request.headers.origin;
|
||||
if (origin) return isLocalOrigin(origin);
|
||||
const referer = request.headers.referer;
|
||||
if (referer) return isLocalOrigin(referer);
|
||||
return true;
|
||||
}
|
||||
```
|
||||
- [ ] **Step 4 — run, expect pass**
|
||||
- [ ] **Step 5 — refactor `vault.ts`** to use the shared helper (replace the inline `isLocalOrigin` closure at lines 169-186 with an import + `if (!isLocalRequest(request)) return reply.code(403).send({ error: 'Forbidden: external origin not allowed for vault reveal' });`). Keep the 403 message. Run `npx vitest run packages/server/tests/local/security-middleware.test.ts` (vault origin test) → still green.
|
||||
- [ ] **Step 6 — commit**: `fix(server): extract shared same-origin guard (origin-guard.ts), reuse in vault [R2-006/R6-005 prep]`
|
||||
|
||||
## Task 2: Bind loopback by default (R1-001a)
|
||||
|
||||
**Files:** Create `net-config.ts`; Modify `service.ts:202`, `index.ts:278`, `Dockerfile`, `render.yaml`, `docker-compose.production.yml`
|
||||
|
||||
- [ ] **Step 1 — failing test** (`network-auth.test.ts`):
|
||||
```ts
|
||||
import { resolveBindHost } from '../../src/local/net-config.js';
|
||||
describe('resolveBindHost', () => {
|
||||
it('defaults to loopback', () => expect(resolveBindHost({})).toBe('127.0.0.1'));
|
||||
it('honors WAGGLE_HOST', () => expect(resolveBindHost({ WAGGLE_HOST: '0.0.0.0' })).toBe('0.0.0.0'));
|
||||
it('ignores blank WAGGLE_HOST', () => expect(resolveBindHost({ WAGGLE_HOST: ' ' })).toBe('127.0.0.1'));
|
||||
});
|
||||
```
|
||||
- [ ] **Step 2 — run, expect fail**
|
||||
- [ ] **Step 3 — implement** `net-config.ts`:
|
||||
```ts
|
||||
/** Host the sidecar binds to. Loopback by default (desktop product is
|
||||
* localhost-only); deploys that must accept external traffic set WAGGLE_HOST. */
|
||||
export function resolveBindHost(env: NodeJS.ProcessEnv = process.env): string {
|
||||
const h = env.WAGGLE_HOST?.trim();
|
||||
return h && h.length > 0 ? h : '127.0.0.1';
|
||||
}
|
||||
```
|
||||
- [ ] **Step 4 — wire it:** `service.ts:202` → `await server.listen({ port, host: resolveBindHost() });` (import it). `index.ts:278` → `host: resolveBindHost(),`.
|
||||
- [ ] **Step 5 — preserve cloud reachability** (CRITICAL — these deploys are public and currently rely on the old default):
|
||||
- `Dockerfile` (after line 99 `ENV WAGGLE_DATA_DIR=/data`): add `ENV WAGGLE_HOST=0.0.0.0`
|
||||
- `render.yaml` envVars: add `- key: WAGGLE_HOST` / `value: 0.0.0.0`
|
||||
- `docker-compose.production.yml`: add `WAGGLE_HOST=0.0.0.0` to the server service `environment:` block (read file first to place correctly)
|
||||
- [ ] **Step 6 — run** `resolveBindHost` test + `npx tsc --noEmit -p packages/server/tsconfig.json` → green
|
||||
- [ ] **Step 7 — commit**: `fix(server): bind 127.0.0.1 by default; deploys opt into 0.0.0.0 via WAGGLE_HOST [R1-001a]`
|
||||
|
||||
## Task 3: Stop `/health` leaking the bearer token (R1-001b)
|
||||
|
||||
**Files:** `index.ts:2329`, `apps/web/src/lib/adapter.ts`
|
||||
|
||||
- [ ] **Step 1 — failing test** (`network-auth.test.ts`) — build a minimal server exposing the real `/health` shape is heavy; instead assert the contract at the route via a focused integration that mounts security middleware + a `/health` that must NOT echo the token. Simplest durable test: assert the `/health` handler object built in `index.ts` has no `wsToken`. Pragmatic approach — add a route-level test using `buildLocalServer` is too heavy; use a contract test on the adapter instead (Step 4). For the server side, the regression guard is: grep-proof + the existing `security-middleware.test.ts` line 400 expectation must be updated. Update that test's `/health` stub to NOT include wsToken and assert `res.json().wsToken` is `undefined`.
|
||||
- [ ] **Step 2 — server fix:** remove `wsToken: server.agentState.wsSessionToken,` from the `/health` return object (`index.ts:2329`). Leave the rest of the health payload intact.
|
||||
- [ ] **Step 3 — adapter fix** (`apps/web/src/lib/adapter.ts`):
|
||||
- `connect()` (106-117): drop `this.authToken = data.wsToken;`. Change signature to `async connect(): Promise<SystemHealth>` and `return data;`.
|
||||
- Find the `SystemHealth` type def (`grep -rn "wsToken\|SystemHealth" apps/web/src`) and remove the `wsToken` field.
|
||||
- `authToken` stays a settable field (kept for explicit out-of-band token config), just no longer auto-harvested from `/health`. `fetch()` already guards `if (this.authToken)`, so an unset token simply sends no header — correct for the localhost-trusted desktop.
|
||||
- [ ] **Step 4 — verify:** `cd apps/web && npx tsc --noEmit` → 0 errors (proves no caller depended on the removed field). `npx vitest run packages/server/tests/local/security-middleware.test.ts` → green.
|
||||
- [ ] **Step 5 — commit**: `fix(server,web): drop wsToken from unauthenticated /health; adapter no longer harvests it [R1-001b]`
|
||||
|
||||
## Task 4: Exact-match CORS (R2-003)
|
||||
|
||||
**Files:** `index.ts:1904`
|
||||
|
||||
- [ ] **Step 1 — failing test:** unit-test the origin predicate. Extract the CORS check is overkill; instead assert via the existing `ALLOWED_ORIGINS` + `.includes`. Add test importing `ALLOWED_ORIGINS` from `cors-config.js` and asserting a `corsOriginAllowed(origin)` helper. Create a tiny exported helper in `cors-config.ts`:
|
||||
```ts
|
||||
export function corsOriginAllowed(origin: string | undefined): boolean {
|
||||
return !origin || ALLOWED_ORIGINS.includes(origin);
|
||||
}
|
||||
```
|
||||
Test: `corsOriginAllowed('http://localhost:1420')===true`; `corsOriginAllowed('http://localhost:1420.evil.com')===false`; `corsOriginAllowed(undefined)===true`.
|
||||
- [ ] **Step 2 — run, expect fail**
|
||||
- [ ] **Step 3 — implement** the helper, then change `index.ts:1904` CORS callback to:
|
||||
```ts
|
||||
origin: (origin, cb) => {
|
||||
if (corsOriginAllowed(origin)) cb(null, true);
|
||||
else cb(new Error('CORS: origin not allowed'), false);
|
||||
},
|
||||
```
|
||||
- [ ] **Step 4 — run, expect pass** + `tsc -p packages/server`
|
||||
- [ ] **Step 5 — commit**: `fix(server): CORS exact-origin match, no startsWith prefix bypass [R2-003]`
|
||||
|
||||
## Task 5: Gate `/api/debug/logs` + drop key names (R2-006)
|
||||
|
||||
**Files:** `index.ts:1920-1937`
|
||||
|
||||
- [ ] **Step 1 — fix:** at the top of the `/api/debug/logs` handler add:
|
||||
```ts
|
||||
if (!isLocalRequest(_request)) return reply.code(403).send({ error: 'Forbidden: external origin' });
|
||||
```
|
||||
(rename `_request` → `request` since it is now used; import `isLocalRequest`). Remove the `payload.providerKeys = ...` block entirely (vault key names are recon material; the health + audit rows are enough for support).
|
||||
- [ ] **Step 2 — test** (`network-auth.test.ts`): mount a Fastify server with the route + assert external origin → 403, and that the payload has no `providerKeys`. (Build a minimal server registering just this handler, or reuse `buildLocalServer` if cheap; prefer a focused mini-server.)
|
||||
- [ ] **Step 3 — run green** + `tsc`
|
||||
- [ ] **Step 4 — commit**: `fix(server): same-origin gate /api/debug/logs; drop vault key names [R2-006]`
|
||||
|
||||
## Task 6: Gate `/api/browse/*` to local (R6-005)
|
||||
|
||||
**Files:** `routes/browse.ts:20-94`
|
||||
|
||||
- [ ] **Step 1 — fix:** at the start of BOTH `/api/browse/local` (GET) and `/api/browse/local/mkdir` (POST) handlers add:
|
||||
```ts
|
||||
if (!isLocalRequest(request)) return reply.status(403).send({ error: 'Forbidden: external origin' });
|
||||
```
|
||||
(import `isLocalRequest`). Filesystem browse legitimately needs host access (workspace path picker) — confine by *origin*, not by path.
|
||||
- [ ] **Step 2 — test:** mini-server with `browseRoutes`; external-origin GET + mkdir → 403; no-origin GET → 200.
|
||||
- [ ] **Step 3 — run green** + `tsc`
|
||||
- [ ] **Step 4 — commit**: `fix(server): same-origin gate /api/browse/* [R6-005]`
|
||||
|
||||
## Task 7: Host-header allowlist (R2-004)
|
||||
|
||||
**Files:** `security-middleware.ts:236-294`
|
||||
|
||||
- [ ] **Step 1 — fix:** in the `onRequest` hook, before the localhost-trust exemption, reject requests whose `Host` header is neither a loopback host nor an allowlisted name (defeats DNS-rebinding that would otherwise satisfy the IP-based localhost trust). Add a `HOST_ALLOWLIST` (`127.0.0.1`, `localhost`, `::1`, plus `WAGGLE_ALLOWED_HOSTS` comma-env for cloud). Skip the check entirely when `WAGGLE_HOST` is explicitly set to a non-loopback (cloud deploy behind its own proxy) to avoid breaking Render's host header — gate it: only enforce when bound to loopback.
|
||||
```ts
|
||||
// after requestPath is computed
|
||||
const hostHeader = (request.headers.host ?? '').split(':')[0];
|
||||
const boundLoopback = resolveBindHost() === '127.0.0.1';
|
||||
if (boundLoopback && hostHeader && !HOST_ALLOWLIST.has(hostHeader)) {
|
||||
return reply.code(403).send({ error: 'Forbidden', code: 'BAD_HOST' });
|
||||
}
|
||||
```
|
||||
- [ ] **Step 2 — test:** server with middleware; `Host: evil.com` → 403; `Host: 127.0.0.1` / `localhost` → pass. Use `server.inject({ headers: { host: 'evil.com' } })`.
|
||||
- [ ] **Step 3 — run green** (and re-run full `security-middleware.test.ts`) + `tsc`
|
||||
- [ ] **Step 4 — commit**: `fix(server): Host-header allowlist when bound loopback (anti DNS-rebind) [R2-004]`
|
||||
|
||||
## Task 8: Repair the dead quality gates (cross-cutting)
|
||||
|
||||
**Files:** Create root `eslint.config.js`; Modify `app/tsconfig.json`
|
||||
|
||||
- [ ] **Step 1 — lint:** create a root `eslint.config.js` (flat) scoped to shippable source (`packages/server/src`, `apps/web/src`) using `typescript-eslint` recommended, with stylistic/noisy rules relaxed so current code passes; bug-catching rules (`no-undef`, `no-empty` with allowEmptyCatch, `no-unused-vars` warn) on. Run `npm run lint`; iterate rule severities until exit 0. (Goal: gate provides signal + is green, not a lint-cleanup campaign.)
|
||||
- [ ] **Step 2 — tauri-tsc:** `app/src/` is empty (CLAUDE.md §2 drift — cockpit UI no longer there; desktop loads `apps/web` dist). Repoint `app/tsconfig.json` `include` at the TS that actually exists in `app/` (`scripts`, `tailwind.config.ts`) OR, if those have their own configs, narrow the gate. Run `npx tsc --noEmit -p app/tsconfig.json` → exit 0.
|
||||
- [ ] **Step 3 — reconcile docs:** fix the CLAUDE.md §2 line claiming `app/src/components/cockpit/` ships UI (it does not). One-line factual correction.
|
||||
- [ ] **Step 4 — commit**: `fix(build): restore lint + tauri-tsc gates; reconcile CLAUDE.md app/src drift`
|
||||
|
||||
---
|
||||
|
||||
## Self-Review
|
||||
|
||||
- **Spec coverage:** R1-001 (Tasks 2+3), R2-003 (Task 4), R2-006 (Task 5), R6-005 (Task 6), R2-004 (Task 7), gate repair (Task 8). ✅ all Phase-1 finding ids covered.
|
||||
- **Cross-cutting safety:** bind flip paired with deploy `WAGGLE_HOST` (Task 2 Step 5); `/health` token removal paired with adapter + type fix (Task 3); Host-allowlist only enforced when loopback-bound so Render is unaffected (Task 7).
|
||||
- **Type consistency:** `isLocalRequest`/`isLocalOrigin` (origin-guard.ts) reused in vault/debug/browse; `resolveBindHost` (net-config.ts) reused in service.ts/index.ts/security-middleware.ts; `corsOriginAllowed`/`ALLOWED_ORIGINS` (cors-config.ts).
|
||||
- **Verdict caveat:** R2-004, R2-006, R6-005 were `unverified` in the audit (plausible, evidence cited). The failing-test-first step for each independently re-confirms the issue before the fix lands.
|
||||
803
docs/superpowers/plans/2026-06-28-ai-os-positioning-audit.md
Normal file
803
docs/superpowers/plans/2026-06-28-ai-os-positioning-audit.md
Normal file
@@ -0,0 +1,803 @@
|
||||
# AI OS Positioning Audit Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Build a report-mode Playwright E2E audit that grades Waggle across five personas for AI OS positioning, addiction/return signals, competitor comparability, and prioritized improvement areas.
|
||||
|
||||
**Architecture:** Add one self-contained Playwright spec that probes the existing app shell and API surfaces, scores deterministic evidence, and writes Markdown plus JSON artifacts through Playwright's per-test output directory. The audit passes when report generation succeeds and records low product scores as findings, not test failures.
|
||||
|
||||
**Tech Stack:** TypeScript, Playwright test runner, Node `fs/promises`, existing `WAGGLE_E2E_BASE_URL`, existing local server from `playwright.config.ts` or `playwright-e2e.config.ts`.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
- Create: `tests/e2e/ai-os-positioning-audit.spec.ts`
|
||||
- Owns persona definitions, endpoint probes, scoring functions, report rendering, artifact writing, and the Playwright test.
|
||||
- No app/product files change.
|
||||
- No committed report artifacts.
|
||||
- Runtime output goes under `testInfo.outputPath('ai-os-positioning-audit.md')` and `testInfo.outputPath('ai-os-positioning-audit.json')`.
|
||||
- Existing docs remain unchanged after this plan:
|
||||
- Spec source: `docs/superpowers/specs/2026-06-28-ai-os-positioning-audit-design.md`
|
||||
|
||||
---
|
||||
|
||||
### Task 1: RED - Add Audit Contract Spec Shell
|
||||
|
||||
**Files:**
|
||||
- Create: `tests/e2e/ai-os-positioning-audit.spec.ts`
|
||||
|
||||
- [ ] **Step 1: Write the failing report contract test**
|
||||
|
||||
Create `tests/e2e/ai-os-positioning-audit.spec.ts` with this initial content:
|
||||
|
||||
```typescript
|
||||
import { expect, test } from '@playwright/test';
|
||||
|
||||
test.describe('AI OS positioning audit', () => {
|
||||
test('generates a report-mode audit with five personas and improvement areas', async ({ page }, testInfo) => {
|
||||
const audit = await runAiOsPositioningAudit(page, testInfo);
|
||||
|
||||
expect(audit.personas).toHaveLength(5);
|
||||
expect(audit.overall.score).toBeGreaterThanOrEqual(0);
|
||||
expect(audit.overall.score).toBeLessThanOrEqual(100);
|
||||
expect(audit.overall.grade).toMatch(/AI OS|chat|niche|promising|plausible/i);
|
||||
expect(audit.addictionLevel).toMatch(/weak|emerging|strong|very strong/i);
|
||||
expect(audit.improvementAreas.length).toBeGreaterThan(0);
|
||||
expect(audit.artifacts.markdownPath).toMatch(/ai-os-positioning-audit\.md$/);
|
||||
expect(audit.artifacts.jsonPath).toMatch(/ai-os-positioning-audit\.json$/);
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the spec to verify RED**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
node node_modules/playwright/cli.js test tests/e2e/ai-os-positioning-audit.spec.ts --project=chromium --reporter=list
|
||||
```
|
||||
|
||||
Expected: FAIL before implementation, with a TypeScript/runtime error equivalent to `runAiOsPositioningAudit is not defined`.
|
||||
|
||||
- [ ] **Step 3: Commit nothing yet**
|
||||
|
||||
Do not commit the red state. Continue to Task 2.
|
||||
|
||||
---
|
||||
|
||||
### Task 2: GREEN - Add Data Model and Report Renderer
|
||||
|
||||
**Files:**
|
||||
- Modify: `tests/e2e/ai-os-positioning-audit.spec.ts`
|
||||
|
||||
- [ ] **Step 1: Add imports, types, personas, and constants above the test**
|
||||
|
||||
Replace the file contents with the contract test plus these declarations above it:
|
||||
|
||||
```typescript
|
||||
import { expect, type APIRequestContext, type Page, test, type TestInfo } from '@playwright/test';
|
||||
import { mkdir, writeFile } from 'node:fs/promises';
|
||||
import { dirname } from 'node:path';
|
||||
|
||||
const BASE = process.env.WAGGLE_E2E_BASE_URL ?? 'http://127.0.0.1:3333';
|
||||
const SKIP = 'skipOnboarding=true&skipBoot=true&tier=power&skipBriefing=true';
|
||||
|
||||
type DimensionId =
|
||||
| 'onboarding'
|
||||
| 'timeToValue'
|
||||
| 'memory'
|
||||
| 'workflowCoverage'
|
||||
| 'competitiveAdvantage'
|
||||
| 'addiction';
|
||||
|
||||
interface PersonaDefinition {
|
||||
id: string;
|
||||
name: string;
|
||||
role: string;
|
||||
currentDefault: string[];
|
||||
jobToBeDone: string;
|
||||
oneToolCriterion: string;
|
||||
memoryAnchor: string;
|
||||
memoryQuery: string;
|
||||
expectedPersonaIds: string[];
|
||||
expectedSkillTerms: string[];
|
||||
expectedConnectorTerms: string[];
|
||||
externalTriggerNeed: string;
|
||||
internalTriggerNeed: string;
|
||||
competitorBaseline: string;
|
||||
}
|
||||
|
||||
interface DimensionScore {
|
||||
id: DimensionId;
|
||||
label: string;
|
||||
max: number;
|
||||
score: number;
|
||||
evidence: string[];
|
||||
gaps: string[];
|
||||
}
|
||||
|
||||
interface PersonaScore {
|
||||
id: string;
|
||||
name: string;
|
||||
role: string;
|
||||
total: number;
|
||||
grade: string;
|
||||
positioning: string;
|
||||
currentDefault: string[];
|
||||
competitorBaseline: string;
|
||||
oneToolCriterion: string;
|
||||
dimensions: DimensionScore[];
|
||||
improvementAreas: string[];
|
||||
}
|
||||
|
||||
interface ImprovementArea {
|
||||
priority: number;
|
||||
personaId: string;
|
||||
personaName: string;
|
||||
dimension: string;
|
||||
impact: number;
|
||||
recommendation: string;
|
||||
evidence: string[];
|
||||
}
|
||||
|
||||
interface AuditResult {
|
||||
generatedAt: string;
|
||||
overall: {
|
||||
score: number;
|
||||
grade: string;
|
||||
positioningVerdict: string;
|
||||
};
|
||||
addictionLevel: string;
|
||||
personas: PersonaScore[];
|
||||
improvementAreas: ImprovementArea[];
|
||||
artifacts: {
|
||||
markdownPath: string;
|
||||
jsonPath: string;
|
||||
};
|
||||
}
|
||||
|
||||
interface ProbeContext {
|
||||
shellLoaded: boolean;
|
||||
shellText: string;
|
||||
consoleErrors: string[];
|
||||
api: Record<string, { status: number; ok: boolean; ms: number; body: unknown }>;
|
||||
personaIds: string[];
|
||||
personaText: string;
|
||||
skillsText: string;
|
||||
connectorsText: string;
|
||||
marketplaceText: string;
|
||||
memory: Record<string, { saved: boolean; recalled: boolean; isolated: boolean; evidence: string[] }>;
|
||||
}
|
||||
|
||||
const PERSONAS: PersonaDefinition[] = [
|
||||
{
|
||||
id: 'sofia-operator',
|
||||
name: 'Sofia',
|
||||
role: 'Small business operator',
|
||||
currentDefault: ['ChatGPT', 'Gmail', 'Canva'],
|
||||
jobToBeDone: 'Draft customer replies, campaign ideas, and supplier follow-ups.',
|
||||
oneToolCriterion: 'Daily communications and decisions happen in Waggle.',
|
||||
memoryAnchor: 'Sofia runs a neighborhood studio and prefers short, warm customer replies with clear next steps.',
|
||||
memoryQuery: 'short warm customer replies next steps',
|
||||
expectedPersonaIds: ['support-agent', 'marketer', 'executive-assistant'],
|
||||
expectedSkillTerms: ['email', 'marketing', 'document'],
|
||||
expectedConnectorTerms: ['gmail', 'google', 'slack'],
|
||||
externalTriggerNeed: 'daily brief or reply reminder',
|
||||
internalTriggerNeed: 'I need to answer customers without sounding generic',
|
||||
competitorBaseline: 'ChatGPT is fast for drafting but does not own her customer context or operating rhythm.',
|
||||
},
|
||||
{
|
||||
id: 'mara-writer',
|
||||
name: 'Mara',
|
||||
role: 'Marketing writer',
|
||||
currentDefault: ['ChatGPT', 'Claude', 'Notion AI'],
|
||||
jobToBeDone: 'Turn notes and research into branded copy.',
|
||||
oneToolCriterion: 'Voice, drafts, and campaign memory compound in Waggle.',
|
||||
memoryAnchor: 'Mara writes in a crisp, specific brand voice and tracks campaign decisions by launch.',
|
||||
memoryQuery: 'brand voice campaign decisions launch',
|
||||
expectedPersonaIds: ['writer', 'marketer', 'creative-director'],
|
||||
expectedSkillTerms: ['writing', 'brand', 'markdown'],
|
||||
expectedConnectorTerms: ['notion', 'google', 'slack'],
|
||||
externalTriggerNeed: 'weekly wins digest or draft reminder',
|
||||
internalTriggerNeed: 'I need the AI to remember my voice and the campaign angle',
|
||||
competitorBaseline: 'Claude is excellent at prose but does not act as a durable operating workspace.',
|
||||
},
|
||||
{
|
||||
id: 'imran-consultant',
|
||||
name: 'Imran',
|
||||
role: 'Independent consultant',
|
||||
currentDefault: ['Claude', 'ChatGPT', 'Gamma'],
|
||||
jobToBeDone: 'Convert calls and notes into frameworks, briefs, and follow-ups.',
|
||||
oneToolCriterion: 'Client context and recurring strategy work live in Waggle.',
|
||||
memoryAnchor: 'Imran uses 2x2 frameworks and wants every client decision remembered by account.',
|
||||
memoryQuery: '2x2 framework client decision account',
|
||||
expectedPersonaIds: ['consultant', 'researcher', 'analyst'],
|
||||
expectedSkillTerms: ['presentation', 'document', 'research'],
|
||||
expectedConnectorTerms: ['calendar', 'google', 'slack'],
|
||||
externalTriggerNeed: 'client follow-up reminder',
|
||||
internalTriggerNeed: 'What did we decide for this client last time?',
|
||||
competitorBaseline: 'Claude and Gamma help produce artifacts, but the client memory loop is fragmented.',
|
||||
},
|
||||
{
|
||||
id: 'daniel-finance',
|
||||
name: 'Daniel',
|
||||
role: 'Finance and operations analyst',
|
||||
currentDefault: ['Excel Copilot', 'ChatGPT', 'Looker'],
|
||||
jobToBeDone: 'Explain variance, summarize metrics, and prepare board commentary.',
|
||||
oneToolCriterion: 'Data commentary and recurring monthly memory live in Waggle.',
|
||||
memoryAnchor: 'Daniel prepares monthly board commentary and cares about variance drivers, ARR, NPS, and burn.',
|
||||
memoryQuery: 'monthly board commentary variance ARR NPS burn',
|
||||
expectedPersonaIds: ['finance-owner', 'analyst', 'ops-manager'],
|
||||
expectedSkillTerms: ['spreadsheet', 'csv', 'analysis'],
|
||||
expectedConnectorTerms: ['excel', 'google', 'microsoft'],
|
||||
externalTriggerNeed: 'monthly reporting reminder',
|
||||
internalTriggerNeed: 'I need a variance explanation I can defend',
|
||||
competitorBaseline: 'Excel Copilot is close to the data but weak as cross-month memory and agent workspace.',
|
||||
},
|
||||
{
|
||||
id: 'priya-power-user',
|
||||
name: 'Priya',
|
||||
role: 'AI power user',
|
||||
currentDefault: ['Claude Code', 'Codex', 'Hermes', 'OpenClaw'],
|
||||
jobToBeDone: 'Coordinate AI workflows, skills, connectors, and memory.',
|
||||
oneToolCriterion: 'Waggle is the front door for non-coding agent work.',
|
||||
memoryAnchor: 'Priya wants a non-coding AI command center with skills, connectors, memory, and agent coordination.',
|
||||
memoryQuery: 'non coding command center skills connectors agent coordination',
|
||||
expectedPersonaIds: ['coordinator', 'planner', 'verifier', 'coder'],
|
||||
expectedSkillTerms: ['skill', 'automation', 'agent'],
|
||||
expectedConnectorTerms: ['github', 'mcp', 'webhook'],
|
||||
externalTriggerNeed: 'OS hotkey, launcher, or scheduled automation',
|
||||
internalTriggerNeed: 'I need one control surface for all my AI work',
|
||||
competitorBaseline: 'Developer tools are powerful for code but do not give a non-coding AI OS cockpit.',
|
||||
},
|
||||
];
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add report helper functions below the constants**
|
||||
|
||||
Add these helpers:
|
||||
|
||||
```typescript
|
||||
function clampScore(score: number, max: number): number {
|
||||
return Math.max(0, Math.min(max, Math.round(score)));
|
||||
}
|
||||
|
||||
function gradeFor(score: number): string {
|
||||
if (score >= 90) return 'Strong AI OS position';
|
||||
if (score >= 75) return 'Strong niche AI OS fit';
|
||||
if (score >= 60) return 'Promising but still competitor-dependent';
|
||||
if (score >= 40) return 'Plausible positioning, weak product proof';
|
||||
return 'Likely perceived as another AI chat/tool wrapper';
|
||||
}
|
||||
|
||||
function addictionLevelFor(score: number): string {
|
||||
if (score >= 85) return 'very strong';
|
||||
if (score >= 70) return 'strong';
|
||||
if (score >= 50) return 'emerging';
|
||||
return 'weak';
|
||||
}
|
||||
|
||||
function textIncludesAny(haystack: string, needles: string[]): boolean {
|
||||
const lower = haystack.toLowerCase();
|
||||
return needles.some((needle) => lower.includes(needle.toLowerCase()));
|
||||
}
|
||||
|
||||
function renderMarkdown(audit: AuditResult): string {
|
||||
const lines: string[] = [];
|
||||
lines.push('# Waggle AI OS Positioning Audit');
|
||||
lines.push('');
|
||||
lines.push(`Generated: ${audit.generatedAt}`);
|
||||
lines.push(`Overall score: ${audit.overall.score}/100`);
|
||||
lines.push(`Grade: ${audit.overall.grade}`);
|
||||
lines.push(`AI OS verdict: ${audit.overall.positioningVerdict}`);
|
||||
lines.push(`Addiction level: ${audit.addictionLevel}`);
|
||||
lines.push('');
|
||||
lines.push('## Persona Scores');
|
||||
lines.push('');
|
||||
lines.push('| Persona | Role | Score | Grade | Current default |');
|
||||
lines.push('|---|---|---:|---|---|');
|
||||
for (const persona of audit.personas) {
|
||||
lines.push(`| ${persona.name} | ${persona.role} | ${persona.total} | ${persona.grade} | ${persona.currentDefault.join(', ')} |`);
|
||||
}
|
||||
lines.push('');
|
||||
lines.push('## Improvement Areas');
|
||||
lines.push('');
|
||||
for (const item of audit.improvementAreas) {
|
||||
lines.push(`${item.priority}. **${item.personaName} - ${item.dimension}** (${item.impact} pts): ${item.recommendation}`);
|
||||
for (const evidence of item.evidence.slice(0, 2)) {
|
||||
lines.push(` - Evidence: ${evidence}`);
|
||||
}
|
||||
}
|
||||
lines.push('');
|
||||
lines.push('## Persona Detail');
|
||||
for (const persona of audit.personas) {
|
||||
lines.push('');
|
||||
lines.push(`### ${persona.name} - ${persona.role}`);
|
||||
lines.push('');
|
||||
lines.push(`Score: ${persona.total}/100`);
|
||||
lines.push(`Positioning: ${persona.positioning}`);
|
||||
lines.push(`One-tool criterion: ${persona.oneToolCriterion}`);
|
||||
lines.push(`Competitor baseline: ${persona.competitorBaseline}`);
|
||||
lines.push('');
|
||||
lines.push('| Dimension | Score | Evidence | Gaps |');
|
||||
lines.push('|---|---:|---|---|');
|
||||
for (const dim of persona.dimensions) {
|
||||
lines.push(`| ${dim.label} | ${dim.score}/${dim.max} | ${dim.evidence.join('<br>')} | ${dim.gaps.join('<br>')} |`);
|
||||
}
|
||||
}
|
||||
lines.push('');
|
||||
return `${lines.join('\n')}\n`;
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run the spec**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
node node_modules/playwright/cli.js test tests/e2e/ai-os-positioning-audit.spec.ts --project=chromium --reporter=list
|
||||
```
|
||||
|
||||
Expected: still FAIL because `runAiOsPositioningAudit` is not implemented. This keeps the red loop honest while type definitions and rendering are in place.
|
||||
|
||||
---
|
||||
|
||||
### Task 3: GREEN - Add Deterministic Probes
|
||||
|
||||
**Files:**
|
||||
- Modify: `tests/e2e/ai-os-positioning-audit.spec.ts`
|
||||
|
||||
- [ ] **Step 1: Add endpoint and browser probe helpers below report helpers**
|
||||
|
||||
Add:
|
||||
|
||||
```typescript
|
||||
async function timedGet(request: APIRequestContext, path: string) {
|
||||
const started = Date.now();
|
||||
const response = await request.get(`${BASE}${path}`).catch(() => null);
|
||||
const ms = Date.now() - started;
|
||||
if (!response) return { status: 0, ok: false, ms, body: null };
|
||||
const body = await response.json().catch(async () => response.text().catch(() => null));
|
||||
return { status: response.status(), ok: response.ok(), ms, body };
|
||||
}
|
||||
|
||||
async function saveMemory(request: APIRequestContext, workspace: string, content: string): Promise<boolean> {
|
||||
const response = await request.post(`${BASE}/api/memory/frames`, {
|
||||
data: { content, workspace, source: 'user_stated', importance: 'normal' },
|
||||
}).catch(() => null);
|
||||
return !!response?.ok();
|
||||
}
|
||||
|
||||
async function searchMemory(request: APIRequestContext, workspace: string, query: string): Promise<{ found: boolean; evidence: string[] }> {
|
||||
const response = await request.get(
|
||||
`${BASE}/api/memory/search?q=${encodeURIComponent(query)}&workspace=${encodeURIComponent(workspace)}&limit=5`,
|
||||
).catch(() => null);
|
||||
if (!response) return { found: false, evidence: ['Memory search did not return a response.'] };
|
||||
if (!response.ok()) return { found: false, evidence: [`Memory search returned ${response.status()}.`] };
|
||||
const body = await response.json().catch(() => ({}));
|
||||
const raw = JSON.stringify(body);
|
||||
const results = body.results ?? body.recalled ?? [];
|
||||
return {
|
||||
found: Array.isArray(results) ? results.length > 0 || raw.toLowerCase().includes(query.split(' ')[0].toLowerCase()) : raw.length > 20,
|
||||
evidence: [`Memory search returned ${Array.isArray(results) ? results.length : 'unknown'} result(s).`],
|
||||
};
|
||||
}
|
||||
|
||||
async function probeMemory(request: APIRequestContext, persona: PersonaDefinition) {
|
||||
const workspace = `ai-os-audit-${persona.id}-${Date.now()}`;
|
||||
const otherWorkspace = `${workspace}-other`;
|
||||
const saved = await saveMemory(request, workspace, persona.memoryAnchor);
|
||||
await new Promise((resolve) => setTimeout(resolve, 200));
|
||||
const recall = await searchMemory(request, workspace, persona.memoryQuery);
|
||||
const isolation = await searchMemory(request, otherWorkspace, persona.memoryAnchor);
|
||||
return {
|
||||
saved,
|
||||
recalled: saved && recall.found,
|
||||
isolated: !isolation.found,
|
||||
evidence: [
|
||||
saved ? 'Persona memory anchor saved.' : 'Persona memory anchor could not be saved.',
|
||||
...recall.evidence,
|
||||
isolation.found ? 'Potential cross-workspace memory leakage detected.' : 'No cross-workspace recall detected for the persona anchor.',
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
async function collectProbeContext(page: Page): Promise<ProbeContext> {
|
||||
const consoleErrors: string[] = [];
|
||||
page.on('console', (message) => {
|
||||
if (message.type() === 'error') consoleErrors.push(message.text());
|
||||
});
|
||||
|
||||
await page.goto(`${BASE}/home?${SKIP}`, { waitUntil: 'domcontentloaded' });
|
||||
await page.waitForSelector('.waggle-sidebar, [role="navigation"], main', { timeout: 15_000 }).catch(() => {});
|
||||
await page.waitForTimeout(600);
|
||||
const shellText = await page.locator('body').innerText().catch(() => '');
|
||||
const shellLoaded = shellText.length > 80 && !/error boundary|something went wrong/i.test(shellText);
|
||||
|
||||
const endpoints = {
|
||||
health: '/health',
|
||||
personas: '/api/personas',
|
||||
skills: '/api/skills',
|
||||
connectors: '/api/connectors',
|
||||
marketplace: '/api/marketplace/search?query=&limit=10',
|
||||
workspaces: '/api/workspaces',
|
||||
hooks: '/api/hooks',
|
||||
fleet: '/api/fleet',
|
||||
events: '/api/events?limit=3',
|
||||
tier: '/api/tier',
|
||||
};
|
||||
|
||||
const apiEntries = await Promise.all(
|
||||
Object.entries(endpoints).map(async ([key, path]) => [key, await timedGet(page.request, path)] as const),
|
||||
);
|
||||
const api = Object.fromEntries(apiEntries);
|
||||
const personaBody = api.personas?.body as { personas?: Array<{ id?: string; name?: string; description?: string }> } | unknown[];
|
||||
const personaRows = Array.isArray(personaBody) ? personaBody : Array.isArray(personaBody?.personas) ? personaBody.personas : [];
|
||||
const personaIds = personaRows.map((persona) => String(persona.id ?? ''));
|
||||
|
||||
const memory: ProbeContext['memory'] = {};
|
||||
for (const persona of PERSONAS) {
|
||||
memory[persona.id] = await probeMemory(page.request, persona);
|
||||
}
|
||||
|
||||
return {
|
||||
shellLoaded,
|
||||
shellText,
|
||||
consoleErrors,
|
||||
api,
|
||||
personaIds,
|
||||
personaText: JSON.stringify(api.personas?.body ?? ''),
|
||||
skillsText: JSON.stringify(api.skills?.body ?? ''),
|
||||
connectorsText: JSON.stringify(api.connectors?.body ?? ''),
|
||||
marketplaceText: JSON.stringify(api.marketplace?.body ?? ''),
|
||||
memory,
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the spec**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
node node_modules/playwright/cli.js test tests/e2e/ai-os-positioning-audit.spec.ts --project=chromium --reporter=list
|
||||
```
|
||||
|
||||
Expected: still FAIL because `runAiOsPositioningAudit` is not implemented.
|
||||
|
||||
---
|
||||
|
||||
### Task 4: GREEN - Add Scoring and Artifact Writing
|
||||
|
||||
**Files:**
|
||||
- Modify: `tests/e2e/ai-os-positioning-audit.spec.ts`
|
||||
|
||||
- [ ] **Step 1: Add scoring helpers below probe helpers**
|
||||
|
||||
Add:
|
||||
|
||||
```typescript
|
||||
function scorePersona(persona: PersonaDefinition, context: ProbeContext): PersonaScore {
|
||||
const memory = context.memory[persona.id];
|
||||
const fastCoreApis = ['health', 'personas', 'workspaces'].filter((key) => context.api[key]?.ok && context.api[key].ms < 800);
|
||||
const relevantPersona = persona.expectedPersonaIds.some((id) => context.personaIds.includes(id));
|
||||
const relevantSkills = textIncludesAny(`${context.skillsText} ${context.marketplaceText}`, persona.expectedSkillTerms);
|
||||
const relevantConnectors = textIncludesAny(context.connectorsText, persona.expectedConnectorTerms);
|
||||
const osSurfaces = ['hooks', 'fleet', 'events', 'tier'].filter((key) => context.api[key]?.ok || [403, 404].includes(context.api[key]?.status ?? 0));
|
||||
|
||||
const dimensions: DimensionScore[] = [
|
||||
{
|
||||
id: 'onboarding',
|
||||
label: 'Onboarding clarity',
|
||||
max: 15,
|
||||
score: clampScore((context.shellLoaded ? 10 : 0) + (context.consoleErrors.length === 0 ? 3 : 0) + (/Waggle|workspace|AI/i.test(context.shellText) ? 2 : 0), 15),
|
||||
evidence: [
|
||||
context.shellLoaded ? 'App shell loaded meaningful content.' : 'App shell did not load meaningful content.',
|
||||
`${context.consoleErrors.length} console error(s) captured on first load.`,
|
||||
],
|
||||
gaps: [
|
||||
...(context.shellLoaded ? [] : ['Make first-load shell resilient and clearly explain what Waggle is.']),
|
||||
...(/AI OS|operating system/i.test(context.shellText) ? [] : ['AI OS positioning is not explicit in the loaded shell text.']),
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'timeToValue',
|
||||
label: 'Time to first value',
|
||||
max: 15,
|
||||
score: clampScore(fastCoreApis.length * 4 + (context.api.marketplace?.ok ? 3 : 0), 15),
|
||||
evidence: fastCoreApis.map((key) => `${key} responded in ${context.api[key].ms}ms.`),
|
||||
gaps: fastCoreApis.length >= 3 ? [] : ['Core first-value APIs should respond quickly and consistently.'],
|
||||
},
|
||||
{
|
||||
id: 'memory',
|
||||
label: 'Memory and continuity',
|
||||
max: 20,
|
||||
score: clampScore((memory?.saved ? 7 : 0) + (memory?.recalled ? 8 : 0) + (memory?.isolated ? 5 : 0), 20),
|
||||
evidence: memory?.evidence ?? ['Memory probe did not run.'],
|
||||
gaps: [
|
||||
...(memory?.saved ? [] : ['Memory anchor save failed.']),
|
||||
...(memory?.recalled ? [] : ['Saved memory was not confidently recalled.']),
|
||||
...(memory?.isolated ? [] : ['Workspace isolation needs clearer proof.']),
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'workflowCoverage',
|
||||
label: 'Workflow coverage',
|
||||
max: 15,
|
||||
score: clampScore((relevantPersona ? 5 : 0) + (relevantSkills ? 5 : 0) + (relevantConnectors ? 5 : 0), 15),
|
||||
evidence: [
|
||||
relevantPersona ? 'Relevant persona is present.' : `Missing obvious persona match from ${persona.expectedPersonaIds.join(', ')}.`,
|
||||
relevantSkills ? 'Relevant skill or marketplace language found.' : `No clear skill match for ${persona.expectedSkillTerms.join(', ')}.`,
|
||||
relevantConnectors ? 'Relevant connector language found.' : `No clear connector match for ${persona.expectedConnectorTerms.join(', ')}.`,
|
||||
],
|
||||
gaps: [
|
||||
...(relevantPersona ? [] : ['Add or surface a persona that matches this workflow.']),
|
||||
...(relevantSkills ? [] : ['Improve skill/template coverage for this workflow.']),
|
||||
...(relevantConnectors ? [] : ['Improve connector coverage or setup guidance for this workflow.']),
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'competitiveAdvantage',
|
||||
label: 'Competitive advantage',
|
||||
max: 15,
|
||||
score: clampScore((memory?.recalled ? 5 : 0) + (osSurfaces.length >= 3 ? 5 : 0) + (relevantPersona && relevantSkills ? 5 : 0), 15),
|
||||
evidence: [
|
||||
persona.competitorBaseline,
|
||||
`${osSurfaces.length}/4 OS-like surfaces responded or degraded gracefully.`,
|
||||
],
|
||||
gaps: osSurfaces.length >= 3 && memory?.recalled ? [] : ['Make the advantage over the current default more visible and more provable.'],
|
||||
},
|
||||
{
|
||||
id: 'addiction',
|
||||
label: 'Addiction/return signal',
|
||||
max: 20,
|
||||
score: clampScore((memory?.recalled ? 7 : 0) + (context.api.events?.ok ? 3 : 0) + (context.api.hooks?.ok ? 4 : 0) + (context.api.workspaces?.ok ? 3 : 0) + (context.api.tier?.ok ? 3 : 0), 20),
|
||||
evidence: [
|
||||
`External trigger need: ${persona.externalTriggerNeed}.`,
|
||||
`Internal trigger need: ${persona.internalTriggerNeed}.`,
|
||||
],
|
||||
gaps: [
|
||||
...(context.api.hooks?.ok ? [] : ['Durable external trigger surface is weak or not reachable.']),
|
||||
...(memory?.recalled ? [] : ['Stored value is not strong enough to create a return habit.']),
|
||||
],
|
||||
},
|
||||
];
|
||||
|
||||
const total = dimensions.reduce((sum, dimension) => sum + dimension.score, 0);
|
||||
return {
|
||||
id: persona.id,
|
||||
name: persona.name,
|
||||
role: persona.role,
|
||||
total,
|
||||
grade: gradeFor(total),
|
||||
positioning: total >= 75 ? 'Can credibly position Waggle as an AI OS for this persona.' : 'Needs sharper proof before AI OS positioning will feel earned.',
|
||||
currentDefault: persona.currentDefault,
|
||||
competitorBaseline: persona.competitorBaseline,
|
||||
oneToolCriterion: persona.oneToolCriterion,
|
||||
dimensions,
|
||||
improvementAreas: dimensions.flatMap((dimension) => dimension.gaps.map((gap) => `${dimension.label}: ${gap}`)),
|
||||
};
|
||||
}
|
||||
|
||||
function collectImprovementAreas(personas: PersonaScore[]): ImprovementArea[] {
|
||||
const areas: ImprovementArea[] = [];
|
||||
for (const persona of personas) {
|
||||
for (const dimension of persona.dimensions) {
|
||||
const impact = dimension.max - dimension.score;
|
||||
if (impact <= 0) continue;
|
||||
areas.push({
|
||||
priority: 0,
|
||||
personaId: persona.id,
|
||||
personaName: persona.name,
|
||||
dimension: dimension.label,
|
||||
impact,
|
||||
recommendation: dimension.gaps[0] ?? `Improve ${dimension.label.toLowerCase()} for ${persona.name}.`,
|
||||
evidence: dimension.evidence,
|
||||
});
|
||||
}
|
||||
}
|
||||
return areas
|
||||
.sort((a, b) => b.impact - a.impact || a.personaName.localeCompare(b.personaName))
|
||||
.slice(0, 12)
|
||||
.map((area, index) => ({ ...area, priority: index + 1 }));
|
||||
}
|
||||
|
||||
function positioningVerdict(score: number): string {
|
||||
if (score >= 85) return 'Waggle can lead with AI OS positioning now, with persona-specific proof.';
|
||||
if (score >= 70) return 'Waggle has credible AI OS positioning for selected niches, but first-session proof must sharpen.';
|
||||
if (score >= 55) return 'Waggle should position as a memory-native AI workspace before claiming full AI OS broadly.';
|
||||
return 'Waggle should fix core value proof before using AI OS as the main market claim.';
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add the main audit runner below scoring helpers**
|
||||
|
||||
Add:
|
||||
|
||||
```typescript
|
||||
async function runAiOsPositioningAudit(page: Page, testInfo: TestInfo): Promise<AuditResult> {
|
||||
const context = await collectProbeContext(page);
|
||||
const personas = PERSONAS.map((persona) => scorePersona(persona, context));
|
||||
const overallScore = Math.round(personas.reduce((sum, persona) => sum + persona.total, 0) / personas.length);
|
||||
const generatedAt = new Date().toISOString();
|
||||
const markdownPath = testInfo.outputPath('ai-os-positioning-audit.md');
|
||||
const jsonPath = testInfo.outputPath('ai-os-positioning-audit.json');
|
||||
const audit: AuditResult = {
|
||||
generatedAt,
|
||||
overall: {
|
||||
score: overallScore,
|
||||
grade: gradeFor(overallScore),
|
||||
positioningVerdict: positioningVerdict(overallScore),
|
||||
},
|
||||
addictionLevel: addictionLevelFor(Math.round(
|
||||
personas.reduce((sum, persona) => {
|
||||
const addiction = persona.dimensions.find((dimension) => dimension.id === 'addiction');
|
||||
return sum + (addiction ? (addiction.score / addiction.max) * 100 : 0);
|
||||
}, 0) / personas.length,
|
||||
)),
|
||||
personas,
|
||||
improvementAreas: collectImprovementAreas(personas),
|
||||
artifacts: {
|
||||
markdownPath,
|
||||
jsonPath,
|
||||
},
|
||||
};
|
||||
|
||||
const markdown = renderMarkdown(audit);
|
||||
await mkdir(dirname(markdownPath), { recursive: true });
|
||||
await writeFile(markdownPath, markdown, 'utf8');
|
||||
await writeFile(jsonPath, JSON.stringify(audit, null, 2), 'utf8');
|
||||
await testInfo.attach('ai-os-positioning-audit.md', { path: markdownPath, contentType: 'text/markdown' });
|
||||
await testInfo.attach('ai-os-positioning-audit.json', { path: jsonPath, contentType: 'application/json' });
|
||||
return audit;
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run the spec to verify GREEN**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
node node_modules/playwright/cli.js test tests/e2e/ai-os-positioning-audit.spec.ts --project=chromium --reporter=list
|
||||
```
|
||||
|
||||
Expected: PASS, with output showing one passing test and Playwright attachments for Markdown and JSON.
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Refactor and Harden Audit Behavior
|
||||
|
||||
**Files:**
|
||||
- Modify: `tests/e2e/ai-os-positioning-audit.spec.ts`
|
||||
|
||||
- [ ] **Step 1: Add assertions that low scores do not fail the audit**
|
||||
|
||||
In the existing test body, keep the score range assertions but do not assert minimum product score. Ensure the only product-score assertions are:
|
||||
|
||||
```typescript
|
||||
expect(audit.overall.score).toBeGreaterThanOrEqual(0);
|
||||
expect(audit.overall.score).toBeLessThanOrEqual(100);
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add artifact content assertions to the same test**
|
||||
|
||||
Add:
|
||||
|
||||
```typescript
|
||||
const markdown = await testInfo.outputPath('ai-os-positioning-audit.md');
|
||||
expect(markdown).toMatch(/ai-os-positioning-audit\.md$/);
|
||||
for (const persona of ['Sofia', 'Mara', 'Imran', 'Daniel', 'Priya']) {
|
||||
expect(audit.personas.map((p) => p.name)).toContain(persona);
|
||||
}
|
||||
for (const area of audit.improvementAreas) {
|
||||
expect(area.impact).toBeGreaterThan(0);
|
||||
expect(area.recommendation.length).toBeGreaterThan(10);
|
||||
}
|
||||
```
|
||||
|
||||
If TypeScript flags `await` as unnecessary for `testInfo.outputPath`, remove `await` and keep the same assertions.
|
||||
|
||||
- [ ] **Step 3: Run the spec**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
node node_modules/playwright/cli.js test tests/e2e/ai-os-positioning-audit.spec.ts --project=chromium --reporter=list
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Verification Sweep
|
||||
|
||||
**Files:**
|
||||
- Verify: `tests/e2e/ai-os-positioning-audit.spec.ts`
|
||||
|
||||
- [ ] **Step 1: Run focused Playwright audit**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
node node_modules/playwright/cli.js test tests/e2e/ai-os-positioning-audit.spec.ts --project=chromium --reporter=list
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 2: Run TypeScript syntax check via Playwright transpilation**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
node node_modules/playwright/cli.js test tests/e2e/ai-os-positioning-audit.spec.ts --project=chromium --reporter=list --list
|
||||
```
|
||||
|
||||
Expected: test is listed without TypeScript parse errors.
|
||||
|
||||
- [ ] **Step 3: Inspect generated report**
|
||||
|
||||
Open the newest Playwright output attachment or test result directory and verify the Markdown contains:
|
||||
|
||||
```text
|
||||
# Waggle AI OS Positioning Audit
|
||||
Overall score:
|
||||
AI OS verdict:
|
||||
Addiction level:
|
||||
## Persona Scores
|
||||
## Improvement Areas
|
||||
Sofia
|
||||
Mara
|
||||
Imran
|
||||
Daniel
|
||||
Priya
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Check git diff**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
git diff -- tests/e2e/ai-os-positioning-audit.spec.ts
|
||||
git status --short
|
||||
```
|
||||
|
||||
Expected: only `tests/e2e/ai-os-positioning-audit.spec.ts` is modified/added for implementation, plus any existing unrelated untracked files remain untouched.
|
||||
|
||||
---
|
||||
|
||||
### Task 7: Commit Implementation
|
||||
|
||||
**Files:**
|
||||
- Stage: `tests/e2e/ai-os-positioning-audit.spec.ts`
|
||||
|
||||
- [ ] **Step 1: Stage only the audit spec**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
git add -- tests/e2e/ai-os-positioning-audit.spec.ts
|
||||
git status --short
|
||||
```
|
||||
|
||||
Expected: the new audit spec is staged. Existing unrelated untracked files remain unstaged.
|
||||
|
||||
- [ ] **Step 2: Commit**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
git commit -m "test: add ai os positioning audit"
|
||||
```
|
||||
|
||||
Expected: commit succeeds.
|
||||
|
||||
- [ ] **Step 3: Final status**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
git status --short
|
||||
```
|
||||
|
||||
Expected: no staged changes. The pre-existing untracked brief may still appear and must not be touched.
|
||||
361
docs/superpowers/plans/2026-06-30-goal-ancestry.md
Normal file
361
docs/superpowers/plans/2026-06-30-goal-ancestry.md
Normal file
@@ -0,0 +1,361 @@
|
||||
# Goal-Ancestry Context Chain Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Inject a durable "why" breadcrumb (`# Why You're Here` — project + goal) into the agent system prompt each run, complementing recall and the live awareness task state.
|
||||
|
||||
**Architecture:** A `GoalAncestry` type (4 optional levels) + a pure `renderGoalAncestry()` helper + an `Orchestrator.setGoalAncestry()` setter and a cached `# Why You're Here` section in `buildSystemPrompt()`. The chat handler populates `project` from the active workspace name; `goal` lights up wherever an `AgentDef.goal` is in play (agent runs).
|
||||
|
||||
**Tech Stack:** TypeScript, Vitest. No new deps, no UI, no persistence, no feature flag.
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- **Pure renderer.** `Orchestrator`/`renderGoalAncestry` never reach into the DB for ancestry — they render what they're handed.
|
||||
- **No duplication.** The live current task stays in the self-awareness section; goal-ancestry omits `task`.
|
||||
- **Self-suppressing.** Empty/absent ancestry ⇒ renders `''` ⇒ filtered out ⇒ prompt byte-identical to today.
|
||||
- **Truncate** each level to ≤200 chars so a verbose goal can't bloat every turn.
|
||||
- **Order:** the section renders levels mission→project→goal→task, and sits **after identity, before self-awareness** in `buildSystemPrompt()`.
|
||||
- **Gates:** `tsc --noEmit` 0 (shared/agent/server); new units RED→GREEN; existing orchestrator suite green.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: `GoalAncestry` type
|
||||
|
||||
**Files:**
|
||||
- Modify: `packages/shared/src/types.ts`
|
||||
- Test: covered transitively (type-only; exercised by Tasks 2–3).
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: `interface GoalAncestry { mission?: string; project?: string; goal?: string; task?: string }`.
|
||||
|
||||
- [ ] **Step 1: Add the type** — append near the other agent/task types in `packages/shared/src/types.ts`:
|
||||
|
||||
```ts
|
||||
/**
|
||||
* AI-OS #6 — durable "why" injected into the agent system prompt each run
|
||||
* (the purpose above the current turn; complements recall + live awareness).
|
||||
* All levels optional. Today `project` (workspace) and `goal` (agent goal) are
|
||||
* populated; `mission` (no workspace-charter field yet) and `task` (already in
|
||||
* the self-awareness section) are reserved/omitted.
|
||||
*/
|
||||
export interface GoalAncestry {
|
||||
mission?: string;
|
||||
project?: string;
|
||||
goal?: string;
|
||||
task?: string;
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Typecheck**
|
||||
|
||||
Run: `npx tsc --noEmit --project packages/shared/tsconfig.json`
|
||||
Expected: exit 0.
|
||||
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add packages/shared/src/types.ts
|
||||
git commit -m "feat(shared): GoalAncestry type (#6)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: `renderGoalAncestry` pure helper
|
||||
|
||||
**Files:**
|
||||
- Create: `packages/agent/src/goal-ancestry.ts`
|
||||
- Modify: `packages/agent/src/index.ts` (export)
|
||||
- Test: `packages/agent/tests/goal-ancestry.test.ts`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `GoalAncestry` (Task 1) from `@waggle/shared`.
|
||||
- Produces: `renderGoalAncestry(a: GoalAncestry | null | undefined): string`.
|
||||
|
||||
- [ ] **Step 1: Write the failing test** — `packages/agent/tests/goal-ancestry.test.ts`:
|
||||
|
||||
```ts
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import { renderGoalAncestry } from '../src/goal-ancestry.js';
|
||||
|
||||
describe('renderGoalAncestry', () => {
|
||||
it('renders a heading + one line per present level, in mission→project→goal→task order', () => {
|
||||
const out = renderGoalAncestry({ project: 'Acme', goal: 'Ship the pane', mission: 'Win', task: 'x' });
|
||||
expect(out).toBe("# Why You're Here\nMission: Win\nProject: Acme\nGoal: Ship the pane\nTask: x");
|
||||
});
|
||||
|
||||
it('renders only the present levels', () => {
|
||||
expect(renderGoalAncestry({ project: 'Acme' })).toBe("# Why You're Here\nProject: Acme");
|
||||
});
|
||||
|
||||
it('returns empty string for null / undefined / all-empty', () => {
|
||||
expect(renderGoalAncestry(null)).toBe('');
|
||||
expect(renderGoalAncestry(undefined)).toBe('');
|
||||
expect(renderGoalAncestry({})).toBe('');
|
||||
expect(renderGoalAncestry({ goal: '' })).toBe('');
|
||||
});
|
||||
|
||||
it('truncates an over-long level to 200 chars', () => {
|
||||
const long = 'x'.repeat(300);
|
||||
const out = renderGoalAncestry({ goal: long });
|
||||
const line = out.split('\n')[1];
|
||||
expect(line.length).toBeLessThanOrEqual('Goal: '.length + 200);
|
||||
expect(line.endsWith('...')).toBe(true);
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `npx vitest run packages/agent/tests/goal-ancestry.test.ts`
|
||||
Expected: FAIL — module not found.
|
||||
|
||||
- [ ] **Step 3: Implement** `packages/agent/src/goal-ancestry.ts`:
|
||||
|
||||
```ts
|
||||
/**
|
||||
* AI-OS #6 — pure renderer for the goal-ancestry "# Why You're Here" prompt
|
||||
* section. Total (never throws); returns '' when there is no durable "why" so
|
||||
* the section self-suppresses and the prompt stays byte-identical to today.
|
||||
*/
|
||||
import type { GoalAncestry } from '@waggle/shared';
|
||||
|
||||
const MAX = 200;
|
||||
const cap = (s: string): string => (s.length > MAX ? s.slice(0, MAX - 3) + '...' : s);
|
||||
|
||||
export function renderGoalAncestry(a: GoalAncestry | null | undefined): string {
|
||||
if (!a) return '';
|
||||
const lines: string[] = [];
|
||||
if (a.mission) lines.push(`Mission: ${cap(a.mission)}`);
|
||||
if (a.project) lines.push(`Project: ${cap(a.project)}`);
|
||||
if (a.goal) lines.push(`Goal: ${cap(a.goal)}`);
|
||||
if (a.task) lines.push(`Task: ${cap(a.task)}`);
|
||||
if (lines.length === 0) return '';
|
||||
return "# Why You're Here\n" + lines.join('\n');
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Export + run** — add to `packages/agent/src/index.ts`:
|
||||
|
||||
```ts
|
||||
export { renderGoalAncestry } from './goal-ancestry.js';
|
||||
```
|
||||
|
||||
Run: `npx vitest run packages/agent/tests/goal-ancestry.test.ts`
|
||||
Expected: PASS (4 cases).
|
||||
|
||||
- [ ] **Step 5: Typecheck + commit**
|
||||
|
||||
```bash
|
||||
npx tsc --noEmit --project packages/agent/tsconfig.json
|
||||
git add packages/agent/src/goal-ancestry.ts packages/agent/src/index.ts packages/agent/tests/goal-ancestry.test.ts
|
||||
git commit -m "feat(agent): renderGoalAncestry pure helper (#6)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Orchestrator — setter + cached section
|
||||
|
||||
**Files:**
|
||||
- Modify: `packages/agent/src/orchestrator.ts`
|
||||
- Test: `packages/agent/tests/orchestrator-goal-ancestry.test.ts`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `renderGoalAncestry` (Task 2), `GoalAncestry` (Task 1).
|
||||
- Produces: `OrchestratorConfig.goalAncestry?: GoalAncestry`; `Orchestrator.setGoalAncestry(a: GoalAncestry | null): void`; a `# Why You're Here` section in `buildSystemPrompt()`.
|
||||
|
||||
- [ ] **Step 1: Write the failing test** — `packages/agent/tests/orchestrator-goal-ancestry.test.ts` (mirror the existing orchestrator test's MindDB setup — import `MindDB` + a mock embedder from the sibling tests; the minimal harness is `new Orchestrator({ db, embedder })`):
|
||||
|
||||
```ts
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import { MindDB } from '@waggle/core';
|
||||
import { Orchestrator } from '../src/orchestrator.js';
|
||||
|
||||
// Mock embedder — orchestrator construction needs one but buildSystemPrompt doesn't embed.
|
||||
const embedder = { embed: async () => [0], dimensions: 1 } as unknown as ConstructorParameters<typeof Orchestrator>[0]['embedder'];
|
||||
|
||||
function orch() {
|
||||
return new Orchestrator({ db: new MindDB(':memory:'), embedder });
|
||||
}
|
||||
|
||||
describe('buildSystemPrompt goal-ancestry (#6)', () => {
|
||||
it('renders the "Why You\'re Here" section after setGoalAncestry', () => {
|
||||
const o = orch();
|
||||
o.setGoalAncestry({ project: 'Acme Redesign', goal: 'Ship the live-output pane' });
|
||||
const prompt = o.buildSystemPrompt();
|
||||
expect(prompt).toContain("# Why You're Here");
|
||||
expect(prompt).toContain('Project: Acme Redesign');
|
||||
expect(prompt).toContain('Goal: Ship the live-output pane');
|
||||
});
|
||||
|
||||
it('omits the section entirely when no ancestry is set', () => {
|
||||
const prompt = orch().buildSystemPrompt();
|
||||
expect(prompt).not.toContain("# Why You're Here");
|
||||
});
|
||||
|
||||
it('accepts goalAncestry via the constructor config', () => {
|
||||
const o = new Orchestrator({ db: new MindDB(':memory:'), embedder, goalAncestry: { project: 'P' } });
|
||||
expect(o.buildSystemPrompt()).toContain('Project: P');
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `npx vitest run packages/agent/tests/orchestrator-goal-ancestry.test.ts`
|
||||
Expected: FAIL — `setGoalAncestry`/`goalAncestry` not defined.
|
||||
|
||||
- [ ] **Step 3: Implement** in `packages/agent/src/orchestrator.ts`:
|
||||
|
||||
(a) Imports — add the helper + type:
|
||||
|
||||
```ts
|
||||
import { renderGoalAncestry } from './goal-ancestry.js';
|
||||
import type { GoalAncestry } from '@waggle/shared';
|
||||
```
|
||||
|
||||
(b) `OrchestratorConfig` (after `reranker?: Reranker;`, before the closing brace at line ~85):
|
||||
|
||||
```ts
|
||||
/** AI-OS #6 — durable "why" breadcrumb injected into buildSystemPrompt. */
|
||||
goalAncestry?: GoalAncestry;
|
||||
```
|
||||
|
||||
(c) Field + constructor (beside `private skills: string[];` add the field; in the constructor beside `this.skills = config.skills ?? [];` add the assignment):
|
||||
|
||||
```ts
|
||||
private goalAncestry: GoalAncestry | null = null;
|
||||
```
|
||||
```ts
|
||||
this.goalAncestry = config.goalAncestry ?? null;
|
||||
```
|
||||
|
||||
(d) Setter (place near `setWorkspaceMind` — mutable like it):
|
||||
|
||||
```ts
|
||||
/** AI-OS #6 — set/replace the goal-ancestry breadcrumb for the next prompt build. */
|
||||
setGoalAncestry(ancestry: GoalAncestry | null): void {
|
||||
this.goalAncestry = ancestry;
|
||||
}
|
||||
```
|
||||
|
||||
(e) Cached section in `buildSystemPrompt()` — after the `identitySection` block, before the `awarenessSection` block:
|
||||
|
||||
```ts
|
||||
// ── GOAL ANCESTRY (the durable "why"; changes only when re-set) ──
|
||||
const goalAncestrySection = this.cachedSection(
|
||||
'goal_ancestry',
|
||||
JSON.stringify(this.goalAncestry) || 'empty',
|
||||
() => renderGoalAncestry(this.goalAncestry),
|
||||
);
|
||||
```
|
||||
|
||||
(f) Insert into the parts array (was `[identitySection, awarenessSection, contextSection]`):
|
||||
|
||||
```ts
|
||||
const parts = [identitySection, goalAncestrySection, awarenessSection, contextSection].filter(Boolean);
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run test + existing orchestrator suite**
|
||||
|
||||
Run: `npx vitest run packages/agent/tests/orchestrator-goal-ancestry.test.ts && npx vitest run packages/agent/tests/orchestrator.test.ts`
|
||||
Expected: PASS (new 3 + existing orchestrator suite green).
|
||||
|
||||
- [ ] **Step 5: Typecheck + commit**
|
||||
|
||||
```bash
|
||||
npx tsc --noEmit --project packages/agent/tsconfig.json
|
||||
git add packages/agent/src/orchestrator.ts packages/agent/tests/orchestrator-goal-ancestry.test.ts
|
||||
git commit -m "feat(agent): orchestrator goal-ancestry setter + cached section (#6)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Chat wiring — populate `project` from the active workspace
|
||||
|
||||
**Files:**
|
||||
- Modify: `packages/server/src/local/routes/chat.ts` (the local `buildSystemPrompt`, around the `orch.buildSystemPrompt()` call at line ~301)
|
||||
- Test: `packages/server/tests/chat-api.test.ts` (extend) OR a focused assertion that the system prompt carries the workspace name.
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `Orchestrator.setGoalAncestry` (Task 3); the workspace-name resolver (`server.agentState.listWorkspaces?.().find(w => w.id === <wsId>)?.name`, the pattern at chat.ts:1811).
|
||||
|
||||
- [ ] **Step 1: Locate + read** the local `buildSystemPrompt(orch, workspacePath, sessionId, historyLen, effectiveWorkspace, personaOverride, assembled)` function (chat.ts ~252–305). Confirm `effectiveWorkspace` (workspace id) and `server` are in scope.
|
||||
|
||||
- [ ] **Step 2: Write the failing test** — extend `chat-api.test.ts`: drive a chat turn in a named workspace and assert the streamed/assembled system context references the workspace name. (If the test harness doesn't expose the system prompt, add a focused unit around a small extracted `resolveChatAncestry(server, effectiveWorkspace)` helper instead — see Step 3.)
|
||||
|
||||
```ts
|
||||
// Focused unit (preferred — no need to crack open the SSE turn):
|
||||
import { resolveChatAncestry } from '../src/local/routes/chat.js';
|
||||
it('resolves project from the active workspace name', () => {
|
||||
const fakeServer = { agentState: { listWorkspaces: () => [{ id: 'ws1', name: 'Acme Redesign' }] } };
|
||||
expect(resolveChatAncestry(fakeServer as never, 'ws1')).toEqual({ project: 'Acme Redesign' });
|
||||
expect(resolveChatAncestry(fakeServer as never, 'missing')).toEqual({});
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run test to verify it fails**
|
||||
|
||||
Run: `npx vitest run packages/server/tests/chat-api.test.ts -t "project from the active workspace"`
|
||||
Expected: FAIL — `resolveChatAncestry` not exported.
|
||||
|
||||
- [ ] **Step 4: Implement** in `packages/server/src/local/routes/chat.ts`:
|
||||
|
||||
Add a tiny exported helper near the top of the route module (keeps the wiring testable + DRY):
|
||||
|
||||
```ts
|
||||
/** AI-OS #6 — resolve the durable goal-ancestry for a chat turn. `project` is
|
||||
* the active workspace name; `goal` is omitted in chat (personas carry no goal
|
||||
* — it lights up for agent runs that carry an AgentDef.goal). */
|
||||
export function resolveChatAncestry(
|
||||
server: { agentState?: { listWorkspaces?: () => Array<{ id: string; name: string }> } },
|
||||
workspaceId: string | undefined,
|
||||
): import('@waggle/shared').GoalAncestry {
|
||||
const name = workspaceId
|
||||
? server.agentState?.listWorkspaces?.().find((w) => w.id === workspaceId)?.name
|
||||
: undefined;
|
||||
return name ? { project: name } : {};
|
||||
}
|
||||
```
|
||||
|
||||
Then, in the local `buildSystemPrompt`, immediately before `prompt += assembled?.system ?? orch.buildSystemPrompt();`:
|
||||
|
||||
```ts
|
||||
orch.setGoalAncestry(resolveChatAncestry(server, effectiveWorkspace));
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Run test to verify it passes + server suite sanity**
|
||||
|
||||
Run: `npx vitest run packages/server/tests/chat-api.test.ts`
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 6: Typecheck + commit**
|
||||
|
||||
```bash
|
||||
npx tsc --noEmit --project packages/server/tsconfig.json
|
||||
git add packages/server/src/local/routes/chat.ts packages/server/tests/chat-api.test.ts
|
||||
git commit -m "feat(server): chat populates goal-ancestry project from workspace (#6)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Final Gate
|
||||
|
||||
- [ ] **Full typecheck:** `npx tsc --noEmit --project packages/shared/tsconfig.json && npx tsc --noEmit --project packages/agent/tsconfig.json && npx tsc --noEmit --project packages/server/tsconfig.json` → 0.
|
||||
- [ ] **Touched suites:** `npx vitest run packages/agent/tests/goal-ancestry.test.ts packages/agent/tests/orchestrator-goal-ancestry.test.ts packages/agent/tests/orchestrator.test.ts packages/server/tests/chat-api.test.ts`
|
||||
- [ ] **Lint** touched files.
|
||||
|
||||
## Fast-follow (documented, NOT silently dropped)
|
||||
|
||||
**Agent-run `goal` wiring.** The `goal` level is designed to carry `AgentDef.goal`, but chat
|
||||
personas have no goal field, so chat populates only `project`. Lighting up `goal` requires calling
|
||||
`orch.setGoalAncestry({ project, goal: agentDef.goal })` on the **agent-run** path (fleet spawn /
|
||||
`runAgentLoop`), which constructs/uses its own orchestrator. That path is a clean additive follow-up
|
||||
(same setter, same renderer) — it is intentionally out of this S-scoped plan, not overlooked.
|
||||
|
||||
## Self-Review
|
||||
|
||||
**Spec coverage:** §3 Unit A → Task 1 · Unit B → Tasks 2–3 · Unit C → Task 4 · §5 truncation/empty → Task 2 tests · §6 testing → each task's TDD + Final Gate. The §2 `goal`-for-agent-runs mapping is delivered for the *renderer/setter* (Tasks 2–3) and its chat half (Task 4 `project`); the agent-run `goal` populate is explicitly logged as a fast-follow (no silent cap).
|
||||
|
||||
**Placeholder scan:** Task 4 Step 1 is a *read-to-confirm* step (not a code placeholder); every code step ships real code. No TBD/TODO.
|
||||
|
||||
**Type consistency:** `GoalAncestry` (Task 1) is consumed unchanged in Tasks 2–4. `renderGoalAncestry(a)` (Task 2) is called in Task 3. `setGoalAncestry`/`goalAncestry` names match across Tasks 3–4. `resolveChatAncestry(server, workspaceId)` returns `GoalAncestry`, consumed by `setGoalAncestry`.
|
||||
1186
docs/superpowers/plans/2026-06-30-launcher-live-output.md
Normal file
1186
docs/superpowers/plans/2026-06-30-launcher-live-output.md
Normal file
File diff suppressed because it is too large
Load Diff
577
docs/superpowers/plans/2026-06-30-tool-adapter-registry.md
Normal file
577
docs/superpowers/plans/2026-06-30-tool-adapter-registry.md
Normal file
@@ -0,0 +1,577 @@
|
||||
# Pluggable Tool-Adapter Registry Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Collapse the 7 hardcoded external tools (duplicated across 8 structures) into one derived `BUILTIN_TOOL_MANIFESTS` source of truth + an agent registry, and let a self-hosted operator add a PATH-based CLI runtime via `~/.waggle/adapters/*.json` (zod-validated, no `require()`).
|
||||
|
||||
**Architecture:** Declarative `ToolManifest` data lives in `@waggle/shared` (the web bundle + sidecar both read it); the cohort/name/pointer consts derive from it. The agent owns behavior: a `tool-registry.ts` that merges built-in manifests with loaded third-party ones, a `tool-manifest-loader.ts` (declarative, data-only), and a `detectInstalledTools` loop driven by the registry. The 3 desktop candidate-path resolvers stay built-in code (escape hatch); third-party adapters are PATH-only.
|
||||
|
||||
**Tech Stack:** TypeScript, zod, Vitest. No new runtime deps.
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- **`SUPPORTED_TOOLS` stays the `as const` literal anchor** — `ToolId` (the union) is unchanged → zero blast radius on the ~5 `Record<ToolId,…>` consumers.
|
||||
- **Derived consts must equal today's values** for the 7 built-ins (regression-lock).
|
||||
- **Third-party adapters: `detect.kind:'path'` only.** `'candidates'` is built-in-code-only (the escape hatch).
|
||||
- **Loader is data-only:** zod + safe-string refinement (no `; | & $ \` ( )`, no `..`, no path separators in `binaryName`); never `require()`/eval; never throws into detection; missing dir → `[]`.
|
||||
- **Built-in ids win** over any third-party manifest claiming the same id.
|
||||
- **Registry = pure data (no functions);** the candidate resolvers stay in `tool-detection.ts` keyed by id (no `registry → detection` cycle: loader ← registry ← detection).
|
||||
- **Gates:** `tsc` 0 (shared/agent/server/web); existing `tool-detection`/`tool-launcher`/`tools-routes*`/`LauncherApp` suites stay green; new units RED→GREEN.
|
||||
- **Fast-follow (NOT in v1):** applying a third-party `promptArgTemplate` (the prompt-arg path is web-side `apps/web/src/lib/launcher-prompt-args.ts`; the field is captured but unwired).
|
||||
|
||||
## Built-in manifest data (the canonical 7 — used verbatim in Task 1)
|
||||
|
||||
| id | displayName | launchable | hookCapable | hookPointer | detect |
|
||||
|---|---|---|---|---|---|
|
||||
| claude-code | Claude Code | yes | yes | `.claude/hive-mind-install.json` | `path` binary `claude` |
|
||||
| claude-desktop | Claude Desktop | yes | **no** | `.config/Claude/hive-mind-install.json` | `candidates` |
|
||||
| cursor | Cursor | yes | yes | `.cursor/hive-mind-install.json` | `candidates` |
|
||||
| codex | Codex CLI | yes | yes | `.codex/hive-mind-install.json` | `path` binary `codex` |
|
||||
| codex-desktop | Codex Desktop | yes | yes | `.codex/hive-mind-install.json` | `candidates` |
|
||||
| hermes | Hermes Agent | yes | yes | `.hermes/hive-mind-install.json` | `path` binary `hermes` |
|
||||
| openclaw | OpenClaw | yes | yes | `.openclaw/hive-mind-install.json` | `path` binary `openclaw` |
|
||||
|
||||
---
|
||||
|
||||
### Task 1: `shared` — `ToolManifest` + `BUILTIN_TOOL_MANIFESTS` + derived consts
|
||||
|
||||
**Files:**
|
||||
- Modify: `packages/shared/src/tool-detection.ts`
|
||||
- Test: `packages/shared/tests/tool-manifests.test.ts` (new; if `packages/shared/tests` doesn't exist, place under the shared package's test glob — check `packages/shared` for an existing `*.test.ts` to mirror the location)
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: `ToolDetectSpec`, `ToolManifest`, `BUILTIN_TOOL_MANIFESTS: readonly ToolManifest[]`, and **re-derived** `TOOL_DISPLAY_NAMES` / `LAUNCH_COHORT`. `SUPPORTED_TOOLS` / `ToolId` unchanged.
|
||||
|
||||
- [ ] **Step 1: Write the failing test** — `packages/shared/tests/tool-manifests.test.ts`:
|
||||
|
||||
```ts
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import {
|
||||
BUILTIN_TOOL_MANIFESTS, SUPPORTED_TOOLS, LAUNCH_COHORT, TOOL_DISPLAY_NAMES,
|
||||
} from '../src/tool-detection.js';
|
||||
|
||||
describe('BUILTIN_TOOL_MANIFESTS', () => {
|
||||
it('has one manifest per supported tool, ids matching SUPPORTED_TOOLS', () => {
|
||||
expect(BUILTIN_TOOL_MANIFESTS.map((m) => m.id).sort()).toEqual([...SUPPORTED_TOOLS].sort());
|
||||
});
|
||||
it('marks every built-in as builtin:true and launchable', () => {
|
||||
for (const m of BUILTIN_TOOL_MANIFESTS) { expect(m.builtin).toBe(true); expect(m.launchable).toBe(true); }
|
||||
});
|
||||
it('claude-desktop is the only non-hook-capable tool', () => {
|
||||
const nonHook = BUILTIN_TOOL_MANIFESTS.filter((m) => !m.hookCapable).map((m) => m.id);
|
||||
expect(nonHook).toEqual(['claude-desktop']);
|
||||
});
|
||||
it('derives TOOL_DISPLAY_NAMES + LAUNCH_COHORT from the manifests (unchanged values)', () => {
|
||||
expect(TOOL_DISPLAY_NAMES['claude-code']).toBe('Claude Code');
|
||||
expect(TOOL_DISPLAY_NAMES['codex']).toBe('Codex CLI');
|
||||
expect([...LAUNCH_COHORT].sort()).toEqual([...SUPPORTED_TOOLS].sort());
|
||||
});
|
||||
it('claude-code detects by PATH binary "claude" (not its id)', () => {
|
||||
const cc = BUILTIN_TOOL_MANIFESTS.find((m) => m.id === 'claude-code')!;
|
||||
expect(cc.detect).toEqual({ kind: 'path', binaryName: 'claude' });
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `npx vitest run packages/shared/tests/tool-manifests.test.ts`
|
||||
Expected: FAIL — `BUILTIN_TOOL_MANIFESTS` not exported.
|
||||
|
||||
- [ ] **Step 3: Implement** in `packages/shared/src/tool-detection.ts` — keep `SUPPORTED_TOOLS`/`ToolId`/`DetectedTool`/`ToolDetectionResult` exactly as-is. Add the types + manifests, and REPLACE the hand-authored `LAUNCH_COHORT` and `TOOL_DISPLAY_NAMES` with derived versions:
|
||||
|
||||
```ts
|
||||
export type ToolDetectSpec =
|
||||
| { kind: 'path'; binaryName: string }
|
||||
| { kind: 'candidates' };
|
||||
|
||||
export interface ToolManifest {
|
||||
id: string;
|
||||
displayName: string;
|
||||
launchable: boolean;
|
||||
hookCapable: boolean;
|
||||
hookPointer: string;
|
||||
detect: ToolDetectSpec;
|
||||
/** Declarative inline-prompt arg template for THIRD-PARTY path adapters (e.g.
|
||||
* ['--print', '{prompt}']). Built-ins keep their logic in launcher-prompt-args.ts.
|
||||
* Captured in v1; application is a documented fast-follow. */
|
||||
promptArgTemplate?: string[];
|
||||
/** true = first-party; false/absent = loaded third-party. */
|
||||
builtin?: boolean;
|
||||
}
|
||||
|
||||
export const BUILTIN_TOOL_MANIFESTS: readonly ToolManifest[] = [
|
||||
{ id: 'claude-code', displayName: 'Claude Code', launchable: true, hookCapable: true, hookPointer: '.claude/hive-mind-install.json', detect: { kind: 'path', binaryName: 'claude' }, builtin: true },
|
||||
{ id: 'claude-desktop', displayName: 'Claude Desktop', launchable: true, hookCapable: false, hookPointer: '.config/Claude/hive-mind-install.json', detect: { kind: 'candidates' }, builtin: true },
|
||||
{ id: 'cursor', displayName: 'Cursor', launchable: true, hookCapable: true, hookPointer: '.cursor/hive-mind-install.json', detect: { kind: 'candidates' }, builtin: true },
|
||||
{ id: 'codex', displayName: 'Codex CLI', launchable: true, hookCapable: true, hookPointer: '.codex/hive-mind-install.json', detect: { kind: 'path', binaryName: 'codex' }, builtin: true },
|
||||
{ id: 'codex-desktop', displayName: 'Codex Desktop', launchable: true, hookCapable: true, hookPointer: '.codex/hive-mind-install.json', detect: { kind: 'candidates' }, builtin: true },
|
||||
{ id: 'hermes', displayName: 'Hermes Agent', launchable: true, hookCapable: true, hookPointer: '.hermes/hive-mind-install.json', detect: { kind: 'path', binaryName: 'hermes' }, builtin: true },
|
||||
{ id: 'openclaw', displayName: 'OpenClaw', launchable: true, hookCapable: true, hookPointer: '.openclaw/hive-mind-install.json', detect: { kind: 'path', binaryName: 'openclaw' }, builtin: true },
|
||||
] as const;
|
||||
|
||||
// Derived from the manifests (single source of truth). SUPPORTED_TOOLS stays the
|
||||
// type anchor above; these keep their ToolId-typed shapes via the cast.
|
||||
export const TOOL_DISPLAY_NAMES = Object.fromEntries(
|
||||
BUILTIN_TOOL_MANIFESTS.map((m) => [m.id, m.displayName]),
|
||||
) as Record<ToolId, string>;
|
||||
|
||||
export const LAUNCH_COHORT: readonly ToolId[] =
|
||||
BUILTIN_TOOL_MANIFESTS.filter((m) => m.launchable).map((m) => m.id as ToolId);
|
||||
```
|
||||
|
||||
(Delete the old literal `LAUNCH_COHORT` and `TOOL_DISPLAY_NAMES` blocks.)
|
||||
|
||||
- [ ] **Step 4: Run test + typecheck**
|
||||
|
||||
Run: `npx vitest run packages/shared/tests/tool-manifests.test.ts && npx tsc --noEmit -p packages/shared/tsconfig.json`
|
||||
Expected: PASS + tsc 0.
|
||||
|
||||
- [ ] **Step 5: Build shared dist + commit** (downstream tsc reads dist)
|
||||
|
||||
```bash
|
||||
npx tsc -b packages/shared
|
||||
git add packages/shared/src/tool-detection.ts packages/shared/tests/tool-manifests.test.ts
|
||||
git commit -m "feat(shared): BUILTIN_TOOL_MANIFESTS source of truth + derived consts (#5)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: `agent` — declarative manifest loader
|
||||
|
||||
**Files:**
|
||||
- Create: `packages/agent/src/tool-manifest-loader.ts`
|
||||
- Test: `packages/agent/tests/tool-manifest-loader.test.ts`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `ToolManifest` (Task 1).
|
||||
- Produces: `loadThirdPartyManifests(deps?: ManifestLoaderDeps): ToolManifest[]`; `ManifestLoaderDeps { dir?: string; readDir?: (dir: string) => string[]; readFile?: (p: string) => string }`.
|
||||
|
||||
- [ ] **Step 1: Write the failing test** — `packages/agent/tests/tool-manifest-loader.test.ts`:
|
||||
|
||||
```ts
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import { loadThirdPartyManifests } from '../src/tool-manifest-loader.js';
|
||||
|
||||
function deps(files: Record<string, unknown>) {
|
||||
return {
|
||||
dir: '/fake',
|
||||
readDir: () => Object.keys(files),
|
||||
readFile: (p: string) => JSON.stringify(files[p.split('/').pop()!]),
|
||||
};
|
||||
}
|
||||
|
||||
describe('loadThirdPartyManifests', () => {
|
||||
it('loads a valid PATH manifest, stamped builtin:false', () => {
|
||||
const out = loadThirdPartyManifests(deps({
|
||||
'foo.json': { id: 'foo-cli', displayName: 'Foo', launchable: true, hookCapable: false, hookPointer: '.foo/hm.json', detect: { kind: 'path', binaryName: 'foo' } },
|
||||
}));
|
||||
expect(out).toHaveLength(1);
|
||||
expect(out[0]).toMatchObject({ id: 'foo-cli', builtin: false, detect: { kind: 'path', binaryName: 'foo' } });
|
||||
});
|
||||
|
||||
it('rejects detect.kind:candidates (code-only strategy)', () => {
|
||||
const out = loadThirdPartyManifests(deps({
|
||||
'bad.json': { id: 'bad', displayName: 'B', launchable: true, hookCapable: false, hookPointer: '.b/hm.json', detect: { kind: 'candidates' } },
|
||||
}));
|
||||
expect(out).toEqual([]);
|
||||
});
|
||||
|
||||
it('rejects shell-metachar / traversal in fields', () => {
|
||||
const out = loadThirdPartyManifests(deps({
|
||||
'evil.json': { id: 'evil', displayName: 'E', launchable: true, hookCapable: false, hookPointer: '../../etc/passwd', detect: { kind: 'path', binaryName: 'foo; rm -rf /' } },
|
||||
}));
|
||||
expect(out).toEqual([]);
|
||||
});
|
||||
|
||||
it('returns [] when the dir is missing (readDir throws)', () => {
|
||||
expect(loadThirdPartyManifests({ dir: '/none', readDir: () => { throw new Error('ENOENT'); }, readFile: () => '' })).toEqual([]);
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `npx vitest run packages/agent/tests/tool-manifest-loader.test.ts`
|
||||
Expected: FAIL — module not found.
|
||||
|
||||
- [ ] **Step 3: Implement** `packages/agent/src/tool-manifest-loader.ts`:
|
||||
|
||||
```ts
|
||||
/**
|
||||
* AI-OS #5 — declarative loader for third-party tool adapters from
|
||||
* ~/.waggle/adapters/*.json. Data only: zod-validated, safe-string-refined,
|
||||
* PATH-detection only, never require()/eval. Never throws into detection.
|
||||
*/
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
import fs from 'node:fs';
|
||||
import { z } from 'zod';
|
||||
import type { ToolManifest } from '@waggle/shared';
|
||||
|
||||
// No shell metacharacters, no path traversal, non-empty, bounded.
|
||||
const SAFE = /^[A-Za-z0-9._/\\-]+$/;
|
||||
const safe = (max: number) =>
|
||||
z.string().min(1).max(max).refine((s) => SAFE.test(s) && !s.includes('..'), 'unsafe string');
|
||||
|
||||
const ManifestSchema = z.object({
|
||||
id: safe(64),
|
||||
displayName: z.string().min(1).max(80),
|
||||
launchable: z.boolean(),
|
||||
hookCapable: z.boolean(),
|
||||
hookPointer: safe(256),
|
||||
detect: z.object({ kind: z.literal('path'), binaryName: safe(128).refine((s) => !/[\\/]/.test(s), 'binaryName has a path separator') }),
|
||||
promptArgTemplate: z.array(z.string().max(256)).max(20).optional(),
|
||||
});
|
||||
|
||||
export interface ManifestLoaderDeps {
|
||||
dir?: string;
|
||||
readDir?: (dir: string) => string[];
|
||||
readFile?: (p: string) => string;
|
||||
}
|
||||
|
||||
export function loadThirdPartyManifests(deps: ManifestLoaderDeps = {}): ToolManifest[] {
|
||||
const dir = deps.dir ?? path.join(os.homedir(), '.waggle', 'adapters');
|
||||
const readDir = deps.readDir ?? ((d) => fs.readdirSync(d));
|
||||
const readFile = deps.readFile ?? ((p) => fs.readFileSync(p, 'utf8'));
|
||||
let names: string[];
|
||||
try { names = readDir(dir).filter((n) => n.endsWith('.json')); } catch { return []; }
|
||||
const out: ToolManifest[] = [];
|
||||
for (const name of names) {
|
||||
try {
|
||||
const parsed = ManifestSchema.safeParse(JSON.parse(readFile(path.join(dir, name))));
|
||||
if (parsed.success) out.push({ ...parsed.data, builtin: false });
|
||||
} catch { /* skip malformed file */ }
|
||||
}
|
||||
return out;
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `npx vitest run packages/agent/tests/tool-manifest-loader.test.ts`
|
||||
Expected: PASS (4 cases).
|
||||
|
||||
- [ ] **Step 5: Commit** (export wired in Task 3's commit)
|
||||
|
||||
```bash
|
||||
git add packages/agent/src/tool-manifest-loader.ts packages/agent/tests/tool-manifest-loader.test.ts
|
||||
git commit -m "feat(agent): declarative third-party tool-manifest loader (#5)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: `agent` — `getToolRegistry()`
|
||||
|
||||
**Files:**
|
||||
- Create: `packages/agent/src/tool-registry.ts`
|
||||
- Modify: `packages/agent/src/index.ts` (exports)
|
||||
- Test: `packages/agent/tests/tool-registry.test.ts`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `BUILTIN_TOOL_MANIFESTS` (Task 1), `loadThirdPartyManifests` (Task 2).
|
||||
- Produces: `getToolRegistry(deps?: ManifestLoaderDeps): ToolManifest[]` (built-ins + loaded third-party; built-in ids win).
|
||||
|
||||
- [ ] **Step 1: Write the failing test** — `packages/agent/tests/tool-registry.test.ts`:
|
||||
|
||||
```ts
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import { getToolRegistry } from '../src/tool-registry.js';
|
||||
import { SUPPORTED_TOOLS } from '@waggle/shared';
|
||||
|
||||
const fakeLoaderDeps = (manifests: unknown[]) => ({
|
||||
dir: '/fake',
|
||||
readDir: () => manifests.map((_, i) => `m${i}.json`),
|
||||
readFile: (p: string) => JSON.stringify(manifests[Number(p.match(/m(\d+)\.json/)![1])]),
|
||||
});
|
||||
|
||||
describe('getToolRegistry', () => {
|
||||
it('includes all built-in tools when no third-party present', () => {
|
||||
const ids = getToolRegistry({ dir: '/none', readDir: () => { throw new Error('ENOENT'); }, readFile: () => '' }).map((m) => m.id);
|
||||
expect(ids.sort()).toEqual([...SUPPORTED_TOOLS].sort());
|
||||
});
|
||||
|
||||
it('merges a valid third-party PATH adapter', () => {
|
||||
const reg = getToolRegistry(fakeLoaderDeps([
|
||||
{ id: 'foo-cli', displayName: 'Foo', launchable: true, hookCapable: false, hookPointer: '.foo/hm.json', detect: { kind: 'path', binaryName: 'foo' } },
|
||||
]));
|
||||
expect(reg.find((m) => m.id === 'foo-cli')).toMatchObject({ builtin: false });
|
||||
expect(reg.length).toBe(SUPPORTED_TOOLS.length + 1);
|
||||
});
|
||||
|
||||
it('built-in id wins a third-party collision', () => {
|
||||
const reg = getToolRegistry(fakeLoaderDeps([
|
||||
{ id: 'claude-code', displayName: 'HIJACK', launchable: true, hookCapable: false, hookPointer: '.x/hm.json', detect: { kind: 'path', binaryName: 'x' } },
|
||||
]));
|
||||
const cc = reg.filter((m) => m.id === 'claude-code');
|
||||
expect(cc).toHaveLength(1);
|
||||
expect(cc[0].displayName).toBe('Claude Code');
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `npx vitest run packages/agent/tests/tool-registry.test.ts`
|
||||
Expected: FAIL — module not found.
|
||||
|
||||
- [ ] **Step 3: Implement** `packages/agent/src/tool-registry.ts`:
|
||||
|
||||
```ts
|
||||
/**
|
||||
* AI-OS #5 — the tool-adapter registry. Pure data: the built-in manifests
|
||||
* (source of truth in @waggle/shared) merged with validated third-party
|
||||
* manifests from the loader. Built-in ids always win a collision so a
|
||||
* third-party file can never hijack a first-party tool.
|
||||
*/
|
||||
import { BUILTIN_TOOL_MANIFESTS, type ToolManifest } from '@waggle/shared';
|
||||
import { loadThirdPartyManifests, type ManifestLoaderDeps } from './tool-manifest-loader.js';
|
||||
|
||||
export function getToolRegistry(deps?: ManifestLoaderDeps): ToolManifest[] {
|
||||
const builtins = [...BUILTIN_TOOL_MANIFESTS];
|
||||
const builtinIds = new Set(builtins.map((m) => m.id));
|
||||
const thirdParty = loadThirdPartyManifests(deps).filter((m) => !builtinIds.has(m.id));
|
||||
return [...builtins, ...thirdParty];
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Export + run** — add to `packages/agent/src/index.ts` (after the existing tool exports):
|
||||
|
||||
```ts
|
||||
export { getToolRegistry } from './tool-registry.js';
|
||||
export { loadThirdPartyManifests, type ManifestLoaderDeps } from './tool-manifest-loader.js';
|
||||
```
|
||||
|
||||
Run: `npx vitest run packages/agent/tests/tool-registry.test.ts`
|
||||
Expected: PASS (3 cases).
|
||||
|
||||
- [ ] **Step 5: Typecheck + commit**
|
||||
|
||||
```bash
|
||||
npx tsc --noEmit -p packages/agent/tsconfig.json
|
||||
git add packages/agent/src/tool-registry.ts packages/agent/src/index.ts
|
||||
git commit -m "feat(agent): getToolRegistry — built-ins + third-party, built-in wins (#5)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: `agent` — drive `detectInstalledTools` from the registry
|
||||
|
||||
**Files:**
|
||||
- Modify: `packages/agent/src/tool-detection.ts` (the `detectInstalledTools` orchestrator + remove per-tool wrappers + `detectorsById`; keep `detectByPath`/`detectByCandidates`/`probeHooks`/candidate-path helpers)
|
||||
- Test: `packages/agent/tests/tool-detection.test.ts` (existing — must stay green; add 1 case for a third-party path adapter)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `getToolRegistry` (Task 3), `BUILTIN_TOOL_MANIFESTS` (for the candidate-resolver keys).
|
||||
|
||||
- [ ] **Step 1: Read** the existing `tool-detection.test.ts` to confirm the injected deps (`pathFromEnv`, `exists`, `execVersion`) and that it drives `detectInstalledTools(opts)`. The refactor must keep those exact seams.
|
||||
|
||||
- [ ] **Step 2: Write the failing test** — append to `tool-detection.test.ts` (mirror its existing `detectInstalledTools({...injected deps...})` setup; add a manifest-loader dep so a third-party adapter is present):
|
||||
|
||||
```ts
|
||||
it('detects a third-party PATH adapter from the registry', async () => {
|
||||
const result = await detectInstalledTools({
|
||||
platform: 'linux',
|
||||
pathFromEnv: async (bin: string) => (bin === 'foo' ? '/usr/bin/foo' : null),
|
||||
exists: async (p: string) => p === '/usr/bin/foo',
|
||||
execVersion: async () => '1.0.0',
|
||||
// NEW: inject the manifest-loader deps so the registry includes a 3rd-party tool
|
||||
manifestLoader: {
|
||||
dir: '/fake',
|
||||
readDir: () => ['foo.json'],
|
||||
readFile: () => JSON.stringify({ id: 'foo-cli', displayName: 'Foo', launchable: true, hookCapable: false, hookPointer: '.foo/hm.json', detect: { kind: 'path', binaryName: 'foo' } }),
|
||||
},
|
||||
});
|
||||
const foo = result.tools.find((t) => t.id === 'foo-cli');
|
||||
expect(foo?.installed).toBe(true);
|
||||
expect(foo?.installedPath).toBe('/usr/bin/foo');
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run test to verify it fails**
|
||||
|
||||
Run: `npx vitest run packages/agent/tests/tool-detection.test.ts -t "third-party"`
|
||||
Expected: FAIL — registry not wired; `manifestLoader` dep unknown.
|
||||
|
||||
- [ ] **Step 4: Implement** in `packages/agent/src/tool-detection.ts`:
|
||||
|
||||
(a) Imports — add `getToolRegistry` + the manifest type, and `BUILTIN_TOOL_MANIFESTS` is already imported via shared (it imports `SUPPORTED_TOOLS`/`TOOL_DISPLAY_NAMES`):
|
||||
|
||||
```ts
|
||||
import { getToolRegistry } from './tool-registry.js';
|
||||
import type { ManifestLoaderDeps } from './tool-manifest-loader.js';
|
||||
import type { ToolManifest } from '@waggle/shared';
|
||||
```
|
||||
|
||||
(b) A candidate-resolver map keyed by built-in id (the escape hatch — the 3 desktop tools). Place it near the candidate-path helpers:
|
||||
|
||||
```ts
|
||||
const CANDIDATE_RESOLVERS: Record<string, (deps: ResolvedDeps) => string[]> = {
|
||||
'cursor': cursorCandidatePaths,
|
||||
'claude-desktop': claudeDesktopCandidatePaths,
|
||||
'codex-desktop': codexDesktopCandidatePaths,
|
||||
};
|
||||
```
|
||||
|
||||
(c) A manifest-driven single detector replacing the per-tool wrappers:
|
||||
|
||||
```ts
|
||||
async function detectFromManifest(m: ToolManifest, deps: ResolvedDeps): Promise<DetectedTool> {
|
||||
if (m.detect.kind === 'path') {
|
||||
return detectByPath(m.id as ToolId, m.detect.binaryName, deps, m.hookPointer, m.displayName);
|
||||
}
|
||||
const resolver = CANDIDATE_RESOLVERS[m.id];
|
||||
const candidates = resolver ? resolver(deps) : [];
|
||||
return detectByCandidates(m.id as ToolId, candidates, deps, /* withVersion */ false, m.hookPointer, m.displayName);
|
||||
}
|
||||
```
|
||||
|
||||
(d) Generalize `detectByPath` / `detectByCandidates` / `probeHooks` to take `hookPointer` + `displayName` as params instead of reading `TOOL_DISPLAY_NAMES[id]` / `HOOK_POINTER_BY_TOOL[id]` (so third-party ids work). Change their signatures to accept `hookPointer: string, displayName: string`, and have `probeHooks(hookPointer, deps)` use the passed pointer. Keep `HOOK_POINTER_BY_TOOL` as a derived export for back-compat:
|
||||
|
||||
```ts
|
||||
// derived from the manifests (was a hand-authored Record)
|
||||
export const HOOK_POINTER_BY_TOOL = Object.fromEntries(
|
||||
BUILTIN_TOOL_MANIFESTS.map((m) => [m.id, m.hookPointer]),
|
||||
) as Record<ToolId, string>;
|
||||
```
|
||||
|
||||
(Import `BUILTIN_TOOL_MANIFESTS` from `@waggle/shared` for this.)
|
||||
|
||||
(e) Replace `detectInstalledTools`'s body:
|
||||
|
||||
```ts
|
||||
export async function detectInstalledTools(
|
||||
opts: ToolDetectionDeps & { manifestLoader?: ManifestLoaderDeps } = {},
|
||||
): Promise<ToolDetectionResult> {
|
||||
const deps = resolveDeps(opts);
|
||||
const registry = getToolRegistry(opts.manifestLoader);
|
||||
const tools = await Promise.all(registry.map((m) => detectFromManifest(m, deps)));
|
||||
return { platform: deps.platform, detectedAt: new Date().toISOString(), tools };
|
||||
}
|
||||
```
|
||||
|
||||
(f) Delete `detectClaudeCode`, `detectCursor`, `detectClaudeDesktop`, `detectCodex`, `detectHermes`, `detectOpenClaw`, `detectCodexDesktop`, the old `detectorsById`, and the old hand-authored `HOOK_POINTER_BY_TOOL` block. Keep `detectByPath`, `detectByCandidates`, `probeHooks`, and the 3 `*CandidatePaths` helpers.
|
||||
|
||||
> Note: `detectByPath` currently hardcodes `pathFromEnv('claude')` only in the inline claude-code path — the generic `detectByPath` already takes `binaryName`, so claude-code now flows through it with `binaryName: 'claude'` from its manifest. Drop the special-case inline detector.
|
||||
|
||||
- [ ] **Step 5: Run the FULL existing suite + new case**
|
||||
|
||||
Run: `npx vitest run packages/agent/tests/tool-detection.test.ts`
|
||||
Expected: PASS — all existing 7-tool detection assertions + the new third-party case.
|
||||
|
||||
- [ ] **Step 6: Typecheck + commit**
|
||||
|
||||
```bash
|
||||
npx tsc --noEmit -p packages/agent/tsconfig.json
|
||||
git add packages/agent/src/tool-detection.ts packages/agent/tests/tool-detection.test.ts
|
||||
git commit -m "feat(agent): registry-driven detectInstalledTools (#5)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 5: `agent` — derive launcher cohorts from the registry
|
||||
|
||||
**Files:**
|
||||
- Modify: `packages/agent/src/tool-launcher.ts` (`HOOKS_COHORT`, `hookPackageFor`, the `launchTool` cohort guard)
|
||||
- Test: `packages/agent/tests/tool-launcher.test.ts` (existing — stays green; add a manifest-derivation assertion)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `BUILTIN_TOOL_MANIFESTS` (Task 1).
|
||||
|
||||
- [ ] **Step 1: Write the failing test** — append to `tool-launcher.test.ts`:
|
||||
|
||||
```ts
|
||||
import { BUILTIN_TOOL_MANIFESTS } from '@waggle/shared';
|
||||
it('HOOKS_COHORT equals the hook-capable manifests (claude-desktop excluded)', () => {
|
||||
const expected = BUILTIN_TOOL_MANIFESTS.filter((m) => m.hookCapable).map((m) => m.id).sort();
|
||||
expect([...HOOKS_COHORT].sort()).toEqual(expected);
|
||||
expect(HOOKS_COHORT).not.toContain('claude-desktop');
|
||||
});
|
||||
```
|
||||
|
||||
(`HOOKS_COHORT` is already imported/exported from `tool-launcher.ts`; add the import if missing.)
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `npx vitest run packages/agent/tests/tool-launcher.test.ts -t "HOOKS_COHORT equals"`
|
||||
Expected: FAIL — if the literal differs OR (more likely) passes already; if it passes, still proceed to make the derivation real so it can't drift.
|
||||
|
||||
- [ ] **Step 3: Implement** in `packages/agent/src/tool-launcher.ts` — replace the literal `HOOKS_COHORT` with a derived one and let `hookPackageFor` honor a manifest override:
|
||||
|
||||
```ts
|
||||
import { BUILTIN_TOOL_MANIFESTS, type ToolId } from '@waggle/shared';
|
||||
|
||||
export const HOOKS_COHORT: readonly ToolId[] =
|
||||
BUILTIN_TOOL_MANIFESTS.filter((m) => m.hookCapable).map((m) => m.id as ToolId);
|
||||
```
|
||||
|
||||
(Keep `LAUNCH_COHORT`'s import from shared as-is — it's already derived in Task 1. The `launchTool` guard `if (!LAUNCH_COHORT.includes(opts.id))` is unchanged.)
|
||||
|
||||
- [ ] **Step 4: Run the full launcher suite**
|
||||
|
||||
Run: `npx vitest run packages/agent/tests/tool-launcher.test.ts`
|
||||
Expected: PASS (existing 39 + new).
|
||||
|
||||
- [ ] **Step 5: Typecheck + commit**
|
||||
|
||||
```bash
|
||||
npx tsc --noEmit -p packages/agent/tsconfig.json
|
||||
git add packages/agent/src/tool-launcher.ts packages/agent/tests/tool-launcher.test.ts
|
||||
git commit -m "feat(agent): derive HOOKS_COHORT from the manifest registry (#5)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 6: `web` — derive LauncherApp cohorts from shared manifests
|
||||
|
||||
**Files:**
|
||||
- Modify: `apps/web/src/components/os/apps/LauncherApp.tsx` (the local `LAUNCH_COHORT` / `HOOKS_COHORT` copies, ~lines 41–59)
|
||||
- Test: `apps/web/src/components/os/apps/LauncherApp.test.tsx` (existing — stays green)
|
||||
|
||||
- [ ] **Step 1: Implement** — replace the hand-maintained local arrays with derivations from the shared manifests (kills the "kept local to avoid a runtime import" duplication the comments call out):
|
||||
|
||||
```ts
|
||||
import { BUILTIN_TOOL_MANIFESTS } from '@waggle/shared';
|
||||
|
||||
const LAUNCH_COHORT = BUILTIN_TOOL_MANIFESTS.filter((m) => m.launchable).map((m) => m.id);
|
||||
const HOOKS_COHORT = BUILTIN_TOOL_MANIFESTS.filter((m) => m.hookCapable).map((m) => m.id);
|
||||
```
|
||||
|
||||
(Delete the two local literal arrays + their explanatory comments.)
|
||||
|
||||
- [ ] **Step 2: Run the LauncherApp suite + web typecheck**
|
||||
|
||||
Run: `cd apps/web && npx vitest run -c vitest.config.ts src/components/os/apps/LauncherApp.test.tsx && npx tsc --noEmit`
|
||||
Expected: PASS (6 tests) + tsc 0.
|
||||
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add apps/web/src/components/os/apps/LauncherApp.tsx
|
||||
git commit -m "feat(web): derive LauncherApp cohorts from shared manifests (#5)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Final Gate
|
||||
|
||||
- [ ] **Build dists then full typecheck:** `npx tsc -b packages/shared packages/agent && npx tsc --noEmit -p packages/server/tsconfig.json && (cd apps/web && npx tsc --noEmit)` → 0.
|
||||
- [ ] **Touched suites:** `npx vitest run packages/shared/tests/tool-manifests.test.ts packages/agent/tests/tool-manifest-loader.test.ts packages/agent/tests/tool-registry.test.ts packages/agent/tests/tool-detection.test.ts packages/agent/tests/tool-launcher.test.ts` + `(cd apps/web && npx vitest run -c vitest.config.ts src/components/os/apps/LauncherApp.test.tsx)`.
|
||||
- [ ] **Server route sanity:** `npx vitest run packages/server/tests/tools-routes.test.ts packages/server/tests/tools-routes-launch.test.ts` (they consume detect/launch — must stay green).
|
||||
- [ ] **Lint** touched files.
|
||||
|
||||
## Fast-follow (documented, NOT silently dropped)
|
||||
|
||||
**Third-party `promptArgTemplate` application.** The field is captured + validated in v1 but unwired:
|
||||
the inline-prompt arg logic lives web-side (`apps/web/src/lib/launcher-prompt-args.ts`) and the dock
|
||||
computes args before POSTing. Wiring a loaded manifest's `promptArgTemplate` means surfacing the
|
||||
registry (or its prompt templates) to the web bundle — additive, deferred. Built-in prompt-args are
|
||||
unchanged.
|
||||
|
||||
## Self-Review
|
||||
|
||||
**Spec coverage:** §3 Unit A → Task 1 · Unit B (registry) → Task 3 · Unit B (detection) → Task 4 · Unit B (launcher) → Task 5 · Unit C (loader) → Task 2 · web derivation (§7 file list) → Task 6 · §5 security → Task 2 tests. The `promptArgTemplate` application is logged as a fast-follow (field captured in Task 1).
|
||||
|
||||
**Placeholder scan:** Task 4 Step 1 + Task 6 are read/edit steps with concrete code; no TBD/TODO. Every code step ships real code.
|
||||
|
||||
**Type consistency:** `ToolManifest`/`ToolDetectSpec` (Task 1) consumed unchanged in Tasks 2–6. `loadThirdPartyManifests(deps)` (Task 2) → `getToolRegistry(deps)` (Task 3) → `detectInstalledTools({manifestLoader})` (Task 4). `getToolRegistry` returns `ToolManifest[]`. `detectFromManifest`/`CANDIDATE_RESOLVERS` names consistent within Task 4. `HOOKS_COHORT`/`LAUNCH_COHORT` derived identically in Tasks 1/5/6.
|
||||
659
docs/superpowers/plans/2026-07-08-ux-phase-1-corrections.md
Normal file
659
docs/superpowers/plans/2026-07-08-ux-phase-1-corrections.md
Normal file
@@ -0,0 +1,659 @@
|
||||
# UX Phase 1 Corrections Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Remove the trust, coherence, responsive layout, shortcut, visual-regression, and route-evidence blockers that currently prevent the five-persona UX judge gate from honestly reaching 9/10.
|
||||
|
||||
**Architecture:** Keep all fixes inside existing Waggle surfaces. Do not create new pages, flows, or abstractions unless a tiny local helper is needed to make an existing surface testable. Use current route shell, Settings app, auth provider, marketplace/visual tests, and judge artifacts as the proof path.
|
||||
|
||||
**Tech Stack:** React 19, TypeScript, Vite, Tailwind 4, Fastify local sidecar, Vitest, Playwright.
|
||||
|
||||
---
|
||||
|
||||
## Source Artifacts
|
||||
|
||||
- Findings: `docs/audits/2026-07-08-complete-ux-usage-audit.md`
|
||||
- Route/scenario manifest: `docs/audits/2026-07-08-ux-route-scenario-manifest.md`
|
||||
- Judge scorecards: `docs/audits/2026-07-08-five-persona-judge-scorecards.md`
|
||||
- Correction register: `docs/audits/2026-07-08-ux-correction-register.md`
|
||||
- Mobile Executive evidence supplement: `docs/audits/2026-07-08-mobile-executive-t2-t12-analysis.md`
|
||||
- First-run onboarding evidence supplement: `docs/audits/2026-07-08-first-run-onboarding-t1-t2-t12-analysis.md`
|
||||
|
||||
## Files by Responsibility
|
||||
|
||||
- `packages/server/src/local/security-middleware.ts`: local CSP/security headers.
|
||||
- `packages/server/tests/local/security-middleware.test.ts`: CSP/security header assertions.
|
||||
- `apps/web/src/lib/clerk.ts`: Clerk publishable-key resolution and accountless decision point.
|
||||
- `apps/web/src/providers/WaggleClerkProvider.tsx`: optional Clerk mounting.
|
||||
- `apps/web/src/components/os/apps/SettingsApp.tsx`: Settings layout, copy, backup alerts, billing copy.
|
||||
- `apps/web/src/components/os/overlays/OnboardingWizard.tsx`: first-run wizard shell and mobile action row.
|
||||
- `apps/web/src/components/os/overlays/onboarding/WhoAreYouStep.tsx`: first-run Profile step density and Continue reachability.
|
||||
- `apps/web/src/components/os/AppShell.tsx`: active workspace resolution, route-changing overlay behavior, shortcut target.
|
||||
- `apps/web/src/hooks/useKeyboardShortcuts.ts`: `Ctrl+Shift+N` dispatch path.
|
||||
- `apps/web/src/components/os/overlays/WorkspaceSwitcher.tsx`: modal close behavior and route traversal interactions.
|
||||
- `apps/web/src/components/os/apps/MarketplaceApp.tsx`: Marketplace/MCP copy.
|
||||
- `apps/web/src/components/os/apps/mcp/AddCustomMcpForm.tsx`: custom MCP gating copy.
|
||||
- `apps/web/src/lib/command-catalog.ts`: Command Center group labels.
|
||||
- `apps/web/src/components/os/overlays/LoginBriefing.tsx`: login/team copy.
|
||||
- `apps/web/src/components/os/apps/skills/SkillRow.tsx`: skill verification copy.
|
||||
- `apps/web/src/components/os/apps/PaymentSuccessApp.tsx`: Teams success and legacy Pro copy boundary.
|
||||
- `tests/visual/views.spec.ts`: visual regression route list/readiness and baseline triage.
|
||||
- `tests/e2e/phase-ab-verification.spec.ts`: `Ctrl+Shift+N` and CSP console assertions.
|
||||
- `tests/e2e/power-user-stress.spec.ts`: shortcut stress assertion.
|
||||
- `tests/e2e/full-wiring-audit.spec.ts`: full traversal/console route behavior.
|
||||
- `tests/e2e/full-product-audit.spec.ts`: load console health.
|
||||
- `tests/e2e/user-journeys.spec.ts`: route smoke expansion for thin routes.
|
||||
- `docs/audits/2026-07-08-ux-route-scenario-manifest.md`: update route evidence statuses after fixes.
|
||||
- `docs/audits/2026-07-08-five-persona-judge-scorecards.md`: update readiness notes after fixes.
|
||||
|
||||
## Phase Rules
|
||||
|
||||
- Keep work inside the files above unless a test exposes a direct local dependency.
|
||||
- Re-read a file immediately before editing it.
|
||||
- Write or adjust the failing test before the implementation for each task.
|
||||
- Do not update visual baselines until the rendered screenshots are reviewed.
|
||||
- Do not mark the five-persona gate ready until all P0 findings are closed.
|
||||
- If a task reveals unrelated pre-existing dirty files, leave them untouched.
|
||||
|
||||
## Task 1: Local Auth, Clerk, and CSP Console Health
|
||||
|
||||
**Files:**
|
||||
- Modify: `packages/server/src/local/security-middleware.ts`
|
||||
- Modify: `packages/server/tests/local/security-middleware.test.ts`
|
||||
- Modify: `apps/web/src/lib/clerk.ts`
|
||||
- Modify: `apps/web/src/providers/WaggleClerkProvider.tsx`
|
||||
- Test: `tests/e2e/full-product-audit.spec.ts`
|
||||
- Test: `tests/e2e/phase-ab-verification.spec.ts`
|
||||
|
||||
- [x] **Step 1: Add/adjust CSP test for strict local accountless mode**
|
||||
|
||||
Add assertions in `packages/server/tests/local/security-middleware.test.ts` that document the accountless default:
|
||||
|
||||
```ts
|
||||
expect(csp).toContain("script-src 'self'");
|
||||
expect(csp).not.toMatch(/script-src[^;]*clerk/i);
|
||||
expect(csp).not.toMatch(/connect-src[^;]*clerk/i);
|
||||
```
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
npx vitest run packages/server/tests/local/security-middleware.test.ts
|
||||
```
|
||||
|
||||
Expected before implementation review: current assertions pass for strict CSP, but rendered E2E still fails because client-side Clerk mounts when the local env contains a key.
|
||||
|
||||
- [x] **Step 2: Make accountless local mode the default client behavior**
|
||||
|
||||
In `apps/web/src/lib/clerk.ts`, keep `clerkPublishableKey()` shape validation, but add a local opt-in guard so the desktop/local audit lane does not mount Clerk just because `.env.local` contains `VITE_CLERK_PUBLISHABLE_KEY`.
|
||||
|
||||
Use this behavior:
|
||||
|
||||
```ts
|
||||
function clerkEnabledForThisBuild(): boolean {
|
||||
return import.meta.env.VITE_WAGGLE_ENABLE_CLERK === '1';
|
||||
}
|
||||
|
||||
export function clerkPublishableKey(): string | undefined {
|
||||
if (!clerkEnabledForThisBuild()) return undefined;
|
||||
const k = import.meta.env.VITE_CLERK_PUBLISHABLE_KEY;
|
||||
return k && isValidPublishableKey(k) ? k : undefined;
|
||||
}
|
||||
```
|
||||
|
||||
Keep the existing invalid-key fallback. Update the surrounding comment so it says Clerk is enabled only when the publishable key is valid and `VITE_WAGGLE_ENABLE_CLERK=1`.
|
||||
|
||||
- [x] **Step 3: Add a web unit test for accountless default**
|
||||
|
||||
Create `apps/web/src/test/clerk-accountless.test.ts` if no equivalent exists. Test the exported function by stubbing `import.meta.env` in the same style used by nearby web tests. The test must assert:
|
||||
|
||||
```ts
|
||||
expect(clerkPublishableKey()).toBeUndefined();
|
||||
```
|
||||
|
||||
with a valid-looking key present and `VITE_WAGGLE_ENABLE_CLERK` absent.
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
cd apps/web
|
||||
npx vitest run src/test/clerk-accountless.test.ts
|
||||
cd ../..
|
||||
```
|
||||
|
||||
Expected: pass after Step 2.
|
||||
|
||||
- [x] **Step 4: Verify rendered console health**
|
||||
|
||||
Run a fresh-port focused lane:
|
||||
|
||||
```powershell
|
||||
$env:WAGGLE_E2E_PORT='3391'
|
||||
$env:WAGGLE_E2E_BASE_URL='http://127.0.0.1:3391'
|
||||
$env:WAGGLE_E2E_SKIP_LITELLM='1'
|
||||
node node_modules/playwright/cli.js test tests/e2e/full-product-audit.spec.ts tests/e2e/phase-ab-verification.spec.ts --project=chromium --reporter=list
|
||||
```
|
||||
|
||||
Expected after the task: no Clerk 429/load errors and no CSP failure caused by Clerk or the inline startup script.
|
||||
|
||||
- [x] **Step 5: Verify clean-data first-run console health**
|
||||
|
||||
Run or codify a no-skip first-run smoke with a fresh data dir. It must navigate to `/` without `skipOnboarding`, wait for the onboarding takeover, and capture console/page errors from navigation start.
|
||||
|
||||
Minimum manual lane if no test exists yet:
|
||||
|
||||
```powershell
|
||||
$env:WAGGLE_PORT='3392'
|
||||
$env:WAGGLE_TRUST_LOCALHOST='1'
|
||||
$env:WAGGLE_DISABLE_MARKETPLACE_SYNC='1'
|
||||
$env:EMBEDDING_PROVIDER='mock'
|
||||
$env:VITE_CLERK_PUBLISHABLE_KEY=''
|
||||
$env:CLERK_SECRET_KEY=''
|
||||
$env:WAGGLE_DATA_DIR="$env:TEMP\waggle-first-run-auth-3392"
|
||||
npm run build
|
||||
npx tsx packages/server/src/local/start.ts --skip-litellm
|
||||
```
|
||||
|
||||
Expected after the task: clean-data first-run onboarding has no Clerk/CSP/page errors, and the wizard still renders before shell chrome.
|
||||
|
||||
**Completed 2026-07-08:** `clerkPublishableKey()` now requires `VITE_WAGGLE_ENABLE_CLERK=1` in addition to a valid key; the pre-hydration theme bootstrap moved from inline HTML to `/theme-boot.js` so local CSP keeps `script-src 'self'`; console-health E2E assertions now explicitly fail on Clerk/CSP noise. Verification: web auth Vitest 10/10, server security middleware Vitest 43/43, phase-ab initial-load console Playwright 1/1, full-product console Playwright 1/1, clean first-run onboarding Playwright 1/1, and `git diff --check`.
|
||||
|
||||
## Task 2: Mobile Settings and First-Run Onboarding Responsive Layout
|
||||
|
||||
**Files:**
|
||||
- Modify: `apps/web/src/components/os/apps/SettingsApp.tsx`
|
||||
- Modify: `apps/web/src/components/os/overlays/OnboardingWizard.tsx`
|
||||
- Modify: `apps/web/src/components/os/overlays/onboarding/WhoAreYouStep.tsx`
|
||||
- Test: `apps/web/src/test/pr5-settings-reskin.test.tsx`
|
||||
- Test: add or extend Playwright mobile coverage in `tests/e2e/user-journeys.spec.ts`
|
||||
|
||||
- [x] **Step 1: Add mobile Settings assertions that catch visible clipping**
|
||||
|
||||
In `tests/e2e/user-journeys.spec.ts`, add a test that sets the viewport to `390 x 844`, opens `/settings`, `/settings?tab=models`, `/settings?tab=billing`, and `/settings/profile`, and asserts both no document-level horizontal overflow and no visible critical-control overflow.
|
||||
|
||||
Do not rely on this check alone:
|
||||
|
||||
```ts
|
||||
document.documentElement.scrollWidth > document.documentElement.clientWidth
|
||||
```
|
||||
|
||||
The fresh mobile smoke in `docs/audits/2026-07-08-mobile-executive-t2-t12-analysis.md` found clipped/squeezed controls while document scroll width stayed clean.
|
||||
|
||||
Use a helper shaped like this:
|
||||
|
||||
```ts
|
||||
test('J-mobile: Settings is usable at 390px width', async ({ page }) => {
|
||||
await page.setViewportSize({ width: 390, height: 844 });
|
||||
|
||||
const routes = ['/settings', '/settings?tab=models', '/settings?tab=billing', '/settings/profile'];
|
||||
for (const route of routes) {
|
||||
await gotoApp(page, route);
|
||||
if (route !== '/settings/profile') {
|
||||
await expect(page.getByRole('tablist', { name: 'Settings sections' })).toBeVisible();
|
||||
await expect(page.getByRole('tabpanel').first()).toBeVisible();
|
||||
} else {
|
||||
await expect(page.locator('body')).toContainText(/profile|identity|save|writing style/i);
|
||||
}
|
||||
|
||||
const documentOverflow = await page.evaluate(
|
||||
() => document.documentElement.scrollWidth > document.documentElement.clientWidth,
|
||||
);
|
||||
expect(documentOverflow, `${route} document overflow`).toBe(false);
|
||||
|
||||
const visibleOverflow = await page.locator(
|
||||
'button:visible, [role="tab"]:visible, [role="tabpanel"]:visible, input:visible, select:visible, textarea:visible',
|
||||
).evaluateAll(elements => elements
|
||||
.map(el => {
|
||||
const rect = el.getBoundingClientRect();
|
||||
return {
|
||||
text: (el.textContent || el.getAttribute('aria-label') || el.getAttribute('placeholder') || el.tagName).trim(),
|
||||
left: Math.floor(rect.left),
|
||||
right: Math.ceil(rect.right),
|
||||
};
|
||||
})
|
||||
.filter(item => item.left < -1 || item.right > window.innerWidth + 1));
|
||||
|
||||
expect(visibleOverflow, `${route} visible control overflow`).toEqual([]);
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
node node_modules/playwright/cli.js test tests/e2e/user-journeys.spec.ts --project=chromium --grep "Settings is usable at 390px"
|
||||
```
|
||||
|
||||
Expected before implementation: fail on `/settings`, `/settings?tab=models`, or `/settings?tab=billing` because the current two-rail layout squeezes or clips visible controls even without document-level overflow.
|
||||
|
||||
- [x] **Step 2: Add first-run mobile onboarding assertions**
|
||||
|
||||
Add a no-skip mobile first-run assertion using a clean data dir or an isolated server fixture. At 390 x 844:
|
||||
|
||||
- `/` renders the Onboarding Wizard, not shell chrome.
|
||||
- Welcome has no horizontal overflow.
|
||||
- After pressing Continue, the Profile step shows the progress/top bar, profile content, and primary Continue action without the action starting below the visible viewport, or the action is sticky/clearly reachable by design.
|
||||
- Console capture is shared with Task 1, so Clerk/CSP errors remain visible until T1 is fixed.
|
||||
|
||||
Use `docs/audits/2026-07-08-first-run-onboarding-t1-t2-t12-analysis.md` as the red evidence source. The current trace puts the Profile Continue button bottom at `880` in an `844` px viewport.
|
||||
|
||||
- [x] **Step 3: Replace the fixed Settings rail on narrow screens**
|
||||
|
||||
In `SettingsApp.tsx`, replace the root/tabs layout classes:
|
||||
|
||||
```tsx
|
||||
<div className="flex h-full">
|
||||
<div className="w-36 border-r border-border/50 shrink-0 flex flex-col">
|
||||
```
|
||||
|
||||
with responsive classes:
|
||||
|
||||
```tsx
|
||||
<div className="flex h-full min-w-0 flex-col md:flex-row">
|
||||
<div className="shrink-0 border-b border-border/50 md:w-36 md:border-b-0 md:border-r flex flex-col">
|
||||
```
|
||||
|
||||
Change the tablist container from vertical-only spacing to mobile horizontal scroll:
|
||||
|
||||
```tsx
|
||||
<div
|
||||
className="flex gap-1 overflow-x-auto p-2 md:block md:space-y-0.5"
|
||||
role="tablist"
|
||||
aria-label="Settings sections"
|
||||
>
|
||||
```
|
||||
|
||||
Change tab button classes from full-width only to responsive:
|
||||
|
||||
```tsx
|
||||
className={`flex min-w-max items-center gap-2 rounded-lg px-2 py-1.5 text-xs transition-colors md:w-full ${...}`}
|
||||
```
|
||||
|
||||
- [x] **Step 4: Make Settings header controls wrap gracefully**
|
||||
|
||||
Change the header row:
|
||||
|
||||
```tsx
|
||||
<div className="flex items-center justify-between gap-2 px-4 pt-3 pb-2.5 border-b border-border/40 shrink-0">
|
||||
```
|
||||
|
||||
to:
|
||||
|
||||
```tsx
|
||||
<div className="flex flex-wrap items-center justify-between gap-2 px-3 pt-3 pb-2.5 border-b border-border/40 shrink-0 sm:px-4">
|
||||
```
|
||||
|
||||
Change the content padding:
|
||||
|
||||
```tsx
|
||||
<div className="flex-1 p-4 overflow-auto" role="tabpanel">
|
||||
```
|
||||
|
||||
to:
|
||||
|
||||
```tsx
|
||||
<div className="flex-1 overflow-auto p-3 sm:p-4" role="tabpanel">
|
||||
```
|
||||
|
||||
- [x] **Step 5: Make onboarding Profile mobile-reachable**
|
||||
|
||||
Prefer the smallest change that preserves the existing wizard:
|
||||
|
||||
- reduce mobile vertical density in `WhoAreYouStep`, or
|
||||
- make the wizard action row sticky within the scroll container, or
|
||||
- split optional details into a secondary mobile section while keeping the current desktop layout.
|
||||
|
||||
Do not remove the personalization signals; the fix is reachability and visual hierarchy.
|
||||
|
||||
- [x] **Step 6: Verify mobile and desktop**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
node node_modules/playwright/cli.js test tests/e2e/user-journeys.spec.ts --project=chromium --grep "Settings"
|
||||
cd apps/web
|
||||
npx vitest run src/test/pr5-settings-reskin.test.tsx
|
||||
cd ../..
|
||||
```
|
||||
|
||||
Expected: mobile tests pass for Settings general, Models, Billing, and Profile; screenshots show no clipped critical controls; existing desktop Settings tab behavior still passes.
|
||||
|
||||
Additional expected result: clean-data mobile first-run Welcome/Profile screenshots show no hidden primary action and the desktop first-run path still reaches workspace chat after template and first task.
|
||||
|
||||
**Completed 2026-07-08:** Settings now stacks/wraps its tab rail and header controls on narrow screens; the first-run wizard content starts at the top on mobile, and the Profile step uses tighter mobile density so the Continue action is initially reachable at 390 x 844. Verification: `J-mobile` Playwright tests failed red on Settings overflow and Profile Continue bottom `870 > 844`, then passed 2/2 after the responsive changes; `pr5-settings-reskin.test.tsx` passed 3/3.
|
||||
|
||||
## Task 3: Solo/Teams/Enterprise Pricing and Gating Copy
|
||||
|
||||
**Files:**
|
||||
- Modify: `apps/web/src/components/os/apps/MarketplaceApp.tsx`
|
||||
- Modify: `apps/web/src/components/os/apps/mcp/AddCustomMcpForm.tsx`
|
||||
- Modify: `apps/web/src/lib/command-catalog.ts`
|
||||
- Modify: `apps/web/src/components/os/overlays/LoginBriefing.tsx`
|
||||
- Modify: `apps/web/src/components/os/apps/skills/SkillRow.tsx`
|
||||
- Modify: `apps/web/src/components/os/apps/SettingsApp.tsx`
|
||||
- Modify: `apps/web/src/components/os/apps/PaymentSuccessApp.tsx`
|
||||
|
||||
- [x] **Step 1: Add a copy guard command**
|
||||
|
||||
Run before editing:
|
||||
|
||||
```powershell
|
||||
rg -n "Pro|PRO|pro_" apps/web/src/components/os/apps/MarketplaceApp.tsx apps/web/src/components/os/apps/mcp/AddCustomMcpForm.tsx apps/web/src/lib/command-catalog.ts apps/web/src/components/os/overlays/LoginBriefing.tsx apps/web/src/components/os/apps/skills/SkillRow.tsx apps/web/src/components/os/apps/SettingsApp.tsx apps/web/src/components/os/apps/PaymentSuccessApp.tsx
|
||||
```
|
||||
|
||||
Expected current hits include active user-facing Pro copy in Marketplace, AddCustomMcpForm, Command Center, LoginBriefing, SkillRow, and Settings.
|
||||
|
||||
- [x] **Step 2: Replace active Pro copy**
|
||||
|
||||
Use these replacements:
|
||||
|
||||
```ts
|
||||
// MarketplaceApp.tsx
|
||||
mcp: 'Enable MCP servers here (security-scanned). Manage running servers in the MCP Hub.',
|
||||
```
|
||||
|
||||
```tsx
|
||||
// AddCustomMcpForm.tsx
|
||||
Registers a local stdio server (command + args). Teams governance can manage shared use; Solo can run local servers on this device.
|
||||
```
|
||||
|
||||
```ts
|
||||
// command-catalog.ts
|
||||
heading: "Pinned"
|
||||
```
|
||||
|
||||
```tsx
|
||||
// SkillRow.tsx title
|
||||
title="Run the skill against a synthesized test and grade it - mints the verified badge"
|
||||
```
|
||||
|
||||
```tsx
|
||||
// SettingsApp.tsx local-first note
|
||||
it is independent of your Solo/Team plan, and every app stays reachable via Ctrl+K.
|
||||
```
|
||||
|
||||
In `LoginBriefing.tsx`, replace "Pro/Teams" and "Pro/Enterprise" phrasing with "Teams" or "Teams/Enterprise" depending on whether the copy refers to shared workspaces or sovereign deployment.
|
||||
|
||||
- [x] **Step 3: Preserve explicit legacy billing context only**
|
||||
|
||||
In `PaymentSuccessApp.tsx`, keep legacy `PRO` mapping only if the rendered copy clearly says it is a legacy state. Do not present Pro as an active upgrade path.
|
||||
|
||||
- [x] **Step 4: Re-run copy guard**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
rg -n "Pro|PRO|pro_" apps/web/src
|
||||
```
|
||||
|
||||
Expected: remaining hits are comments, type/legacy billing compatibility, or explicit "legacy Pro" servicing only. Active upgrade or gating copy must not say Pro.
|
||||
|
||||
**Completed 2026-07-08:** Active user-facing Pro copy was removed from Marketplace MCP copy, custom MCP copy, Command Center pinned heading, LoginBriefing workspace hints, SkillRow verification tooltip, and Settings local-first/billing comments. Legacy PRO billing remains only as explicit legacy state (`Legacy Pro` / `Pro (legacy)`). Verification: copy guard rerun leaves only false positives (`Props`, `Providers`), internal `isPro`, and explicit legacy PRO handling; `command-catalog.test.ts` passed 2/2 and `pr7a-billing.test.tsx` passed 10/10.
|
||||
|
||||
## Task 4: `Ctrl+Shift+N` and Workspace Switcher Route Contract
|
||||
|
||||
**Files:**
|
||||
- Modify: `apps/web/src/components/os/AppShell.tsx`
|
||||
- Modify: `apps/web/src/hooks/useKeyboardShortcuts.ts` only if comments/tests require it.
|
||||
- Modify: `apps/web/src/components/os/overlays/WorkspaceSwitcher.tsx` only if route-close behavior remains failing.
|
||||
- Test: `tests/e2e/phase-ab-verification.spec.ts`
|
||||
- Test: `tests/e2e/power-user-stress.spec.ts`
|
||||
- Test: `tests/e2e/full-wiring-audit.spec.ts`
|
||||
|
||||
- [x] **Step 1: Preserve the existing failing tests as the red proof**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
node node_modules/playwright/cli.js test tests/e2e/phase-ab-verification.spec.ts tests/e2e/power-user-stress.spec.ts --project=chromium --grep "Ctrl\\+Shift\\+N"
|
||||
```
|
||||
|
||||
Expected current result: failures where the URL does not become `/workspaces/:id/chat`.
|
||||
|
||||
- [x] **Step 2: Implement deterministic chat target resolution**
|
||||
|
||||
In `AppShell.tsx`, replace `navigateToActiveChat` with a resolver that uses:
|
||||
|
||||
1. Explicit active workspace.
|
||||
2. First non-archived workspace from the loaded list.
|
||||
3. Workspace Switcher only when no workspace exists.
|
||||
|
||||
Use this implementation shape:
|
||||
|
||||
```tsx
|
||||
const firstAvailableWorkspaceId = useMemo(
|
||||
() => workspaces.find(ws => ws.status !== 'archived')?.id ?? null,
|
||||
[workspaces],
|
||||
);
|
||||
|
||||
const chatShortcutWorkspaceId = effectiveActiveWorkspaceId ?? firstAvailableWorkspaceId;
|
||||
|
||||
const navigateToActiveChat = useCallback(() => {
|
||||
if (chatShortcutWorkspaceId) {
|
||||
selectWorkspace(chatShortcutWorkspaceId);
|
||||
ov.setShowWorkspaceSwitcher(false);
|
||||
navigate(routeFor('chat', { activeWorkspaceId: chatShortcutWorkspaceId }));
|
||||
return;
|
||||
}
|
||||
ov.toggleWorkspaceSwitcher();
|
||||
}, [chatShortcutWorkspaceId, navigate, ov, selectWorkspace]);
|
||||
```
|
||||
|
||||
`setShowWorkspaceSwitcher` exists in `apps/web/src/hooks/useOverlayState.ts`; do not add a second overlay state store.
|
||||
|
||||
- [x] **Step 3: Close Workspace Switcher during route-changing nav**
|
||||
|
||||
In the sidebar/search handlers that navigate to a new route, close the switcher before or after `navigate(route)`. The minimal target is the failing `full-wiring-audit` traversal path. Use the existing overlay setter if available; otherwise add a small `closeBlockingOverlaysBeforeRouteChange()` helper inside `AppShell.tsx`.
|
||||
|
||||
- [x] **Step 4: Verify shortcuts and full traversal**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
node node_modules/playwright/cli.js test tests/e2e/phase-ab-verification.spec.ts tests/e2e/power-user-stress.spec.ts tests/e2e/full-wiring-audit.spec.ts --project=chromium --grep "Ctrl\\+Shift\\+N|Full Console Error Audit"
|
||||
```
|
||||
|
||||
Expected: both shortcut tests pass and the console traversal no longer fails because a Workspace Switcher backdrop intercepts navigation.
|
||||
|
||||
**Completed 2026-07-08:** `Ctrl+Shift+N` and the Chat sidebar item now resolve chat through the same deterministic target: explicit active workspace, first non-archived loaded workspace, then Workspace Switcher only when no workspace exists. A cold-load race was fixed with a pending shortcut state that waits for `workspacesLoading` to settle before deciding. Route-changing open-app, keyboard app, Command Center, and Workspace Switcher callbacks now close the switcher before navigation. Verification: the red shortcut lane failed 2/2 on port 34130, an intermediate green attempt exposed the cold-load race, then `Ctrl+Shift+N` passed 2/2 on port 34133; the scoped gate passed 4/4 on port 34135 for both shortcut specs plus `Full Console Error Audit`.
|
||||
|
||||
## Task 5: Visual Snapshot Triage
|
||||
|
||||
**Files:**
|
||||
- Inspect/possibly modify: `tests/visual/views.spec.ts`
|
||||
- Inspect/update after review: `tests/visual/baselines/**`
|
||||
- Do not modify UI just to satisfy stale snapshots.
|
||||
|
||||
- [x] **Step 1: Re-run visual suite and preserve artifacts**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
node node_modules/playwright/cli.js test tests/visual/views.spec.ts --project=chromium --reporter=list
|
||||
```
|
||||
|
||||
Expected current result: 14 failures across 7 views in dark and light.
|
||||
|
||||
Analysis note: completed on fresh port 3463 with artifacts under `output/playwright/visual-t5-3463/test-results/`.
|
||||
|
||||
- [x] **Step 2: Review actual screenshots**
|
||||
|
||||
Open the generated `test-results/**` images for each failed view. Classify each as:
|
||||
|
||||
```text
|
||||
intentional drift -> update baseline
|
||||
real UI regression -> fix UI first
|
||||
test readiness problem -> fix wait/stabilization in tests/visual/views.spec.ts
|
||||
```
|
||||
|
||||
Record the classification in `docs/audits/2026-07-08-complete-ux-usage-audit.md` under P0-4 or in a short new visual decision note.
|
||||
|
||||
Analysis note: current classification is recorded in `docs/audits/2026-07-08-visual-t5-classification.md`. Most failures are stale-baseline drift, with Settings waiting for T2/T3 and Memory/Skills/Mission Control retaining targeted follow-up checks.
|
||||
|
||||
- [x] **Step 3: Apply approved baseline updates only after review**
|
||||
|
||||
If the rendered screenshots are coherent and approved, run:
|
||||
|
||||
```powershell
|
||||
node node_modules/playwright/cli.js test tests/visual/views.spec.ts --project=chromium --update-snapshots
|
||||
```
|
||||
|
||||
Expected: visual suite passes afterward.
|
||||
|
||||
**Completed 2026-07-08:** Re-ran the visual suite after Tasks 1-4 and reproduced the classified 14/14 stale-baseline failures on port 34136. Spot-checked the current Chat, Settings, Home, and Memory actual captures; the rendered surfaces were coherent and matched the approved Phase 1 direction. Updated the active `Visual-Regression---...` baseline family on port 34137, then reran without update mode on port 34138; `tests/visual/views.spec.ts` passed 14/14.
|
||||
|
||||
## Task 6: Route Evidence for Thin Phase 1 Judge Paths
|
||||
|
||||
**Files:**
|
||||
- Modify: `tests/e2e/user-journeys.spec.ts`
|
||||
- Modify: `docs/audits/2026-07-08-ux-route-scenario-manifest.md`
|
||||
- Modify: `docs/audits/2026-07-08-route-evidence-t11-analysis.md`
|
||||
- Modify: `docs/audits/2026-07-08-state-failure-t12-analysis.md`
|
||||
- Modify: `docs/audits/2026-07-08-five-persona-judge-scorecards.md`
|
||||
|
||||
- [x] **Step 1: Add smoke coverage for zero-hit and priority thin routes**
|
||||
|
||||
Add a user journey test that verifies `/benchmarks`, `/platform`, and `/payment-cancelled`:
|
||||
|
||||
```ts
|
||||
test('J-route-coverage: thin utility routes render or redirect clearly', async ({ page }) => {
|
||||
await gotoApp(page, '/benchmarks');
|
||||
await expect(page.locator('body')).toContainText(/benchmark|capability|score|memory/i);
|
||||
|
||||
await gotoApp(page, '/platform');
|
||||
await expect(page.locator('body')).toContainText(/platform|local|governance|memory|agent/i);
|
||||
|
||||
await gotoApp(page, '/payment-cancelled');
|
||||
await page.waitForURL(/\/settings\?tab=billing/, { timeout: 10_000 });
|
||||
await expect(page.locator('body')).toContainText(/billing|plan|team|solo|checkout/i);
|
||||
});
|
||||
```
|
||||
|
||||
Add a second user journey test that codifies the ad hoc thin-route smoke from the analysis packet:
|
||||
|
||||
```ts
|
||||
test('J-route-coverage: priority thin routes render meaningful shells', async ({ page }) => {
|
||||
const routeChecks: Array<[string, RegExp]> = [
|
||||
['/launcher', /tool launcher|optional prompt|detecting installed tools|launch/i],
|
||||
['/launcher?watch=1', /tool launcher|optional prompt|detecting installed tools|launch/i],
|
||||
['/waggle-dance', /waggle dance|signals|discovery|handoff/i],
|
||||
['/artifacts', /artifact|library|document|presentation/i],
|
||||
['/settings/profile', /who are you|identity|writing style|save/i],
|
||||
['/settings/timeline', /timeline|workspace|activity/i],
|
||||
['/payment-success', /checkout|paid|plans|nothing to confirm/i],
|
||||
['/automations', /automation|schedule|trigger|history|logs/i],
|
||||
['/mcps', /mcp hub|installed|catalog|custom/i],
|
||||
['/settings/usage', /usage|cost|tokens|budget|upgrade/i],
|
||||
['/files', /storage|files|workspace|local/i],
|
||||
];
|
||||
|
||||
for (const [route, bodyPattern] of routeChecks) {
|
||||
await gotoApp(page, route);
|
||||
await expect(page.locator('body')).toContainText(bodyPattern);
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
node node_modules/playwright/cli.js test tests/e2e/user-journeys.spec.ts --project=chromium --grep "thin utility routes"
|
||||
node node_modules/playwright/cli.js test tests/e2e/user-journeys.spec.ts --project=chromium --grep "priority thin routes"
|
||||
```
|
||||
|
||||
Expected: pass after route behavior is confirmed or copy is clarified. Current analysis evidence already proves these URLs render/redirect in ad hoc smokes, but this step still codifies that evidence and keeps T1 console errors, Launcher detect-in-flight noise, and Usage/Cost 403 noise visible.
|
||||
|
||||
- [x] **Step 2: Update manifest statuses**
|
||||
|
||||
In `docs/audits/2026-07-08-ux-route-scenario-manifest.md`, keep or update the rows for `/benchmarks`, `/platform`, `/payment-cancelled`, `/launcher`, `/launcher?watch=1`, `/waggle-dance`, `/artifacts`, `/settings/profile`, `/settings/timeline`, `/payment-success`, `/automations`, `/mcps`, `/settings/usage`, and `/files` with the codified route-smoke evidence owner. They already have ad hoc `Mixed` evidence from the analysis packet, but the implementation task must replace that with repeatable test evidence.
|
||||
|
||||
Also update `docs/audits/2026-07-08-route-evidence-t11-analysis.md` with the new passing command output and any remaining thin-route owners.
|
||||
|
||||
- [x] **Step 3: Update scorecard prerequisites**
|
||||
|
||||
In `docs/audits/2026-07-08-five-persona-judge-scorecards.md`, add the new route smoke test to the judge evidence packet under Team Admin and Engineer if it is part of their journey.
|
||||
|
||||
In `docs/audits/2026-07-08-state-failure-t12-analysis.md`, add only a short evidence note if the new route smoke also proves a state-bundle field such as payment-cancelled recovery, route-level Launcher state, or disclosure-tier route access.
|
||||
|
||||
**Completed 2026-07-08:** Added two codified route-coverage tests to `tests/e2e/user-journeys.spec.ts`: one for `/benchmarks`, `/platform`, and `/payment-cancelled` redirect recovery; one for priority thin routes (`/launcher`, `/launcher?watch=1`, `/waggle-dance`, `/artifacts`, `/settings/profile`, `/settings/timeline`, `/payment-success`, `/automations`, `/mcps`, `/settings/usage`, `/files`). Verification: `J-route-coverage` passed 2/2 on port 34139. Updated the route manifest, T11 evidence supplement, T12 state supplement, and Engineer/Team Admin scorecards to cite the codified smoke while keeping deeper workflow-state evidence open.
|
||||
|
||||
## Task 7: Phase 1 Verification Gate
|
||||
|
||||
**Files:**
|
||||
- No source change unless a gate fails.
|
||||
- Update: `docs/audits/2026-07-08-complete-ux-usage-audit.md`
|
||||
- Update: `docs/audits/2026-07-08-ux-route-scenario-manifest.md`
|
||||
- Update: `docs/audits/2026-07-08-five-persona-judge-scorecards.md`
|
||||
|
||||
- [x] **Step 1: Run static and unit verification**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
npm run typecheck:web
|
||||
npm run ux:contrast
|
||||
npm run ux:color-guard
|
||||
cd apps/web
|
||||
npx vitest run
|
||||
cd ../..
|
||||
npm run build
|
||||
```
|
||||
|
||||
Expected: all pass. Warnings are allowed only if documented and unrelated to Phase 1.
|
||||
|
||||
- [x] **Step 2: Run focused browser lane**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
$env:WAGGLE_E2E_PORT='3391'
|
||||
$env:WAGGLE_E2E_BASE_URL='http://127.0.0.1:3391'
|
||||
$env:WAGGLE_E2E_SKIP_LITELLM='1'
|
||||
node node_modules/playwright/cli.js test tests/e2e/full-product-audit.spec.ts tests/e2e/full-wiring-audit.spec.ts tests/e2e/phase-ab-verification.spec.ts tests/e2e/power-user-stress.spec.ts tests/e2e/user-journeys.spec.ts tests/visual/views.spec.ts --project=chromium --reporter=list
|
||||
```
|
||||
|
||||
Expected: pass, or visual baseline updates are explicitly reviewed and approved.
|
||||
|
||||
- [x] **Step 3: Capture manual screenshots**
|
||||
|
||||
Capture or inspect:
|
||||
|
||||
```text
|
||||
Desktop: first-run onboarding Welcome/Profile/Model/Template/First Task, /home, /settings, /marketplace, /mcps, /memory, /workspaces/:id/chat, /launcher
|
||||
Mobile 390 x 844: first-run onboarding Welcome/Profile, /home, /settings, /settings?tab=models, /settings?tab=billing, /settings/profile, /memory, /workspaces/:id/chat
|
||||
Overlays: Command Center, Workspace Switcher
|
||||
```
|
||||
|
||||
Expected: no clipped critical controls, no incoherent overlap, no horizontal overflow in required mobile surfaces, and the selected Mobile Executive overlay opens and closes without trapping focus or scroll. If Command Center still remains visible after Escape or touch-close, record it as a T10/T12 blocker or explicitly use Workspace Switcher as the scoped mobile overlay proof.
|
||||
|
||||
- [x] **Step 4: Update analysis artifacts**
|
||||
|
||||
Update the audit docs with current evidence:
|
||||
|
||||
```text
|
||||
P0-1 closed/open evidence
|
||||
P0-2 closed/open evidence
|
||||
P0-7 first-run mobile onboarding evidence
|
||||
P0-3 closed/open evidence
|
||||
P0-4 visual decision log
|
||||
P0-5/P0-6 shortcut and overlay evidence
|
||||
T11 route evidence status
|
||||
```
|
||||
|
||||
Expected: Phase 1 is either ready for a judge dry run or has a short list of remaining blockers.
|
||||
|
||||
**Completed 2026-07-08:** Static/unit verification passed: `npm run typecheck:web`, `npx tsc --noEmit --project packages/server/tsconfig.json`, `npm run ux:contrast`, `npm run ux:color-guard`, full `apps/web` Vitest (174 files / 1578 tests), and `npm run build`. Focused browser verification passed 10/10 on port `34146`. `tests/e2e/user-journeys.spec.ts` passed 16/16 on port `34149`. The full combined browser gate passed 156/156 on port `34150` across `full-product-audit`, `full-wiring-audit`, `phase-ab-verification`, `power-user-stress`, `user-journeys`, and `tests/visual/views.spec.ts`. Visual artifacts were reviewed during Task 5 before baseline updates; the combined gate revalidated the approved baselines. The audit, route manifest, and judge scorecards now record the Phase 1 post-fix evidence and remaining non-Phase-1 blockers.
|
||||
|
||||
## Self-Review Checklist
|
||||
|
||||
- Spec coverage: This plan covers Phase 1 items T1, T2, T3, T4, T5, and T11 from the audit, including first-run onboarding console and mobile reachability evidence. It intentionally defers T6-T10 and T12-T19 except where a Phase 1 test touches them.
|
||||
- Placeholder scan: Each task includes concrete test commands or code shapes for the intended change.
|
||||
- Type consistency: The plan uses existing route names, file paths, and current test names from the repo.
|
||||
- Scope check: The plan does not create new product surfaces. It fixes existing surfaces and evidence gates.
|
||||
|
||||
## Execution Choice
|
||||
|
||||
Plan complete and saved to `docs/superpowers/plans/2026-07-08-ux-phase-1-corrections.md`. Two execution options after user approval:
|
||||
|
||||
1. Subagent-Driven (recommended): dispatch a fresh subagent per task, review between tasks, fast iteration.
|
||||
2. Inline Execution: execute tasks in this session using executing-plans, batch execution with checkpoints.
|
||||
Reference in New Issue
Block a user