# 05 - TEST & COVERAGE REPORT

> **Historical snapshot — superseded.** This report records evidence as of
> 2026-03-20 and is not current launch authority. See
> [`09-LAUNCH_RECOMMENDATION.md`](./09-LAUNCH_RECOMMENDATION.md) for the current
> Windows-first Solo scope, external gates, and recommendation.

**Date:** 2026-03-20
**Auditor:** Claude Opus 4.6 (automated, read-only)
**Scope:** All packages, integration tests, E2E tests, visual regression tests

---

## 1. Test Distribution

### Summary Totals

| Metric | Count |
|--------|-------|
| Total test files | 278 |
| Total test cases (it/test) | ~3,762 (packages + root tests) |
| App test cases | 188 |
| Grand total test cases | ~3,950 |

### Per-Package Breakdown

| Package | Source Files | Test Files | Test Cases | Ratio (tests/src) |
|---------|-------------|------------|------------|-------------------|
| `@waggle/agent` | 95 | 95 | 1,272 | 1.00 |
| `@waggle/server` | 92 | 86 | 904 | 0.93 |
| `@waggle/ui` | 102 | 27 | 809 | 0.26 |
| `@waggle/core` | 27 | 24 | 330 | 0.89 |
| `@waggle/marketplace` | 12 | 6 | 160 | 0.50 |
| `@waggle/sdk` | 8 | 5 | 66 | 0.63 |
| `@waggle/cli` | 7 | 7 | 52 | 1.00 |
| `@waggle/worker` | 9 | 4 | 38 | 0.44 |
| `@waggle/weaver` | 3 | 3 | 30 | 1.00 |
| `@waggle/waggle-dance` | 4 | 3 | 25 | 0.75 |
| `@waggle/optimizer` | 3 | 1 | 17 | 0.33 |
| `@waggle/launcher` | 1 | 1 | 13 | 1.00 |
| `@waggle/shared` | 4 | 1 | 9 | 0.25 |
| `@waggle/admin-web` | 10 | 1 | 5 | 0.10 |
| `app` (Tauri desktop) | 66 | 9 | 188 | 0.14 |
| `sidecar` | 6 | 3 | ~20 | 0.50 |
| `tests/` (root integration) | - | 5 | 32 | - |

### Observations

- **agent** and **server** have excellent file-level coverage (1:1 or near it).
- **ui** has low file-level ratio (0.26) -- 102 source files but only 27 test files. Tests exist for utility functions, hooks, and exports but most React component rendering is deferred to "the desktop app's E2E suite."
- **admin-web** has minimal coverage (1 test file / 10 source files, 5 assertions total).
- **app** (Tauri desktop) has 66 source files but only 9 test files. The E2E tests cover startup, chat, workspaces, and regression scenarios via Fastify `inject()` (not browser automation).
- **shared** has only 1 test file for 4 source files.

---

## 2. Test Quality Assessment

### File 1: `packages/core/tests/vault.test.ts` (11 tests)

| Criterion | Rating | Notes |
|-----------|--------|-------|
| Behavior vs implementation | **Behavior** | Tests CRUD operations, encryption verification, migration -- all user-facing behaviors |
| Mock quality | **Excellent** | No mocks needed -- uses real VaultStore with temp directories |
| Edge cases | **Good** | Covers nonexistent keys, idempotent migration, key file reuse, plaintext leak check |
| Assertions | **Meaningful** | Verifies actual decrypted values, file-on-disk encryption format, metadata propagation |
| Test isolation | **Excellent** | Each test creates its own temp dir; afterEach cleans up |
| **Missing** | Wrong password/corrupted vault file scenarios not tested (see Section 3) |

### File 2: `packages/agent/tests/agent-loop.test.ts` (8 tests)

| Criterion | Rating | Notes |
|-----------|--------|-------|
| Behavior vs implementation | **Behavior** | Tests the full agent loop: text response, tool execution, maxTurns, plugin tools, capability routing |
| Mock quality | **Good** | `mockFetch` returns realistic OpenAI-format responses with proper structure |
| Edge cases | **Good** | Tests maxTurns safety, missing tool routing, plugin+base tool merging |
| Assertions | **Meaningful** | Verifies content, tool usage, token counting, fetch call structure, message threading |
| Test isolation | **Good** | Each test creates fresh mocks |
| **Missing** | No tests for: fetch failures/timeouts, rate limiting, malformed LLM responses, concurrent requests |

### File 3: `packages/server/tests/ws/gateway.test.ts` (10 tests)

| Criterion | Rating | Notes |
|-----------|--------|-------|
| Behavior vs implementation | **Behavior** | Tests connection tracking, broadcast, exclusion, send-to-user, closed socket handling |
| Mock quality | **Adequate** | Uses `{ readyState, OPEN, send: vi.fn() }` -- minimal but sufficient for unit tests |
| Edge cases | **Good** | Nonexistent teams, nonexistent users, closed sockets, last-user-leaves cleanup |
| Assertions | **Meaningful** | Checks exact call counts, JSON payload structure, team count |
| Test isolation | **Excellent** | Fresh ConnectionManager per test |
| **Missing** | No reconnection logic tests, no concurrent broadcast tests, no WebSocket error event handling |

### File 4: `packages/agent/tests/connector-sdk.test.ts` (~25 tests)

| Criterion | Rating | Notes |
|-----------|--------|-------|
| Behavior vs implementation | **Behavior** | Tests interface compliance, definition mapping, tool generation, registry CRUD, audit logging, error handling |
| Mock quality | **Good** | MockConnector extends BaseConnector with realistic actions and risk levels; MockVault has proper credential resolution |
| Edge cases | **Good** | API timeout errors, duplicate registration, disconnected connectors, missing credentials, expired tokens |
| Assertions | **Meaningful** | Checks tool name formats, risk level propagation, audit log payloads, error messages |
| Test isolation | **Good** | Fresh registry per test |
| **Missing** | No concurrent connector execution, no partial failure in multi-connector scenarios |

### File 5: `packages/ui/tests/components/chat.test.ts` (~35 tests)

| Criterion | Rating | Notes |
|-----------|--------|-------|
| Behavior vs implementation | **Mixed** | Tests utility functions (getToolStatusColor, formatDuration, processStreamEvent) and verifies exports exist -- but no actual React component rendering |
| Mock quality | **N/A** | No mocks needed for utility function tests |
| Edge cases | **Good** | Covers all tool statuses (done, pending, denied, error, running), legacy fallback without status field |
| Assertions | **Meaningful** | Return value checks for utility functions |
| Test isolation | **Good** | Pure functions, stateless |
| **Missing** | No rendering tests -- relies entirely on desktop E2E. No tests for user interaction flows, form submission, error display, loading states |

---

## 3. Untested Critical Paths

### CRITICAL Severity

| Gap | Description | Risk |
|-----|-------------|------|
| **SSE stream interruption handling** | Zero tests for SSE connection drops, reconnection, or partial event parsing. `useNotifications` hook has reconnection logic in source but no test covers the error/reconnect path. | Users may lose agent output mid-response with no recovery |
| **Vault corruption/wrong key** | Vault tests verify happy path only. No test for: corrupted vault.json, missing/wrong .vault-key file, concurrent write conflicts. | Silent data loss or unrecoverable state |
| **Sub-agent cleanup on failure** | `subagent-tools.test.ts` tests a single error case (LLM connection failed returns error string) but does not verify resource cleanup (open connections, temp files, memory entries) after sub-agent crash. | Resource leaks under failure |
| **WebSocket reconnection logic** | Source code in `useNotifications.ts` and `useSubAgentStatus.ts` has reconnection logic, but no test verifies reconnection actually re-establishes state. | Silent presence/notification failures after network blips |

### HIGH Severity

| Gap | Description | Risk |
|-----|-------------|------|
| **Server route: fleet** | `packages/server/src/local/routes/fleet.ts` -- Mission Control fleet API has zero test coverage | Broken fleet status could go undetected |
| **Server route: import** | `packages/server/src/local/routes/import.ts` -- memory import (ChatGPT/Claude parsers) route has zero direct test coverage. Core `processImport` is tested in `core/tests/memory-import.test.ts` but the HTTP route layer is not. | Import failures at route level undetected |
| **Server route: anthropic-proxy** | Built-in Anthropic proxy (`/v1/chat/completions`) has no dedicated test file. Only tangentially referenced in `health.test.ts`. | Proxy translation bugs (OpenAI -> Anthropic format) undetected |
| **Worker handlers: group-handler.ts, task-handler.ts** | No test files for these handlers. Only `chat-handler.ts` and `waggle-dispatch.ts` are tested. | Background task execution failures undetected |
| **Marketplace: installer.ts, security.ts** | No dedicated test files. `security.ts` (SecurityGate) is partially covered via `cisco-scanner.test.ts` but the installer module has no direct tests. | Broken install flow or security bypass undetected |
| **Admin-web** | 10 source files, 1 test file, 5 assertions total. All 7 page components effectively untested. | Admin dashboard regressions undetected |

### MEDIUM Severity

| Gap | Description | Risk |
|-----|-------------|------|
| **Agent loop: LLM error responses** | No test for HTTP 429 (rate limit), 500 (server error), malformed JSON, or network timeout from the LLM provider | Agent may hang or crash on provider issues |
| **Memory corruption recovery** | `backup-restore.test.ts` exists but no test verifies recovery from a corrupted `.mind` SQLite file (e.g., truncated WAL, invalid schema version) | Unrecoverable personal data loss |
| **Concurrent vault access** | No test for two processes reading/writing vault.json simultaneously | Race condition could corrupt secrets |
| **Sidecar: agent-session.ts, weaver-scheduler.ts** | No tests for these 2 of 6 sidecar source files | Desktop agent session bugs undetected |
| **UI component rendering** | 102 UI source files but zero React rendering tests. All chat, memory, cockpit, settings components tested only via export-existence checks and utility functions. | Visual/interaction regressions undetected without manual testing |
| **Notification routes** | `packages/server/src/local/routes/notifications.ts` has no dedicated route test. `local/notifications.test.ts` tests the notification system but not the SSE streaming endpoint. | Notification delivery failures undetected |
| **Optimizer signatures.ts** | No test coverage for prompt optimization signatures module | Broken GEPA optimization undetected |

---

## 4. E2E Tests

### Vitest-Based E2E (app/tests/e2e/)

| File | Scenarios Covered |
|------|-------------------|
| `startup.test.ts` | Health check, onboarding wizard, settings persistence across restart |
| `chat.test.ts` | Chat message -> SSE token stream, tool events in stream, approval gate (eventBus blocks/resumes) |
| `workspaces.test.ts` | Workspace CRUD, workspace switching, session isolation |
| `regression.test.ts` | Regression scenarios (specifics not inspected) |

All use Fastify `inject()` -- **no real browser automation**. These are server-level integration tests, not true E2E tests.

### Agent-Level E2E (packages/agent/tests/e2e/)

| File | Scenarios Covered |
|------|-------------------|
| `solo-scenarios.test.ts` | S1: Research report (web_search -> save -> docx), S2: Code review, S3: Project planning, S4: Memory continuity |
| `connector-swarm-scenarios.test.ts` | C1: GitHub issue creation, C2: Email with approval gate, Swarm parallel/sequential/coordinator execution |

These use mock tool execution (no real LLM) -- they verify tool chain orchestration, not real LLM interaction.

### Playwright Visual Regression (tests/visual/)

| File | Coverage |
|------|----------|
| `views.spec.ts` | Screenshot baselines for 7 views (Chat, Memory, Events, Capabilities, Cockpit, Mission Control, Settings) in dark + light mode = 14 baselines |

Requires running server. Uses 0.3% pixel diff threshold. **No user interaction tests** -- screenshots only.

### What is Missing from E2E

- **No browser-automated user journey tests** (typing in chat, clicking buttons, navigating between views)
- **No onboarding flow E2E** (full wizard completion in browser)
- **No multi-workspace switching E2E** in browser
- **No connector setup/teardown flow** E2E
- **No marketplace browse/install flow** E2E
- **No approval gate click-through** E2E in browser

---

## 5. Kill List Coverage Matrix

| # | Kill List Item | Test Coverage | Assessment |
|---|---------------|---------------|------------|
| 1 | **Workspace restart / instant catch-up** | `core/tests/mind/awareness.test.ts`, `core/tests/mind/frames.test.ts`, `server/tests/routes/workspace-context.test.ts`, `server/tests/routes/context-injection.test.ts`, `server/tests/routes/session-state-extraction.test.ts` | **GOOD** -- Context reconstruction, frame retrieval, and session state extraction all tested. Server-side catch-up endpoint tested. No E2E test of the full "open workspace -> see context" flow. |
| 2 | **Draft from accumulated context** | `agent/tests/document-tools.test.ts` (docx generation), `agent/tests/e2e/solo-scenarios.test.ts` (research -> docx chain), `agent/tests/workflow-composer.test.ts` | **ADEQUATE** -- Document generation tested with markdown parsing, tables, lists, subdirectories. Workflow chains tested. No test verifying draft quality uses accumulated workspace memory. |
| 3 | **Decision compression / next-step thinking** | `weaver/tests/consolidation.test.ts`, `weaver/tests/consolidation-enhanced.test.ts`, `agent/tests/orchestrator.test.ts` | **PARTIAL** -- Memory consolidation (the mechanism behind decision compression) is tested. No dedicated test verifying "what was decided about X?" returns compressed decisions. |
| 4 | **Research and synthesis in context** | `agent/tests/e2e/solo-scenarios.test.ts` (S1: research report), `agent/tests/combined-retrieval.test.ts`, `agent/tests/search-memory-combined.test.ts`, `agent/tests/web-search-cache.test.ts` | **GOOD** -- Combined retrieval (memory + KVARK), web search, and research-to-document chains all tested. |
| 5 | **Ongoing project memory for solo operators** | `core/tests/mind/` (10 test files), `core/tests/multi-mind.test.ts`, `agent/tests/search-memory-combined.test.ts`, `server/tests/workspace-api.test.ts` | **STRONG** -- Most thoroughly tested area. Frame storage, FTS5 search, temporal knowledge, identity, awareness, knowledge graph -- all covered with 330+ test cases in core alone. |
| 6 | **Capability discovery and installation** | `agent/tests/capability-acquisition.test.ts`, `agent/tests/capability-marketplace.test.ts`, `server/tests/routes/acquisition-integration.test.ts`, `server/tests/local/marketplace*.test.ts` (5 files), `ui/tests/components/install-center.test.ts`, `core/tests/mind/install-audit.test.ts` | **STRONG** -- Discovery, acquisition, trust model, security scanning, approval gate, install center UI, audit trail -- all tested. 16+ test files cover this flow. |
| 7 | **External action execution (connectors)** | `agent/tests/connector-sdk.test.ts`, `agent/tests/connector-routing.test.ts`, `agent/tests/connectors/` (8 files for GitHub, Slack, Jira, email, etc.), `agent/tests/e2e/connector-swarm-scenarios.test.ts`, `server/tests/local/connectors.test.ts`, `server/tests/local/connector-registry-integration.test.ts` | **GOOD** -- Connector SDK, registry, individual connector types, tool generation, audit logging, error handling all tested. No test for real API calls (all mocked). Missing: connector credential refresh, expired token re-auth. |
| 8 | **Multi-agent workflows (swarm)** | `waggle-dance/tests/` (3 files: protocol, dispatcher, integration), `agent/tests/subagent-tools.test.ts`, `agent/tests/subagent-orchestrator.test.ts`, `agent/tests/e2e/connector-swarm-scenarios.test.ts`, `worker/tests/execution/strategies.test.ts`, `server/tests/daemons/hive-mind.test.ts` | **ADEQUATE** -- Protocol, dispatching, parallel/sequential/coordinator execution strategies, hive-mind daemon all tested. Missing: swarm failure recovery (partial agent failures), swarm cancellation, resource limits under concurrent swarm execution. |

### Kill List Summary

| Rating | Items |
|--------|-------|
| STRONG (4+/5) | #5 (project memory), #6 (capability discovery) |
| GOOD (3/5) | #1 (catch-up), #4 (research), #7 (connectors) |
| ADEQUATE (2.5/5) | #2 (drafting), #3 (decision compression), #8 (swarm) |
| WEAK (<2/5) | None |

---

## 6. Overall Test Health Assessment

### Strengths

1. **Exceptional volume**: ~3,950 test cases across 278 files is substantial for a project of this size. The claimed 3,069 tests all passing is credible based on the file counts.

2. **Core and agent packages are well-covered**: The two most critical packages (core: memory/mind, agent: tools/loop) have near 1:1 file coverage and deep behavioral tests.

3. **Kill List items all have coverage**: Every V1 must-win use case has at least adequate test coverage. No kill list item is completely untested.

4. **Good test isolation**: Tests consistently use temp directories, fresh instances, and proper cleanup. No shared mutable state between tests.

5. **Behavior-focused testing**: The majority of tests verify user-facing behaviors rather than implementation details. Mocks are realistic (proper response structures, not trivial stubs).

6. **Performance benchmarks exist**: Cold start, FTS5 search, batch writes, and session load times are all benchmarked with threshold assertions.

7. **Security paths are tested**: Marketplace security scanning (Cisco scanner), vault encryption, injection scanner, trust model, approval gates, permission checks -- all covered.

### Weaknesses

1. **No real browser E2E tests**: All "E2E" tests use Fastify `inject()` or mock tool execution. The Playwright tests are screenshot-only (visual regression) with no user interaction. There is zero coverage of actual user flows in a real browser.

2. **UI component rendering untested**: 102 UI source files have zero React rendering tests. The test file for `chat.test.ts` explicitly states "no jsdom/React Testing Library." This means any rendering regression (broken layout, missing props, conditional rendering bugs) goes undetected.

3. **Error/failure paths underrepresented**: Happy paths are well-tested, but failure scenarios are sparse. SSE interruption, LLM provider errors, vault corruption, concurrent access, WebSocket reconnection -- these critical failure modes have minimal or zero coverage.

4. **Admin-web is effectively untested**: 5 assertions for 10 source files. The admin dashboard could be completely broken and tests would still pass.

5. **Several server routes have zero test files**: fleet, import, anthropic-proxy, and notifications routes lack dedicated tests.

6. **Worker handlers partially untested**: `group-handler.ts` and `task-handler.ts` have no tests -- these handle background task execution.

### Risk Rating

| Category | Rating |
|----------|--------|
| Unit test coverage | **B+** (strong in core/agent/server, weak in ui/admin-web/shared) |
| Integration test coverage | **B** (good server integration, missing cross-package integration) |
| E2E test coverage | **D+** (exists but no real browser automation; inject-only) |
| Error path coverage | **C-** (happy paths strong, failure/edge paths sparse) |
| Kill List alignment | **B+** (all items covered, some with depth gaps) |
| **Overall** | **B-** |

### Priority Recommendations

1. **P0**: Add browser-automated E2E tests for the daily-use loop (open workspace -> send message -> see response -> navigate views). Even 5 real Playwright interaction tests would dramatically improve confidence.

2. **P0**: Add failure-path tests for SSE stream interruption and LLM provider errors -- these affect every user session.

3. **P1**: Add vault corruption/recovery tests (corrupted JSON, missing key file, concurrent writes).

4. **P1**: Add dedicated tests for untested server routes (fleet, import, anthropic-proxy).

5. **P1**: Add at least basic React rendering tests for critical UI components (ChatArea, ApprovalGate, WorkspaceHome) using jsdom or React Testing Library.

6. **P2**: Add worker handler tests for group-handler.ts and task-handler.ts.

7. **P2**: Increase admin-web test coverage from 5 assertions to meaningful page-level tests.

8. **P2**: Add sub-agent resource cleanup verification tests.