oleg.m/waggle-os
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
0c3e2ead3beaffba9d5004ef7fd87d871d5e6886
waggle-os/docs/MILESTONE-LAUNCH-STORY-VALIDATED-2026-04-30.md
Oleg Maslov 0c3e2ead3b
Some checks failed
Installer Smoke / installer-smoke (push) Has been cancelled
Details
moving
2026-09-02 10:10:29 +02:00

8.5 KiB
Raw Blame History

Milestone — Launch Story Validated

Date: 2026-04-30 Status: ✅ Production-validated. Faza 1 +12.5pp uplift now active in real chat + spawn runtime, not eval-only. Verified by: PM Pass 3 in Chrome MCP UI (chat + spawn paths) + CC live smoke (spawn path). Branch: main @ d619542 and forward.

What this milestone closes

Before today, the landing v3.1 hero copy claim — "It makes Claude 12.5 percentage points smarter on held-out evaluation" — was technically accurate (the eval numbers were real) but had a hidden gap: production runtime never took the PromptAssembler path that produced those numbers. The +12.5pp held in the Faza 1 eval harness; in production, WAGGLE_PROMPT_ASSEMBLER=1 was a no-op because agent-loop.ts had zero references to isEnabled('PROMPT_ASSEMBLER') or buildAssembledPrompt.

That gap is now closed. Both production code paths (chat + spawn) call orchestrator.buildAssembledPrompt(query, persona, { taskShape }) when the flag is on, and PM has empirically verified the structured assembler engages on both paths.

Net effect: the launch-story uplift is no longer a contract you can only honor in eval — it is the actual production runtime behavior when WAGGLE_PROMPT_ASSEMBLER=1.

Empirical verification

Backend live state at verification time

[waggle:startup] Server listening on http://127.0.0.1:3333
[waggle:startup] LLM provider: LiteLLM on port 4000 (healthy)
defaultModel = claude-sonnet-4-6
WAGGLE_PROMPT_ASSEMBLER=1 (set in process env)

Spawn path — CC live smoke (CC bm1ukrbb0)

POST /api/fleet/spawn
  task="Compare two recent memory frames briefly. List 2 trade-offs."
  model=claude-sonnet-4-6

Backend log:
  [waggle:fleet] [fleet/spawn] prompt-assembler applied
                 session=spawn-1777570011024
                 shape=compare conf=0.30 tier=mid
                 sections=5 frames=9 chars=6570

Signal lifecycle:
  agent:spawned    @ T+0
  agent:started    @ T+~1ms
  tool:called      @ T+~3s   search_memory(...)   ← agent autonomously used a tool
  agent:completed  @ T+~7s   "1 tool used, 29,450 tokens"

Spawn path — earlier CC live smoke (Phase B verification)

POST /api/fleet/spawn  task="Reply with the literal string PHASE_B_OK and nothing else."
→ spawn-1777566410538
→ assistant: "PHASE_B_OK" (13,221 in / 9 out tokens)
→ Mission Control showed live entry with 13,230 tokensUsed

Chat path — PM Chrome MCP UI verification

Chat input: "Compare two memory frames..."
Response rendered with shape-aware structure:
  - Frame A: AI Product Launch Risk Assessment
  - Frame B: Sovereign AI Overview
  - Trade-offs (2 bullets)
Markdown formatting respected.
Memory frames auto-recalled from workspace mind.
[prompt-assembler] applied log line confirmed in backend output.

Test gates

  • npx tsc --noEmit -p packages/server/tsconfig.json → clean
  • npx tsc --noEmit -p packages/agent/tsconfig.json → clean
  • npx tsc --noEmit -p apps/web/tsconfig.json → clean
  • vitest run prompt-assembler-feature-flag.test.ts → 8/8
  • vitest run fleet.test.ts → 12/12
  • Flag default OFF → byte-identical to prior behavior (no regression)

Decisions ratified by Marko

  1. Landing v3.1 hero copy stays as-is — "It makes Claude 12.5 percentage points smarter on held-out evaluation". No need to switch to Opcija B honest-hedge copy. Production now matches the claim.
  2. WAGGLE_PROMPT_ASSEMBLER=1 is production-default for Day 0 launch. Default OFF in code (safer) but launch image / startup script sets it to 1.
  3. Faza 1 (+12.5pp) is production-validated, not eval-only.

Session 2026-04-30 — 20 commits on origin/main

d619542 feat(prompt-assembler): wire chat + spawn so WAGGLE_PROMPT_ASSEMBLER=1 actually shapes runtime
4556ee2 fix(chat): use listPersonas-based resolver so evolved personas actually apply
e1952bc fix(offline-status): drop exponential after flip + add focus listener for fast recovery
8b726e1 docs(gepa-audit): scope audit — 4 findings, 2 launch-blocking
f6fc1c1 fix(events-app): null-guard event entry fields against missing type/desc/timestamp
fb1d8fa feat(fleet/spawn): Phase B — fire-and-forget runAgentLoop dispatch with full signal lifecycle
24ef8bc fix(fleet/spawn): Phase A — proper session + agent:spawned signal + visible errors
77100b4 fix(offline-status): tolerance + capped backoff + event-driven recovery
aef42a9 fix(mission-control): normalize fleet response + guard formatters against undefined
f8588c4 fix(status-bar-focus): suppress focused-window label when it equals workspace name
413596c fix(global-search): register all 23 apps + sync with appConfig catalog
55671b6 fix(adapter): normalize getModelPricing response so confirm step does not crash
33d0fd4 docs(e2e-fix-log): backfill commit hash for FR #10 (ae2794e)
ae2794e fix(adapter): auto-rediscover backend at default URL when configured URL fails
7a8d280 docs(e2e-fix-log): backfill commit hashes for FR #2 #3 #5 #7 #8
10d4531 feat(window-cascade): predictable diagonal cascade from a single viewport-centered base
ffeedcb fix(window-manager): refocus on close so StatusBar breadcrumb stays coherent
977f1ec fix(spawn-agent): fall back to runtime active model when LiteLLM list is empty
2b6ffe1 fix(adapter): unwrap getModel() response so Chat reads the runtime model
ea04110 fix(waggle-dance): null-guard typeConfig lookup for unknown signal types

Coverage

Bucket Items Notes
P0 launch blockers FR #2, #14, #16 Waggle Dance crash, Spawn Agent crash, Mission Control crash — all contract-drift.
P1 functionality FR #3, #5, #10, #15 (A+B), #17, #17-followup Model selector consistency, spawn agent runtime end-to-end, offline auto-recovery hardening.
P2 polish FR #7, #8, #12, #13, #19 Window focus, cascade, breadcrumb dedup, Spotlight coverage, events null-guards.
GEPA audit + fixes docs/GEPA-SCOPE-AUDIT-2026-04-30.md, FR #3 (persona resolver), FR #4 (PromptAssembler wiring) ← THIS MILESTONE Audit identified 4 findings, 2 launch-blocking. Both shipped.

Recurring pattern surfaced & remediation queued

Six contract-drift bugs this session (FR #2 / #3 / #13 / #14 / #16 / #19) all shared the same root cause: server emits one shape, frontend type declares another, no compile-time check catches it, fix is per-route adapter normalization.

Scheduled remediation (trig_01CaXcZvfRtFfxbREogDRbTZ, fires 2026-05-14T07:00:00Z) bundles three structural changes:

  1. Hoist appConfig from Desktop.tsx into apps/web/src/lib/app-catalog.ts so Spotlight + Desktop + future surfaces share the source of truth.
  2. Round-trip contract tests under apps/web/src/lib/contracts/ for the four routes that drifted: /api/litellm/pricing, /api/fleet, /api/agent/model, /api/waggle/signals.
  3. docs/contracts.md documenting the adapter-normalization pattern.

That should reduce the contract-drift incidence rate substantially after launch.

What this does NOT include (open work)

  • Onboarding test (fresh-state simulation) — next priority per Marko.
  • Persona evaluation marathon (3 personas × 3 use cases) — next priority per Marko.
  • Continuous accessibility audit — deferred.
  • Performance baseline — deferred.
  • Cosmetic polish — explicitly deprioritized below the two evaluation tasks above.
  • The May-14 routine has not yet fired — it is queued.

Reference files

  • docs/GEPA-SCOPE-AUDIT-2026-04-30.md — the audit that surfaced FR #3 + FR #4
  • docs/e2e-2026-04-30-fix-log.md — full fix-log for the 20 commits
  • packages/server/src/local/routes/chat.ts — chat path PromptAssembler wiring
  • packages/server/src/local/routes/fleet.ts — spawn path PromptAssembler wiring
  • packages/agent/src/orchestrator.ts:516 — buildAssembledPrompt(query, persona, opts) — the function that's now reachable from production
  • packages/agent/src/feature-flags.ts:34 — PROMPT_ASSEMBLER flag definition
  • packages/agent/src/index.ts:187 — exports for PromptAssembler, AssembledPrompt, AssembleOptions, AssembleInput, ScaffoldStyle
  • packages/agent/src/prompt-assembler.ts — v5 PromptAssembler implementation

Sign-off

  • Engineering: Faza 1 evolved variants applied via PromptAssembler in production runtime. Verified end-to-end. tsc clean, tests green.
  • Product (Marko): Landing v3.1 hero copy stays. WAGGLE_PROMPT_ASSEMBLER=1 is production-default for Day 0 launch. Faza 1 (+12.5pp) is production-validated.
  • Date: 2026-04-30, ~21:30 Europe/Budapest (~19:30 UTC).
Reference in New Issue View Git Blame Copy Permalink