moving

judging/round3/judge-1-novice.md

@@ -0,0 +1,145 @@
+# Round 3 — Judge 1: The Complete Novice
+
+## Persona
+
+I have never used anything like this. My phone is for messages, my laptop is for email and the
+web. I do not know what an "agent," an "MCP," a "workspace," or an "API key" is. I judged only
+what I could see and feel, and I judged my first-session path against the novice surface
+(the six-item sidebar in `14-novice-simple-dock.png`), treating the ~20-item sidebar screens as
+the place I might grow into. I verified the claimed plain-language hover descriptions exist in
+`apps/web/src/lib/dock-tiers.ts` (they do, and they are genuinely plain).
+
+## Scores
+
+| # | Criterion | Score (1–5) |
+|---|---|---|
+| 1 | First-session clarity | **4** |
+| 2 | "It knows me" feeling | **4** |
+| 3 | Visible agent growth | **2** |
+| 4 | Desire to return | **4** |
+| 5 | Absence of friction | **2** |
+
+## Per-criterion reasoning
+
+### 1. First-session clarity — 4
+
+The onboarding is honestly good for someone like me. Three steps, a progress bar, "Skip setup"
+always visible, and the privacy line ("Your memory and data stay on your device") in words I
+understand. Step 1 asks things I can answer (name, what kind of work, "What do you want Waggle
+to help with?") and the live preview — "Good evening, Marko — your work will be remembered
+here" — instantly shows me what my answers buy me. Step 2's per-provider how-tos ("In ChatGPT:
+Settings → Data controls → Export data") are exactly the hand-holding I need, and the
+auto-detected "Claude Code detected — Found 425 items / Import my history" button is one-click
+magic for people who have it. The six-item sidebar is calm and mostly self-explanatory.
+
+Why not 5: vocabulary leaks through at the worst moments. "Welcome to the Hive" and "YOUR AI
+OPERATING SYSTEM" tell me nothing (the small subtitle does all the work). Step 3 explains
+workspaces as "its own brain — memory, files, and agents stay isolated" — that's the first time
+the word "agents" appears, undefined, plus "isolated," a word I'd never use. And two of my six
+sidebar items are technical: "Vault" and "Spawn Agent" (complaints #2, #3 below).
+
+### 2. "It knows me" feeling — 4
+
+This is the app's strongest muscle. The returning-user panel (`01-home-welcome-back.png`)
+greets me by name, tells me "23 memories · 271 people, projects & things it knows · 2 awaiting
+your OK," and then — the genuinely visceral part — quotes my own working style back at me under
+"I REMEMBER": "I always work with a draft → critique → rewrite loop. The critique pass is the
+most important…" That is *me*, in my words. The Home header "You've been away 10 days, Marko.
+Here's what happened" plus "YOU WERE WORKING ON" cards with Continue buttons make the
+left-off-here promise concrete, and "Up next: I'll check in on quiet projects at Jun 15" makes
+it feel like it will keep knowing me.
+
+Why not 5: the showcase list undercuts itself. The third "I REMEMBER" item is "Session
+(2026-04-30): What is sovereign AI — 4 messages" — a machine log label with a raw date sitting
+beside two beautifully human memories (complaint #7). And the numbers don't reconcile if I get
+curious: 23 memories in the header, "11 memories across 16 sessions" on the workspace card, and
+a Memory screen showing a single memory card (complaint #6). It feels slightly inflated.
+
+### 3. Visible agent growth — 2
+
+The promise is "it learns my workflows and upgrades its own skills." What I can actually *see*
+is almost nothing — and on my novice sidebar, literally nothing. The Skills Hub, Agent Center,
+and Evolution screens are not in my six-item tier, so my only growth signals are a future-tense
+promise ("I'll suggest a new skill for you at Jun 17") and "5 automations completed" overnight —
+which, on inspection, are mostly internal plumbing like "Index reconciliation" (complaint #10).
+On the power surfaces the story isn't better: the Evolution screen — the showcase for
+"improves itself" — is empty ("Select a run to review") and explains itself with "baseline vs
+winner, which gates fired" (complaint #4). The one skill the assistant actually built itself is
+marked only by a tiny "agent · review" badge that I would never decode as "I built this for you"
+(complaint #5). The left-panel copy "Your agent improves itself here… nothing changes without
+you" is lovely and reassuring — but it's a caption on an empty room. I *read about* growth;
+I never *saw* it.
+
+### 4. Desire to return — 4
+
+The return loop is well designed. "You've been away 10 days… here's what happened" reframes my
+absence as accumulated value. The Overnight panel (10 memories consolidated, 5 automations
+completed) says work happened while I slept. "Up next" gives me two concrete dated reasons to
+come back (Jun 15 check-in, Jun 17 new-skill suggestion). Quick capture ("Jot a note to
+remember…") invites me to deposit things, which is how habits form. I genuinely wanted to click
+"Continue" on the Writer demo card.
+
+Why not 5: the impressive-stats panel shows "0 Artifacts created" — a deflating zero in
+jargon I don't know ("artifacts"? "consolidated"?) right where the app is trying to brag
+(complaint #11). And one of my two "you were working on" cards is "Default Workspace — 57d ago,"
+which feels stale rather than alive.
+
+### 5. Absence of friction — 2
+
+Onboarding and Home are smooth — skippable steps, sensible defaults, one-click import. But the
+single worst moment in the entire evidence set sits exactly where the magic is promised: the
+"resuming work" screen (`07-workspace-resume.png`). My message asks the assistant to run the
+critique pass, and its reply is, verbatim, a raw error dump in the chat bubble:
+
+> `[spawn failed] LLM error (404): {"error":{"message":"Anthropic API error: {"type":"error","error": {"type":"not_found_error","message":"model: auto"},"request_id":"req_011CbyyFYCakcsK8rNTTEok1"}}`
+
+"Spawn failed." A wall of braces. No "try again" button, no plain-language explanation, no
+recovery path I can see. As a novice this reads as "the app is broken and it's probably my
+fault," at the precise moment I was promised "pick up where you left off." Chat is core to my
+novice tier, so this is my path, not the power user's (complaint #1). Add the developer
+concepts pushed into my six-item world ("Vault — your API keys and secrets," "Spawn Agent") and
+friction earns a 2 despite the otherwise polished flow.
+
+## Numbered complaints (concrete, actionable)
+
+1. **Raw JSON error as the assistant's chat reply** (`07-workspace-resume.png`, Chat pane). The
+   agent answers with `[spawn failed] LLM error (404): {"error":…"not_found_error"…"model: auto"…}`.
+   Replace with a human card ("I couldn't connect just now — tap to retry"), put the raw error
+   behind a "Show details" disclosure, and never show `request_id` JSON in a chat bubble.
+2. **"Spawn Agent" in the novice sidebar** (`14-novice-simple-dock.png`, bottom of sidebar).
+   Both words are jargon to a first-timer. Rename for the simple tier (e.g., "New assistant").
+3. **"Vault" occupies one of six novice slots** (`14-novice-simple-dock.png`; tooltip per
+   `dock-tiers.ts`: "Your API keys and secrets, stored locally"). A novice has no API keys.
+   Worse: the simple dock has **no Memory entry** (confirmed in `dock-tiers.ts` lines 122–127),
+   so the tier built for novices hides the app's #1 superpower. Swap Vault for Memory.
+4. **Evolution screen is an empty room with engineer copy** (`13-memory-evolution.png`).
+   "Select a run to review… baseline vs winner, which gates fired." Seed one example run or hide
+   the tab until a proposal exists; rewrite "gates fired" in plain words.
+5. **Self-built skill is invisible as an achievement** (`04b-skills-agent-badge.png`,
+   presentation-design row). The only marker is a small "agent · review" badge. Add an explicit
+   callout: "Waggle built this skill from your workflow — review and approve it."
+6. **Memory counts don't reconcile** (`01-home-welcome-back.png` "23 memories · 271 people…"
+   vs workspace card "11 memories across 16 sessions" vs one visible card in
+   `03-memory-center.png`). Make the headline number match what clicking through reveals.
+7. **Machine log entry in the "I REMEMBER" showcase** (`01-home-welcome-back.png`, third item:
+   "Session (2026-04-30): What is sovereign AI — 4 messages"). Filter session-log labels out of
+   the human-memory list, or rephrase them ("We talked about sovereign AI in April").
+8. **"Welcome to the Hive" / "YOUR AI OPERATING SYSTEM"** (`08-onboarding-welcome.png`) — the
+   headline says nothing to a novice; the subtitle ("Remembers everything. Improves itself.")
+   carries all the meaning. Lead with the plain-language promise.
+9. **First mention of "agents" is undefined** (`11-onboarding-workspace.png`: "memory, files,
+   and agents stay isolated… Your agent learns each workspace's patterns"). One sentence earlier
+   in the flow should introduce what an agent is ("your AI helper").
+10. **Overnight/automation brag is mostly plumbing** (`06-automation-center.png` /
+    Home Overnight panel): "Memory consolidation," "Marketplace sync," "Index reconciliation"
+    counted in "5 automations completed." Count only user-meaningful automations on Home, or
+    label the rest "housekeeping."
+11. **Stat panel jargon + deflating zero** (`02-home-cockpit.png` / `14-novice-simple-dock.png`
+    Overnight panel): "Memories consolidated," "0 Artifacts created." Use plain words
+    ("things it learned," "documents made for you") and hide zero-count stats.
+
+## Verdict in one line
+
+The memory promise lands — I felt greeted, remembered, and given reasons to come back — but the
+self-evolving promise is a caption on an empty room, and one raw JSON error at the
+resume-your-work moment would send a real novice straight back to their email tab.