oleg.m/waggle-os
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

moving

Browse Source
This commit is contained in:
Oleg Maslov
2026-09-02 10:14:22 +02:00
parent 0c3e2ead3b
commit b20b138fe4
771 changed files with 161561 additions and 9027 deletions
Download Patch File Download Diff File Expand all files Collapse all files

35
docs/GETTING-STARTED.md
View File

@@ -2,19 +2,24 @@
Welcome to Waggle — your AI operating system with persistent memory.
## Quick Start (5 minutes)
## Quick Start
### 1. Install Waggle
Download the desktop app for your platform:
- **Windows**: Download the `.msi` installer
- **macOS**: Download the `.dmg` installer
### 1. Install Waggle for Windows
Run the installer and launch Waggle.
Use only the signed Windows Solo `.exe` installer and SHA-256 identified by a **GO** [launch recommendation](production-readiness/09-LAUNCH_RECOMMENDATION.md). If that recommendation is not GO, no packaged desktop artifact is release-approved; use the source-development instructions in the repository instead. macOS desktop packaging and certification remain roadmap work.
### 2. Set Up Your API Key
- Open Waggle — the onboarding wizard appears on first launch
- Get an Anthropic API key at https://console.anthropic.com/settings/keys
- Paste it in the wizard and click "Validate & save"
Run the installer and launch Waggle. The Windows Solo release contract does not require developer Node.js, Docker, Python, LiteLLM, administrator access, or a separate Ollama installation.
### 2. Connect a model
The first-launch wizard offers two supported paths:
- **Private local model:** open **Local model**, choose **Install private runtime**, then download and verify the recommended model. Waggle keeps the managed runtime and model in its private data directory; no system Ollama or Docker installation is required.
- **Cloud provider:** open **API key**, choose a provider, and paste your own provider API key. Waggle validates it live when the provider is reachable, labels any unverified result, and stores it encrypted in Vault.
At least one working local model or cloud provider is needed for chat. You can choose **I'll do this later**, but chat remains unavailable until a model is configured.
External coding-agent authentication is separate from Waggle's model setup. Install each supported client yourself and use its own sign-in flow: Claude Code (`claude auth login --claudeai`), Codex (`codex login`), and Hermes' built-in provider OAuth (`hermes auth add openai-codex --type oauth`). Waggle uses the client's existing user session; do not paste or copy those credentials into Waggle.
### 3. Create Your First Workspace
- Choose a template (Sales, Research, Legal, etc.) or start blank
@@ -82,11 +87,11 @@ Once you start chatting, click the **Workspace Overview** toggle above the messa
## Product Scopes
Waggle works across four product scopes:
- **Solo** — personal productivity with persistent memory
- **Teams** — shared workspaces with collaboration
- **Business** — team management, analytics, and governance
- **Enterprise** — full KVARK integration, compliance, and audit trails
Waggle uses four product tiers:
- **Trial** — 15 days of the Team experience, then Solo
- **Solo (Free)** — personal productivity with persistent memory
- **Team** — shared workspaces, WaggleDance, and governance
- **Enterprise** — sovereign KVARK deployment, compliance, and audit trails
## Need Help?
Type `/help` in chat for a full command reference.

11
docs/PM-SYNC-PRE-DAY0-2026-05-05.md
View File

@@ -1,5 +1,11 @@
# PM Sync — Pre-Day-0 Two-Repo Survey
> [!CAUTION]
> **ARCHIVED / SUPERSEDED — DO NOT EXECUTE.** This May 2026 snapshot predates
> the curated-forward-port policy. Raw subtree/export publication recommendations
> are invalid; current authority is `AGENTS.md` §7.5 and
> `packages/hive-mind-core/CONTRIBUTING.md`.
**Date:** 2026-05-05
**Scope:** `D:\Projects\waggle-os` (proprietary monorepo, branch `main`) + `D:\Projects\hive-mind` (Apache 2.0 OSS, branch `master`)
**Author:** CC inventory pass, no mutations to either repo
@@ -409,6 +415,11 @@ A `feat/sync-to-waggle-os-workflow` branch already exists in hive-mind (origin s
### §8e — Dangling concerns (not raised by PM Claude)
> **CORRECTION (2026-08-22):** Item 1's raw subtree-push recommendation is
> invalid and must not be executed. Reconcile the public mirror only through a
> reviewed maintainer-curated forward-port that excludes private files and
> interleaved `install_audit` logic.
The following are observations from code-side that PM-Claude wouldn't have visibility into. Each surfaces a real risk that should be tracked.
1. **OSS export sync drift is the #1 launch-comms risk.** Day 0 marketing will likely point at `https://github.com/marolinik/hive-mind` as the canonical OSS repo. That repo's master is **8-11 days behind** what waggle-os has prepared in the 12 oss-export branches (Wave-1 doctor command + Windows Quirks docs + postinstall + mcp-health-check; Apache 2.0 boundary + CONTRIBUTING; OSS subtree split + import sweep + smoke). External eyes landing on hive-mind master will see a less-polished extraction than the one PM-side comms might describe. Recommend: subtree-push the 12 oss-export branches into hive-mind master + bump to v0.2.0 + republish to npm BEFORE Day 0 comms go live.

6
docs/audits/2026-07-08-ux-post-phase-1-roadmap.md
View File

@@ -1,5 +1,11 @@
# UX Post-Phase-1 Roadmap
> **Historical roadmap — superseded.** This 2026-07-08 planning artifact predates
> the Windows-first Solo launch scope and its ten-persona acceptance evidence. Cursor,
> OpenClaw, and macOS are roadmap items, not current internal-RC blockers. See
> [`../production-readiness/09-LAUNCH_RECOMMENDATION.md`](../production-readiness/09-LAUNCH_RECOMMENDATION.md)
> for current launch authority.
Status: analysis artifact. This roadmap sequences the remaining correction work after Phase 1. It is not approved for implementation yet.
## Purpose

13
docs/briefs/2026-05-05-claude-md-amendment-invariants.md
View File

@@ -1,9 +1,14 @@
# CC Brief — CLAUDE.md Amendment: Code-Level Invariants
> [!CAUTION]
> **ARCHIVED / SUPERSEDED — DO NOT EXECUTE.** This brief predates `AGENTS.md`
> as the sole operating contract. Its raw subtree/export publication guidance
> is invalid; use `AGENTS.md` §7.5 and the curated-forward-port workflow.
**Brief ID:** `cc-claude-md-amendment-invariants-v1`
**Date:** 2026-05-05
**Author:** PM
**Status:** READY (Marko ratifikovao 2026-05-05 "uradi to sve")
**Status:** ARCHIVED / SUPERSEDED (was READY and ratified 2026-05-05 under the superseded model)
**Stream:** Solo CC sesija (single commit, single PR)
**Wall-clock:** 30-45 min CC implementation
**Cost cap:** $5 hard / $3 halt / $1-2 expected (mostly file edit + lint + commit)
@@ -78,6 +83,12 @@ If a CC session is asked to merge `feature/apps-web-integration` to `main` and a
### OSS subtree split discipline
> **CORRECTION (2026-08-22):** Raw `oss-hive-mind-*-export` refs are local
> inspection artifacts only and must never be published. Author changes in the
> private monorepo, run the fail-closed drift check, and prepare a reviewed
> maintainer-curated forward-port in the OSS checkout. The historical text
> below is retained only to explain the superseded decision.
The `oss-hive-mind-*-export` branches in this repo are not hand-edited. They are generated by `git subtree split` against the latest `feature/hive-mind-monorepo-migration` HEAD and pushed to the public `marolinik/hive-mind` repo as the corresponding package branches.
If a CC session is asked to commit directly to any `oss-hive-mind-*-export` branch, halt-and-PM. The correct workflow is: edit in `packages/hive-mind-*` on the monorepo migration branch, then re-run subtree split to regenerate the export branch.

14
docs/briefs/2026-05-05-day-0-minus-1-runbook.md
View File

@@ -1,9 +1,16 @@
# CC Runbook — Day 0 Minus 1: OSS Push Gate + Track A Tag Ceremony
> [!CAUTION]
> **ARCHIVED / SUPERSEDED — DO NOT EXECUTE.** This May 2026 runbook predates
> the curated-forward-port security boundary. Every raw subtree/export-branch
> publication command below is invalid and retained only as historical audit
> evidence. The only authoritative OSS process is `AGENTS.md` §7.5 plus
> `packages/hive-mind-core/CONTRIBUTING.md`.
**Brief ID:** `cc-day-0-minus-1-runbook-v2` (amended 2026-05-05 sa NPM republish sekcijom posle CC PM-sync survey nalaza #1)
**Date:** 2026-05-05 (v1) + amendment 2026-05-05 (v2)
**Author:** PM
**Status:** RUNBOOK READY (Marko ratifikovao 2026-05-05 "uradi to sve")
**Status:** ARCHIVED / SUPERSEDED (was ratified under the invalidated May 2026 model)
**Stream:** Solo CC sesija (sequential, ne paralelno) — izvršiti dan pre javnog Waggle launch-a
**Wall-clock:** 75-110 min ukupno (30-45 min push gate + 15-20 min NPM republish + 15 min tag ceremony + 10-15 min verifikacija + 10 min rollback drill)
**Cost cap:** $8 hard / $5 halt / $1-3 expected (pure git + npm ops, minimal LLM)
@@ -128,6 +135,11 @@ git log --oneline origin/feature/hive-mind-monorepo-migration -3
## §2 — Push gate Faza B: 12 OSS export grana (15-20 min)
> **INVALIDATED SECTION — DO NOT RUN OR ADAPT THESE COMMANDS.** Raw
> `oss-*-export` refs have the wrong public layout and can include proprietary
> files plus interleaved `install_audit` logic. A reviewed, maintainer-curated
> forward-port from the private monorepo is required instead.
Ovo su subtree split grane koje rebrendiraju istoriju packages-a kao samostalne repo-e za hive-mind javnu distribuciju. Push redosled je bitan zbog dependency hijerarhije.
### §2.1 — Validacija subtree split aktualnosti

37
docs/code-signing-pilot-and-launch.md
View File

@@ -1,5 +1,14 @@
# Code-Signing Playbook — Pilot + Public Day 0
> [!CAUTION]
> Only `tauri:build:win:pilot-signed` below is current internal-pilot guidance.
> macOS is deferred; its historical commands are not release certification.
> **PARTIALLY SUPERSEDED — DO NOT USE THE PUBLIC-SIGNING SECTIONS.** The pilot
> history remains valid for internal RC evidence. Public Windows signing now
> uses the protected exact-tag GitHub workflow with Azure OIDC/federated identity;
> client-secret service-principal flows below are invalid. Current ship authority
> is `docs/production-readiness/09-LAUNCH_RECOMMENDATION.md`.
**Decision (2026-05-07):** Self-sign for pilot (Wave-1 Egzakta-internal, T+10 = 2026-05-16). Procure real certs in parallel for public Day 0 (T+30 = 2026-06-05).
**Why split:** Real Authenticode + Apple Developer ID have 5-15 business-day lead times. Self-signing unblocks the pilot ship date without paying for certs that aren't usable until a vetting check completes.
@@ -25,8 +34,9 @@ npm run tauri:sign:pilot:win:setup
npm run tauri:sign:pilot:win:apply
# 3. Build — Tauri's MSI/NSIS bundlers pick up the thumbprint from the
# override config and sign automatically.
npm run tauri:build:win
# explicitly passed override config and sign automatically. The plain
# tauri:build:win command does not load this override.
npm run tauri:build:win:pilot-signed
# 4. (Optional, redundant safety) Re-sign the produced MSI explicitly
# via signtool. Useful if you want to apply timestamp at a different time
@@ -46,7 +56,10 @@ npm run tauri:sign:pilot:win:sign -- src-tauri/target/release/bundle/msi/Waggle_
### 1.2 macOS self-sign (ad-hoc)
**Already wired (LAUNCH-06).** `app/src-tauri/tauri.build-override.conf.json` ships with `bundle.macOS.signingIdentity = "-"`, so every `npm run tauri:build:mac` produces an ad-hoc-signed `.app` automatically — no operator step required.
> **HISTORICAL / ROADMAP ONLY.** macOS is outside the Windows-first launch cohort
> and has no current package certification. `npm run tauri:build:mac` does not
> automatically load `tauri.build-override.conf.json`; do not label its output
> signed merely because the override contains `signingIdentity = "-"`.
**Re-sign + verify wrapper.** For nested helpers (sidecar, native deps) Tauri's bundler may miss, run:
@@ -85,6 +98,15 @@ Subject: **Welcome to Waggle Wave-1 — install + first-run notes**
---
<details>
<summary><strong>Archived public-signing history — do not execute</strong></summary>
> The remainder of this document preserves obsolete vendor, local-thumbprint,
> client-secret, timeline, and cost notes for audit history only. Current public
> Windows signing is the protected exact-tag Azure OIDC workflow in
> `.github/workflows/release.yml`; current ship authority is
> `docs/production-readiness/09-LAUNCH_RECOMMENDATION.md`.
## 2. Public Day 0 prep — real certificates (start by T+15 = 2026-05-21)
### 2.1 Windows Authenticode
@@ -190,6 +212,11 @@ After notarization + stapling, Mac users see no warning on first launch — `.dm
## 3. CI integration (post-pilot, before public Day 0)
> **INVALIDATED SECTION — DO NOT EXECUTE.** The Azure client-secret example and
> May 2026 target dates below are historical only. Public Windows signing must
> use `.github/workflows/release.yml` with hosted OIDC, exact-tag controls, and
> a publicly trusted certificate after identity validation.
**Target state:** every `main`-branch tag triggers signed builds for both platforms.
**Windows (GitHub Actions sketch):**
@@ -284,5 +311,7 @@ Renewals: ~$348/yr ongoing. Tauri auto-updater also expects signed binaries —
---
Last updated: 2026-05-10 — LAUNCH-06 self-sign automation landed (Phase 2 Step 4). §1.1 now points to the npm-wrapped scripts (`tauri:sign:pilot:win:setup` / `:apply` / `:sign`) backed by `app/scripts/sign-windows-pilot.ps1`, `apply-signing-config.mjs`, and the tested `signing-config.ts` utility. §1.2 ships the macOS ad-hoc identity in the build-override config by default; `tauri:sign:pilot:mac:adhoc` re-signs nested helpers post-build.
Last updated: 2026-08-22 — Windows internal-pilot builds must use `tauri:build:win:pilot-signed`. macOS packaging remains deferred; an ordinary macOS build is not signed by the unused override and must be explicitly signed and verified before it is described as an internal artifact.
Owner: Marko Marković (driving via CC); pilot self-sign actionable T+5; real cert procurement actionable T+14.
</details>

2
docs/launch/drafts/2026-05-12-apps-www-deployment-readiness.md
View File

@@ -165,7 +165,7 @@ environment before deploy.
| `NEXT_PUBLIC_CLERK_SIGN_UP_FALLBACK_REDIRECT_URL` | Where to send post-signup (e.g. `/`) |
| `CLERK_WEBHOOK_SECRET` | For Clerk → tier publicMetadata sync; set if using Clerk webhooks (else can defer) |
| `STRIPE_SECRET_KEY` | Live key, on Egzakta account (`CNCrMQy1f7`). **Marko declined rotation per session call** — left in chat log; consider rotating post-Day-0. |
| `STRIPE_WEBHOOK_SECRET` | `whsec_smwv4fInMEzJ3NSj3AwcNY2pz21eVrBu` (live endpoint created Wave 2 PG3 prep) |
| `STRIPE_WEBHOOK_SECRET` | `[REDACTED - rotate in Stripe and store only in the deployment secret manager]` (live endpoint created Wave 2 PG3 prep) |
| `STRIPE_PRICE_PRO_MONTHLY` | `price_1TVsaMCNCrMQy1f7KwImpxA4` (Pro $19/mo) |
| `STRIPE_PRICE_PRO_ANNUAL` | `price_1TVsaLCNCrMQy1f7YhQGDH2o` (Pro $190/yr) |
| `STRIPE_PRICE_TEAMS_MONTHLY` | `price_1TVsaLCNCrMQy1f7WUhtbZ88` (Teams $49/seat/mo) |

149
docs/paper/2026-07-10-audit-response-plan.md Normal file
View File

@@ -0,0 +1,149 @@
# Audit Response Plan: Clever Memory Loses
**Date:** 2026-07-10
**Input:** publication-strength audit (2026-07-10), verdict no-go arXiv/press, conditional-go corrected blog.
**Decision:** ACCEPT the audit's core finding and PIVOT the thesis. Defend only 3 sub-points (below). The audit refutes the "raw-only, zero structure" claim using our own `benchmarks/results/longmemeval/RESULTS.md` — that is not survivable in review, and internal records confirm it (LongMemEval final = observation extraction + category routing + voting + KG-ledger guard; LoCoMo final = seven-lane layered system).
---
## 1. Triage of the 10 blockers
| # | Blocker | Ruling | Action |
|---|---|---|---|
| 1 | Thesis contradicted by own evidence | **ACCEPT — fatal as written** | Pivot thesis (§2) |
| 2 | LongMemEval routes on `question_type` metadata | **ACCEPT** | No-oracle rerun = new headline (E1) |
| 3 | LongMemEval tie presented as win | **ACCEPT with partial defense** | Reword: "ties 468/500 micro; +0.14 on incumbent's own macro aggregation." Macro is Mastra's own published metric, so reporting it is fair — claiming SOTA on it is not. Post-QA draft already concedes noise; title/abstract/dossier do not. Fix all. |
| 4 | Adaptive test-set tuning invalidates confirmatory p-values | **ACCEPT** | Relabel campaign exploratory; frozen confirmatory reruns (E3) |
| 5 | "33 made it worse" overclaims | **ACCEPT** | Rename Engineering Intervention Log; two tables (BEAM / LME), per-row N, model, baseline, metric, execution status, CI, adoption rule. Correct slogan: "33 not adopted; 4 adopted." |
| 6 | Conflict effect not causally isolated (store vs prompt) | **ACCEPT experiment, CONTEST breadth claim both ways** | Run 2×2 ablation (E2). Also narrow our own incumbent claim: Zep/Graphiti is bitemporal and retains history — say "systems that reconcile at write time on the answer path," not "every incumbent." |
| 7 | "Reproduced each incumbent" false for 2 of 3 | **ACCEPT** | Use audit's replacement wording verbatim |
| 8 | "No memory content leaves the machine" misleading | **ACCEPT** | Use audit's replacement wording verbatim; fix press kit too |
| 9 | SOTA landscape stale (mem0 92.5 LoCoMo, Hindsight 73.9 BEAM-1M) | **ACCEPT with defense** | New numbers are protocol-incomparable (different models/configs) — handle with landscape table + protocol-compatibility column, not silent retitle. But unqualified "state of the art" in title is dead regardless. |
| 10 | Reproducibility uneven (LME pipeline not public) | **ACCEPT** | Publish scripts 34–55 + per-question artifacts, or narrow Appendix B claim |
**Partial defenses to keep (write into rebuttal/limitations, do not overplay):**
- D1: `question_type` is benchmark-provided input, not a gold answer — but Mastra doesn't use it, so head-to-head is still unclean. Disclose + rerun; keep routed number as a labeled secondary result.
- D2: Macro aggregation is the incumbent's own leaderboard metric; we report both and lead with micro.
- D3: BEAM result IS the clean simple-substrate result — one benchmark where the raw-only story is fully true. The pivot thesis keeps it as the flagship.
---
## 2. Thesis pivot
**Old (dead):** one dumb raw-turn substrate, no distillation/graph/routing, wins all three.
**New:** *preserve dated raw evidence as the canonical store; make every derived view reversible; defer conflict resolution to read time.* Raw turns dominate detail- and contradiction-sensitive tasks (BEAM, all-raw win); derived observations and read-time aggregation help breadth/counting (LongMemEval, LoCoMo); nothing on the answer path irreversibly deletes evidence.
**Title candidates** (pick after E1/E2 results):
1. "Clever Memory Loses When It Deletes the Evidence" (keeps the sticky brand, now true)
2. "Preserve First, Transform Later: A Lossless Memory Substrate Across LoCoMo, LongMemEval, and BEAM"
**Contributions restated:** (1) lossless canonical-store architecture; (2) three protocol-matched studies, presented separately; (3) causal conflict-preservation ablation on BEAM; (4) protocol-fidelity audit (unchanged — strongest surviving section); (5) engineering intervention log, honestly labeled.
Bitter Lesson angle survives as: "do not irreversibly discard evidence," not "never build structure."
---
## 3. Phases
### Phase 0 — Verify audit citations (0.5 day, agents, no writes)
Audit is specific and matches memory, but confirm before rewriting on top of it:
- [ ] `RESULTS.md:43-49, 27-40, 64-76` say what audit says
- [ ] `hive-mind/benchmarks/longmemeval/42-compose-final.mjs:15,36-40` routes on `it.question_type`
- [ ] Ledger entries claimed flat/positive (+0.023 temporal-commit, count-hint +3, etc.) — recheck signs in source tables
- [ ] Mastra category counts sum to 468/500 (mastra.ai/research/observational-memory)
- [ ] mem0 memory-benchmarks repo current numbers; Hindsight BEAM-1M 73.9 blog post; LIGHT/Honcho results
- [ ] BEAM repo license split (CC BY-SA 4.0 data / MIT code)
- [ ] Supermemory 85.4 vs 85.9 inconsistency; MemR3 duplicate reference
- **Gate:** any audit claim that fails verification gets struck from the plan; rest proceeds.
### Phase 1 — P0 rewrite (1–2 days, no new compute)
1. Rewrite title/abstract/intro/conclusion around pivot thesis.
2. Kill four false slogans everywhere (draft, blog, deck, posts, press kit): "zero per-benchmark tuning," "swap and change nothing," "33 made it worse," "every transform loses."
3. LongMemEval: micro tie first, macro second, metadata-routing disclosed in results section, not a footnote.
4. Privacy wording per audit (§8). Reproduction wording per audit (§7).
5. Section 4 rewritten: one canonical store, three benchmark-specific read paths, presented as three configurations of one preservation principle.
6. Ledger → Engineering Intervention Log (two tables + qualitative synthesis).
7. Figures: fig1 redrawn as one store / three read paths (or labeled BEAM-only interim); fig2 split into two panels with CIs or adoption matrix.
8. Related work: Memori arXiv:2603.19935 proper cite; Zep bitemporal correction; landscape table with protocol-compatibility column incl. current mem0/Hindsight/LIGHT; fix Supermemory number; dedupe MemR3; real bibliography.
9. Strip "Phase B-1 draft" status line; fix page-count/table-count in arxiv-metadata; fix title duplication p.1; fix orphaned Table 6 / blank half-pages.
10. `DATA_LICENSES.md` (LoCoMo, LongMemEval CC?, BEAM CC BY-SA 4.0 data) + attribution in result JSONL release.
11. Archive `draft.v1.md`, both DOCX, old LaTeX to `docs/paper/archive/` with README note (they present the layered thesis — audit is right that leaving them loose invites "your own files disagree").
### Phase 2 — P1 experiments (compute; sequence by information value)
- **E1 — LongMemEval no-oracle rerun** (highest value, cheapest): frozen config, ONE uniform read policy across all 500 Q (arm A); optional arm B = NL-only question classifier, report its confusion matrix. New headline number = arm A. Routed 95.01 becomes labeled secondary. Risk handled in §4.
- **E2 — BEAM 2×2 store×prompt ablation**: {raw-versioned, reconciled-current-only} × {incumbent prompt, conflict-aware prompt}. Stage 1: contradiction-ability subset (~100 Q × 4 cells) — isolates the +23pp mechanism cheaply. Stage 2 (if stage 1 clean): full 700 × 4. Reconciled store = simulate write-time reconciliation over same turns (mem0-style ADD/UPDATE/DELETE pass).
- **E3 — Confirmatory frozen reruns**: BEAM + LongMemEval final configs, 3 independent answer/judge passes each, report run distributions. Fixes the "judge-noise SE" mislabel with actual re-judging variance.
- **E4 — LoCoMo paired test**: McNemar vs reproduced Memori per-question outcomes + paired CI on accuracy difference; demote one-sample z-test.
- **E5 — Stats hygiene**: paired bootstrap CIs everywhere; exact tests; label exploratory vs confirmatory endpoints; BEAM avg-score delta reported as tie (CI −0.019..+0.034).
### Phase 3 — Release ops (after 1+2)
1. Publish full LME pipeline (scripts 34–55) + per-question artifacts to public repos; verify public-tree parity with Appendix B claims.
2. Regenerate ALL launch assets from ONE claim matrix (single source of truth: claim → evidence file → status). PDF, arxiv-metadata, blog, posts, press kit, deck.
3. Proper bibliography (BibTeX), consider LaTeX/Typst build instead of Chrome print.
4. Re-run internal QA gates (adversarial review, anti-paper) against the NEW draft.
### P2 (only if targeting main conference — defer)
Weaker answerer family + alternative judge; held-out confirmatory slice; cost–quality Pareto; real-world contradiction eval; one-command pinned repro env.
---
## 4. Risk register
| Risk | Handling |
|---|---|
| E1 no-oracle drops below 94.87 | Paper survives — pivot thesis does not require winning LME. Report honestly: "matches/near leader; routed variant reaches X with disclosed metadata routing." Tie-with-simpler-read-path is still a result. |
| E2 shows prompt (not store) carries the +23pp | Also survivable — thesis becomes "read-time conflict policy over preserved evidence"; store retention is the necessary precondition (prompt can't surface deleted history). Interaction cell measures exactly this. |
| Confirmatory reruns regress BEAM pass-rate significance | Report distribution; drop p-value claims to descriptive. BEAM avg was already a tie. |
| mem0/Hindsight newer numbers steal headline | Landscape table with protocol column; claims scoped "under incumbent's published protocol as of [date]." |
| Rewrite drifts back to hype | Claim matrix is the gate: no sentence in any launch asset without a matrix row. |
---
## 5. Go/no-go (mirrors audit gates)
| Target | Gate |
|---|---|
| Corrected blog | Phase 1 items 1–4 + slogan kill |
| Social/HN launch | Blog gate + landscape table |
| arXiv preprint | Phase 1 complete + E1 + E2-stage-1 |
| Workshop paper | + E3, E4 |
| Main conference | + P2 item(s) |
## 6. Suggested execution order
1. Phase 0 verification (today, parallel agents).
2. Decision checkpoint: confirm pivot + title with user.
3. E1 + E2-stage-1 launch (compute runs overnight) in parallel with Phase 1 rewrite.
4. Assemble claim matrix → regenerate assets → QA gates → arXiv.
Rough new-compute cost: E1 ~500–1000 answer+judge calls (gpt-5-mini/gpt-4o); E2 stage 1 ~800 gpt-5 calls; E3 ~3×(700+500) both roles. Order of magnitude comparable to one prior full-700 run — low hundreds of dollars, not thousands.
---
## 7. Phase 0 RESULTS (2026-07-10, three independent verifiers)
**Verdict: audit confirmed on all internal citations and all statistics; 3 external claims softened in our favor.**
Internal (verify-internal): claims 1–7 ALL CONFIRMED with file:line quotes. Ledger decomposition of "33 lost": 21 strictly negative / 6 flat / 3 positive-unadopted / 3 analytical-only — all six audit-named entries executed A/Bs, flat-or-positive as audit said. No NL classifier anywhere in LME scripts 34–55; routing purely on dataset `question_type`. Only audit slip: "6 tables" metadata was accurate (page count 14→16 still our error).
Stats (verify-stats): every number MATCH — BEAM ours 0.6482016 / 518/700; mem0 0.6408656 / 491/700 (gpt-5 answerer+judge, top_200 — like-for-like judge symmetry confirmed); McNemar 425/93/66/116, z=2.141, p=0.0323 (asymptotic), 0.0392 (continuity), 0.0389 (exact); paired delta +0.007336, CI [−0.0193, +0.0340], bootstrap agrees; LoCoMo 1332/1540 recount exact; contradiction ours 0.5875 vs mem0 0.3571 (+0.2304). Audit's judge-noise-SE point confirmed: draft's "0.014 judge-noise band" is across-question sampling SE (0.01348/0.01359), not judge noise. Method note: ours↔mem0 pairing must join on question TEXT (id ordering differs; id-join collapses to 70 rows). **E4 unblocked: Memori per-question reproduction exists at `D:\Projects\memori-repo\benchmarks\results_gemma\eval_20260609T035542Z.json` (1540 entries, join-able).**
External (verify-external): Mastra 94.87 CONFIRMED = macro, gpt-5-mini answerer + gpt-4o judge (protocol-matched to ours; micro tie 468/500 exact). BEAM data license CC BY-SA 4.0 CONFIRMED → DATA_LICENSES.md required. Zep bitemporal CONFIRMED (edge invalidation, history preserved) → our "every incumbent deletes" claim dead. Memori cite arXiv:2603.19935 CONFIRMED. **Softened:** (a) Hindsight 73.9 BEAM-1M uses Llama-4-Maverick judge, unstated metric, headline actually 64.1%@10M — NOT comparable; (b) mem0 92.5 LoCoMo = Top-200 + GPT-5 judge — protocol-incomparable to our Memori-protocol 86.49; (c) no public evidence incumbents do/don't route on `question_type` — reporting per-category ≠ routing. Landscape table with protocol-compatibility column is the right instrument (audit agreed). LongMemEval `_abs` abstention marking (30 Q, id suffix not question_type) must be handled identically in E1.
**Decisions locked:** pivot thesis per §2; all Phase 1 items proceed; E1 arm A = last pre-routing ladder rung config, uniform for all 500 Q; final step after rewrite + E1/E2 = independent re-audit ("re-judge") of the new package.
---
## 8. RIVAL PROTOCOL FORENSICS (2026-07-10) — the "we lost SOTA" numbers dissected
**Mastra 94.87 (LME):** macro artifact. Micro = 468/500 = 93.60, EXACT TIE with our routed run. Same answerer (gpt-5-mini), same official gpt-4o judge. Our macro 95.01 > their 94.87 — but ours oracle-routed, theirs not. True deficit: production-legal only (~92.4 classifier-routed vs their 93.60). Their system: gemini-2.5-flash ingestion-time observation compression, one static ~30k-tok context, single pass, open source. Beat = close ~6–10 questions in temporal-reasoning + knowledge-update without labels. Multi-session already tied (116/133 both).
**mem0 92.5 (LoCoMo):** protocol-inflation stack, NOT a substrate win. Their own paper (arXiv:2504.19413) scored J≈67% on the same 1540. The 92.5 = gpt-5 answerer + gpt-5 judge with maximally lenient prompt (1-of-N list items = CORRECT; ±14-day dates; ±50% durations; same-valence emotions; abstention-banned CoT answerer; cat-3 gold truncated at semicolon; adversarial cat-5 excluded — same 1540 scope as ours) + Platform-v3 closed retriever, top-200 (top-k lever only +0.7pp vs top-50). Judge is directly reusable standalone: `benchmarks/locomo/prompts.get_judge_prompt` + `common/llm_client.LLMClient` — ~20-line script over our (category, question, gold, prediction) triples. Full comparable rerun config documented in forensic report.
**Eywa 81.45 (BEAM):** not a comparable number. Sonnet 4.6 as BOTH answerer AND judge, custom rubric harness (paper misleadingly says BEAM "introduced here"; official-nuggets-or-reauthored unverifiable — artifacts URL 403s, no code, single-author vendor self-report), ZERO in-harness baselines, undisclosed context budget. Answerer edge small (+1.4pp Sonnet-vs-gpt-4o by their own LoCoMo anchor); judge is the story. Triangulated: plain raw-turn substrate under their harness ≈ 0.70–0.74 avg → Eywa's true like-for-like edge ≈ 5–10 pts (concentrated in abstention 92.9, temporal 90.0; their weakness = summarization 64.1, same as ours; contradiction we already own via dated turns). **CRITICAL protocol note: Hindsight's 73.9 is the AMB harness (Gemini answerer + Gemini judge, vendor-run) — ALSO not comparable to our gpt-5/gpt-5.** Honest position: our 0.648/74.0% vs mem0 0.641/70.1% is the ONLY clean like-for-like BEAM-1M comparison in existence; no one has published a comparable number above ours. Landscape table needs judge column: Eywa (Sonnet self-judge) / Hindsight+Honcho (Gemini/Gemini AMB) / mem0+ours (gpt-5 or gpt-4o official-style) — three islands, not one leaderboard.
### Campaign menu (SOTA recovery)
- **C1 LoCoMo unqualified SOTA (cheapest, highest probability):** rerun our substrate under mem0's exact protocol (gpt-5 answerer, their judge prompt verbatim, top-200, cat 1–4). Expected 92–95 given we score 86.49 under a FAR stricter judge. Stage 1 (cheap, no re-answering) = 3-pass judge decomposition over our EXISTING 1540 answers: (a) Memori judge baseline 86.49; (b) mem0 `_JUDGE_TEMPLATE` + gpt-4.1-mini → isolates prompt leniency; (c) mem0 `_JUDGE_TEMPLATE` + gpt-5 → isolates judge model. Residual to 92.5 after (c) = answerer + retrieval + their 7-step CoT answer prompt (abstention banned — third confound lever; for stage 2 rerun, decide ours-vs-theirs answer prompt explicitly). Adapter = ~30 lines importing `benchmarks/locomo/prompts.get_judge_prompt` + `common/llm_client.LLMClient` from D:\Projects\mem0-memory-benchmarks (cat-3 golds get `preprocess_answer` semicolon truncation). Total est. <$50, half a day.
- **C2 LME production-legal lead:** finish E1b exact number (was mid-run, ~$3), then target temporal-reasoning (84.2→) + knowledge-update + multi-session with label-free levers (uniform voting; observation-layer improvements à la Mastra). Need ≥469/500 micro no-oracle. Moderate difficulty.
- **C3 BEAM vs Hindsight 73.9:** abstention gate (biggest structural gap), knowledge-update latest-fact selection, summarization lane. Research campaign, days + iterative pilots. Judge caveat: Hindsight's judge identity (Llama-4-Maverick per their comparison page) still muddies exact comparability — verify before claiming.

34
docs/paper/2026-07-11-codex-consult-prompt.txt Normal file
View File

@@ -0,0 +1,34 @@
IMPORTANT: Do NOT read or execute any files under ~/.claude/, ~/.agents/, .claude/skills/, or agents/. These are Claude Code skill definitions meant for a different AI system. Do NOT modify agents/openai.yaml. Stay focused on this consultation only.
You are a brutally honest technical reviewer (think: hostile NeurIPS reviewer + benchmark methodologist). Another AI (Claude/Fable) orchestrating a memory-systems research program wants your independent verdict on its path to claiming SOTA on three conversational-memory benchmarks. Be direct, terse, no compliments. Challenge assumptions.
=== CONTEXT ===
System under test: "conflict-aware raw-turn memory" substrate (per-conversation stores, verbatim dated raw turns, hybrid dense+BM25 retrieval, conflict-preserving answer policy). Paper draft was audited; original "SOTA on all 3" claim collapsed. Verified current positions:
LOCOMO (1540 Q, cats 1-4, adversarial excluded):
- Ours: 86.49% under Memori protocol (gpt-4.1-mini answerer+judge, strict judge). Beats Memori 81.95 same protocol (paired artifacts exist).
- mem0 claims 92.5 (README) but their released per-question artifact reproduces 91.56 (1410/1540). Their protocol: gpt-5 answerer + gpt-5 judge with codified-lenient prompt (1-of-N list items = CORRECT, plus-minus 14-day dates, 50% duration tolerance, abstention-banned 7-step CoT answer prompt, cat-3 gold truncated at first semicolon) + top-200 memories. mem0's own 2025 paper scored ~67 on same set with stricter judge.
- Judge decomposition DONE on our existing answers: our answers + mem0 judge prompt on gpt-4.1-mini = 93.90; + mem0 judge prompt on gpt-5 = 90.13. So we sit 1.4pp under their artifact with a far weaker answerer (gpt-4.1-mini vs gpt-5) and top-70 retrieval.
- IN FLIGHT (C1 stage 2): rerun our substrate with gpt-5 answerer + top-200 parity + mem0 gpt-5 judge, McNemar vs their 1410/1540.
LONGMEMEVAL-S (500 Q, official gpt-4o judge, gpt-5-mini answerer both us and incumbent):
- Mastra (incumbent): 94.87 macro / 93.60 micro (468/500). No metadata used.
- Ours published: 95.01 macro / 93.60 micro (468/500) — micro EXACT TIE — but our routing read the benchmark-provided question_type annotation (oracle label). Disclosed now.
- Production-legal (text-only classifier routing, exact, just computed): 92.20 micro / 92.87 macro (461/500). Uniform no-routing: 88.20 micro.
- Miss analysis: 39 misses; classifier misroutes cost exactly 4; ~20 are hard substrate floor (multi-session counting/aggregation); union-of-all-methods ceiling 480/500. Levers: better classifier +2-4 (cap), abstention calibration +2-4, multi-method ensemble +3-6 at 2-5x cost. All exploratory; LongMemEval has NO held-out split; everything tunes on the same 500.
BEAM-1M (700 Q, 35 convs, nugget 0/0.5/1 judge):
- Ours: 0.6482 avg / 74.0% pass@>=0.5, gpt-5 answerer + gpt-5 judge (like-for-like with mem0's released run: 0.6409 / 70.14%). Paired McNemar on pass: p=0.0389. BUT: win is contingent on healing 56 empty-generation rows (pre-heal 0.6050/69.3 = loss; heal legitimate: literal empty completions, regenerated once blind, 24/56 still failed; mem0 file has 0 empties). AND the pass win is carried by the 0.5 partial-credit bucket (strict >0.5: we lose 427 vs 429). Contradiction ability: +23pp (0.588 vs 0.357, p<1e-4), heal-independent — the one bulletproof result. Token budget: our contexts ~21K tok/Q vs their ~5-8K (3-4x) — disclosed.
- Rival numbers all protocol-incomparable: Eywa 81.45 = Sonnet 4.6 SELF-judging with self-authored rubrics, zero baselines, artifacts 403; Hindsight 73.9 avg = vendor AMB harness, Gemini answerer+judge; Honcho 63.1 same. So ours-vs-mem0 is the only clean like-for-like pair in existence.
=== THE PLANNED PATH ===
C1 (in flight): LoCoMo under mem0's exact protocol (gpt-5 answerer, their lenient judge, top-200) -> expect 92-95 -> claim "SOTA under both strict (Memori) and lenient (mem0) protocols".
C2: LME label-free push: classifier upgrade + abstention calibration, frozen config, ONE final run, no iterating (test-set discipline) -> hope 465-467; ensemble variant could reach 468-472 at 2-5x cost. Claim: "ties/near-ties incumbent production-legally; exceeds under disclosed oracle routing".
C3: BEAM substrate work: abstention gate + knowledge-update latest-fact + summarization lane -> target clean avg-score lead over mem0 (currently tie) -> claim "leads the only like-for-like BEAM comparison".
Headline strategy: "SOTA under every protocol where a like-for-like comparison exists" + protocol-fidelity audit as a first-class contribution + falsification/intervention log.
=== QUESTIONS ===
(a) AGREE/DISAGREE per track (C1/C2/C3): is this the right way to defensible SOTA claims on all three?
(b) Where does a hostile reviewer still attack, in priority order?
(c) What would you do differently or additionally? Especially: test-set-tuning discipline when no held-out split exists (LME); making BEAM claims comparable across three protocol islands; whether C1's "win under a lenient judge" claim boomerangs (does beating mem0 under mem0's own inflated protocol undermine our protocol-fidelity story?).
(d) Is "SOTA under every protocol where a like-for-like comparison exists" a defensible headline? If not, what is the strongest defensible headline?
Return structured verdict: AGREE/DISAGREE per track with reasons + ranked additional proposals.

74
docs/paper/2026-07-11-fable-codex-consensus.md Normal file
View File

@@ -0,0 +1,74 @@
# Fable × Codex consensus on the SOTA path (2026-07-11)
## 0. Second opinion: gpt-5.6-sol (high reasoning) — CONCURS, stricter on 3 points
Independent verdict before seeing prior reviewer, then adjudicated: agrees on all central conclusions (C1 → cross-judge protocol-sensitivity study; C2 SOTA chase dead — "frozen one final run is ceremony, not test discipline"; C3 contradiction result is the contribution; headline = protocol shopping, "an existentially quantified loophole"). Stricter additions:
1. **C1 2×2 insufficient** unless answerer, retrieval units, token budget, preprocessing, retry behavior are also controlled or explicitly factored. Report judge-flip transition counts + human-audit a stratified disagreement sample.
2. **BEAM McNemar likely INVALID**: 700 questions nested in 35 conversations — question-level McNemar assumes independent pairs, anti-conservative. Recompute with conversation-cluster bootstrap / cluster randomization; p=0.0389 may not survive. More fundamental than the partial-credit issue.
3. **LME oracle score gets no "exceeds incumbent" language anywhere** outside an ablation; disclosure does not repair task leakage.
Approved headline (both models): contradiction-handling gain + protocol-sensitivity audit. Performance-forward variant: "competitive aggregate performance and a robust contradiction advantage in controlled evaluations."
Codex (gpt-5-era, medium reasoning) consulted via /codex with full plan context (§7 Phase 0, §8 forensics, live C1/C2 results). Full verbatim output preserved below (§4). Prompt archived: `2026-07-11-codex-consult-prompt.txt`.
## 0.5 SMOKE TIER (agreed 2026-07-11, gpt-5.6-sol converged; <$15, 1–2 days, run BEFORE funding the package)
| # | Smoke | Kill criterion | Load-bearing? |
|---|---|---|---|
| S0 | Protocol/claim freeze doc + judge-calibration panel (30 frozen answers, official vs proposed judge, need ≥90% agreement, κ≥0.80, bias ≤2/30) | Any comparison inexpressible as one frozen protocol; calibration failure forbids cross-judge comparisons | YES |
| S1 | mem0-474 forensics ($0): their repo says GPT-5 answerer+judge for LME → 474 likely off-protocol; verify artifact-level | If 474 comparable → LME bar rises to 474 | bar-setting |
| S2 | Cluster bootstrap on existing BEAM data ($0, local, 10k paired conv bootstraps) | BEAM aggregate win dies unless cluster-aware 95% CI lower bound > 0 | YES |
| S7 | C1 stage 2 (sunk, in flight) | Green ≥+3pp vs best comparable lenient result; red if tie/lose/<1pp | YES |
| S4 | mem0-OSS under strict LoCoMo, 1 conv ~150 Q (~$3), sequential 2nd conv if ±5pp | Red if mem0 leads ≥5pp | YES |
| S6′ | Paired LME 50 (stratified, frozen): Mastra exact config + ours, same 50 (~$2) | Mastra repro <41/50 kills comparability; ours trailing ≥3/50 = red | YES (paired part) |
| S5 | AMB harness spike, ours + BM25/oracle control, 40 Q (~$5, needs Gemini key) | Kill AMB plank unless ≥38/40 complete, no manual repair, ≤$0.15/Q | YES |
Settled en route: LongMemEval-V2 EXISTS (451 Q, agent-trajectory task, own harness) — excluded from scope. GO = all load-bearing green + S0 clean. FALLBACK to contradiction+audit paper if any load-bearing red, two grays, or judge calibration fails. No averaging failures across benchmarks.
## 1. Verdicts and agreement
| Track | Codex | Fable | Consensus |
|---|---|---|---|
| C1 LoCoMo parity rerun | AGREE with restrictions | agree | PROCEED, reframed as protocol-sensitivity experiment: report a 2×2 matrix (both systems × strict Memori judge and mem0 judge). The claim is ordering stability across protocols, not "we won under the lenient judge." We hold mem0's per-question answers → their cells cost only judge calls. |
| C2 LME lever push to beat Mastra | DISAGREE (adaptive tuning; "one final run" doesn't restore independence) | agree — flagged same risk before consult | PARK the lever push. Ship 92.20 production-legal + oracle-inflation quantification (93.60→92.20→88.20 ladder) as a methodology contribution. No LME SOTA claim without a genuinely unseen eval. |
| C3 BEAM substrate work for avg-score lead | DISAGREE as framed (retry asymmetry, threshold sensitivity, would be tuned on judged 700) | agree | REFRAME: contradiction result (+23pp, heal-independent, p<1e-4) is the headline contribution. Aggregate reported as full ordinal distribution + first-attempt primary / retry-normalized secondary under a pre-declared policy. Substrate improvements (abstention gate etc.) only with dev/eval separation. |
| Headline "SOTA under every protocol where like-for-like exists" | NOT defensible (movable denominator = protocol shopping) | accept | New headline: **conflict-preserving raw-turn memory delivers a large, reproducible contradiction-handling gain under controlled comparison, and a protocol-fidelity audit shows conversational-memory rankings are underidentified (judges, routing metadata, retries, budgets).** Conditional upgrade if C1 matrix shows stable ordering: "leads matched LoCoMo and BEAM comparisons." |
Fable's two disagreements with Codex (minor):
1. Codex: "competitor had no equivalent regeneration opportunity" on BEAM. mem0's released file has 0 empties — their managed pipeline either never failed or retried internally and invisibly. True symmetric rerun of their platform is impossible; the feasible fix is the pre-declared policy + first-attempt-primary reporting Codex also proposes.
2. Codex treats C1 as unfinished evidence; it is in flight, and stage-1 judge decomposition already delivered half the protocol-sensitivity matrix.
## 2. New actions proposed (need user approval)
| # | Action | Cost | Yield |
|---|---|---|---|
| A | Cross-judge matrix on frozen answers: ours + mem0's LoCoMo answers under BOTH judges (completes C1 2×2); ours + mem0's BEAM answers under a second judge (gpt-4o) + judge-agreement stats (kappa) | ~$20–40 | Turns "protocol islands" into a first-class result; Codex proposals #3, #10 |
| B | BEAM reporting overhaul: pre-declared retry policy, first-attempt primary + healed secondary, full ordinal distribution (0/0.5/1 rates), cost/token disclosure | ~$0 (reporting) | Kills attacks #5, #6, #7 |
| C | Resume E2 causal ablation (store×prompt 2×2 on contradiction subset; was mid-flight: retrieval+reconciled store built, cells A/B partial) | ~$10 remaining | Codex proposal #4 — isolates conflict-preservation causally; upgrades the headline mechanism claim |
| D | Pre-registration doc: freeze metrics, comparisons, retry/exclusion rules, stat tests, stopping rules before any further runs | ~$0 (writing) | Codex proposal #5; discipline for everything after |
| E | Pareto cost–quality frontiers from existing artifacts | ~$0–5 | Codex proposal #6; converts token-budget liability into a result |
| F | External/unseen LME eval set (new generated conversations, independent adjudication) | days + $$ | Only path to any future LME SOTA claim; DEFER unless user wants it |
C1 stage 2 continues unchanged (already covers the ours×mem0-judge cells; A adds the mem0-answers×strict-judge cells).
## 3. Codex's ranked attack list (top 5 retained for the re-audit gate)
1. Test-set adaptation presented as evaluation. 2. SOTA defined after seeing favorable comparisons. 3. Attribution unsupported (answerer/judge/budget/substrate all vary). 4. Oracle-label contamination. 5. BEAM retry asymmetry.
## 4. Codex verbatim output
(preserved in full for the record)
You do not have a defensible "SOTA on all three benchmarks" path. You have: one likely protocol-specific LoCoMo win; one LongMemEval result below the incumbent under production-legal conditions; one narrow BEAM win on a permissive threshold, plus a much stronger contradiction-specific result. That can make a credible paper. Calling it universal SOTA would weaken it.
[C1 AGREE with restrictions — necessary; supports "best result in a controlled comparison under the released mem0 protocol"; does not establish dual-protocol SOTA; report paired outcomes + McNemar CI, strict judging of both systems' parity answers, factorial table (answerer/judge/top-k), category-level results, dedup sensitivity, cost parity; don't celebrate a lenient-judge win — call it "released mem0 evaluation protocol" and audit leniencies separately.]
[C2 DISAGREE — the process is not test-set discipline; classifier, abstention, method selection, miss analysis all informed by the same 500; "ONE final run" doesn't make it independent. 461/500 production-legal is below Mastra 468/500. Oracle result invalid for production-comparable claims; 0.14 macro is noise; "exceeds under oracle routing" is spin. Reframe as robustness/methodology.]
[C3 DISAGREE as framed — aggregate lead fragile: appears only after regenerating 56 failures, competitor had no equivalent opportunity, reverses under strict scoring, 3–4× context budget, significance tied to partial-credit threshold. Contradiction result is the real contribution: large, significant, mechanistically aligned, heal-independent. Center C3 on it. Substrate changes risk benchmark overfitting unless developed outside the 700 judged questions.]
[Attacks ranked: 1 test-set adaptation; 2 movable SOTA denominator / protocol shopping; 3 attribution unsupported; 4 oracle contamination; 5 retry asymmetry; 6 metric selection after outcome inspection; 7 cost-normalized inferiority; 8 permissive-judge optimization; 9 no uncertainty on tiny deltas; 10 benchmark saturation; 11 contradiction subset possibly post-hoc; 12 artifact asymmetry.]
[Do differently: stop targeting SOTA-on-all-three; three falsifiable claims instead. Genuinely untouched eval split for LME (external hold-out / pre-registration / nested CV by conversation; do NOT fabricate pseudo-held-out splits now). Separate comparison from causal attribution via factorial ablation — key question: does conflict preservation cause the contradiction gain. Normalize BEAM inference policy (pre-declare attempts/failure definitions/budgets), report first-attempt + retry-normalized + full ordinal distribution + cost. Bridge BEAM protocol islands with a cross-protocol judge matrix over the same frozen answer artifacts + judge-agreement stats (kappa, favoritism analysis, blinded human sample). Treat C1 as protocol-sensitivity: the result you want is ordering stability across protocols. Pre-register hypotheses and correction rules. Add cost-constrained Pareto frontiers.]
[Headline: "SOTA under every protocol where a like-for-like comparison exists" NOT defensible — LME production-legal is below incumbent, BEAM depends on retry+threshold, LoCoMo unfinished, denominator selective. Strongest defensible now: "Conflict-preserving raw-turn memory substantially improves contradiction handling, while a protocol-fidelity audit shows that benchmark rankings are highly sensitive to judges, routing metadata, retries, and inference budgets." If C1 succeeds under both judges matched: "Conflict-preserving raw-turn memory leads matched LoCoMo and BEAM comparisons, with a 23-point contradiction gain, while exposing substantial protocol-induced inflation in conversational-memory benchmarks."]
[Ranked proposals: 1 unseen external eval set; 2 BEAM symmetric retry rerun; 3 cross-protocol judge matrix on frozen artifacts; 4 causal ablations isolating conflict preservation; 5 pre-registration; 6 Pareto frontiers; 7 correct oracle-routed LME prominently; 8 contradiction/update as primary mechanistic result; 9 all BEAM ordinal outcomes; 10 protocol instability as central evaluation contribution. "The credible paper is not 'we won three leaderboards.' It is 'leaderboard claims in conversational memory are underidentified, and conflict preservation produces one large, reproducible capability gain under controlled comparison.'"]

99
docs/paper/2026-07-11-protocol-freeze.md Normal file
View File

@@ -0,0 +1,99 @@
# Protocol & Claim Freeze (2026-07-11)
Pre-registration for the three planned races. Everything below is fixed **before**
any race result exists. Companion: `2026-07-11-fable-codex-consensus.md` (§0.5 smoke
tier, go/no-go). Calibration evidence: `KorroResearch/benchmarks/locomo-mem0-parity-2026-07/calib/`
(`panel_input.jsonl`, `panel_verdicts.jsonl`, `analyze_panel.py` — this smoke, Part 2).
**Global rules (all three races).**
- **One evaluation run per frozen config.** No reruns-until-win. If a run is voided
it is voided for a *disclosed operational reason* (crash, auth failure), not because
of its score, and the void is logged.
- **Retry policy.** Max **1** regeneration, triggered only on a *literally empty*
answer string (finish_reason truncation or empty content). Applied symmetrically
wherever we control the pipeline. **First-attempt result is the reported primary;**
retry-normalized is a disclosed secondary. Competitor artifacts we do not control
(mem0 released answers) get first-attempt-primary treatment with the asymmetry
disclosed — a true symmetric rerun of a managed platform is infeasible.
- **Statistics.** Paired tests only, **cluster-aware at the conversation level**
(questions are nested in conversations; question-level independence is false).
Report a 95% CI whose lower bound must clear the go threshold. Conversation-cluster
bootstrap (10k resamples) is the primary interval; McNemar is reported but its naive
p-value is treated as anti-conservative and never the sole basis for a claim.
- **Multiplicity.** Three benchmarks × one primary metric each = 3 primary tests.
Holm–Bonferroni across the 3 primaries; per-benchmark secondaries are descriptive,
not claim-bearing.
- **Systems enter only with a frozen, hashed answer artifact.** A system with no
reproducible per-question answer file does not enter the matrix.
---
## Race 1 — LoCoMo (4 systems × 2 judge protocols)
| Item | Freeze |
|---|---|
| Dataset | snap-research `locomo10.json` (Maharana et al., ACL 2024). Canonical build `locomo-1540` |
| Dataset hash | build `39e415e2f3a0fa1bd3cb1804a58d0b440b50d3070b2100698437e4ec402a5b24`; canonical instance_count 1531 |
| Evaluated set | **1540 rows** (mem0's own denominator: cat1=282, cat2=321, cat3=96, cat4=841). The 1531→1540 gap = 9 duplicate-question rows mem0 scores separately; we match their denominator for parity. Cat 5 adversarial (446) **excluded** |
| Answer artifacts | Ours = `locomo-7lane-w4-judgments-N1540.jsonl` (`answer_content`), hash `1252fbde90613ebb5622f6724def7e60a433f4d8bf754829c0a0592972f079fa`. mem0 = released per-question verdicts `mem0_perq_verdicts.json` (their answers held, joined on `conv_idx` + normalized question, gold tiebreak on 11 dup groups). Systems 3 & 4 enter only if a frozen answer file exists; else the matrix is 2×2 and reported as such |
| Answerer | Per-system, frozen in each answer artifact. **Not** re-answered for this race — judge-only |
| Protocol axis (the 2) | **P1 mem0-lenient:** `mem0-memory-benchmarks/benchmarks/locomo/prompts.py` `JUDGE_PROMPT` (partial-credit: ≥1 gold item ⇒ CORRECT), gold via `preprocess_answer` (cat-3 semicolon split). **P2 Memori-strict:** `memori-repo/benchmarks/02_run_benchmark.ipynb` cell 2 `ACCURACY_PROMPT` (verbatim), raw gold |
| Judge model | P1 = gpt-5 (arm c reference) and gpt-4.1-mini (production arm b), both frozen; P2 = gpt-4.1-mini. Judge model held constant within a protocol column |
| Primary metric | Judge accuracy (J-score), cats 1-4, on the 1540 |
| Statistic | Paired McNemar per system-pair **plus** conversation-cluster bootstrap 95% CI (10 conversations) |
| Claim shape | Ordering **stability across the two protocols**, not "we win under the lenient judge." A protocol-specific win is reported as protocol-specific |
The result this race is allowed to support: *ranking is (or is not) stable when the
judge protocol is swapped.* Cross-protocol number-vs-number comparison is licensed only
because S0 calibration cleared the substitution gate (below).
## Race 2 — LongMemEval (one harness: ours + Mastra + mem0)
| Item | Freeze |
|---|---|
| Dataset | `longmemeval_s_cleaned` (HF `xiaowu0162/longmemeval-cleaned`; Wu et al. 2024, arXiv:2410.10813) |
| Dataset hash | build `a8a99545d77a236e3c7aa1f5d0ccfd94d4bcc5c2d5adbd19f938aba844586c56`; jsonl `f21f62027a10e7e08fecdb3386c5ec83f409a5e92056a48be68bbb5473e7f262` |
| Evaluated set | **500 instances, including the 30 abstention (`_abs`) questions.** No-session instances excluded in canonical build. All three systems run on the identical 500 through **our** harness |
| Answerer | Frozen per system; production-legal config (no oracle routing). Oracle numbers, if shown, are ablation-only and never carry "exceeds incumbent" language |
| Judge | LongMemEval reference judge, single model+prompt frozen (source path recorded in run manifest), applied identically to all three systems |
| Retrieval depth/budget | Held identical across the three systems (same top-k, same token budget); disclosed |
| Primary metric | **Micro accuracy** over 500 |
| Statistic | Paired, cluster-aware (question-session) bootstrap 95% CI; Holm-corrected across the 3-benchmark family |
| Claim shape | Competitive aggregate + per-type breakdown. **No LME SOTA claim** — the 500 informed method development; one final run does not restore test-set independence |
## Race 3 — BEAM (AMB harness race)
| Item | Freeze |
|---|---|
| Dataset | `mohammadtavakoli78/BEAM` (Tavakoli et al. 2024, arXiv:2510.27246, ICLR 2026) |
| Dataset hash | `beam-1M`: `cc81ed8f7624261a2fa43a335eb46159b7665074b82ddfb4e2c0783fbf2caa46`, 700 questions across **35 conversations**, 1M chat size |
| Evaluated set | **700 questions, dedup-first.** Full ordinal outcome retained per question |
| Answerer | Ours + control lanes (BM25 / oracle) through the AMB harness; frozen config |
| Judge | Nugget-graded judge; rubric nuggets carried per question; judge model+prompt frozen in run manifest. Judge ported to grade 0 / 0.5 / 1 per nugget |
| Retry policy | Max-1-on-empty, first-attempt primary (global rule). BEAM's earlier 56-failure regeneration is **not** repeated |
| Primary metric | **Mean nugget score** (0/0.5/1) over 700 |
| Secondary | Pass rate; **full ordinal distribution** (rate of 0, 0.5, 1); cost/token per question |
| Statistic | **Conversation-cluster bootstrap** 95% CI over 35 clusters (question-level McNemar is invalid here — 700 nested in 35). Aggregate win requires cluster-aware CI lower bound > 0. The contradiction-subset result (+23pp) is the headline and is reported with its own cluster CI |
| Claim shape | Contradiction-handling gain is primary; aggregate is descriptive with full distribution + cost. No permissive-threshold aggregate claim |
---
## Go / No-Go (from consensus §0.5)
- **GO** = all load-bearing smokes green (S0, S2, S5, S6′ paired, S7) **and** S0 calibration clean.
- **S7:** green ≥ +3pp vs best comparable lenient result; red if tie/lose/<1pp.
- **S6′:** Mastra repro < 41/50 kills comparability; ours trailing ≥ 3/50 = red.
- **S4:** red if mem0-OSS leads ≥ 5pp under strict LoCoMo.
- **S2:** BEAM aggregate dies unless cluster-aware CI lower bound > 0.
- **FALLBACK** to the contradiction+audit paper if any load-bearing red, two grays, or
judge calibration fails. **No averaging failures across benchmarks.**
## Claim freeze
Headline is **not** "SOTA on three leaderboards." It is: *conflict-preserving raw-turn
memory delivers a large, reproducible contradiction-handling gain under controlled
comparison, while a protocol-fidelity audit shows conversational-memory rankings are
underidentified (judges, routing metadata, retries, budgets).* Conditional upgrade only
if the LoCoMo matrix shows stable ordering under both protocols: "leads matched LoCoMo
and BEAM comparisons." Any comparison that cannot be expressed as a single frozen
protocol above is out of scope for this package.

49
docs/plans/E-4-OSS-EXTRACTION-VERIFIED-2026-05-20.md
View File

@@ -1,6 +1,16 @@
# E-4 — OSS Subtree-Split Extraction Verified (2026-05-20)
## Status: ✅ Verified-working, ready for Day 0 push
## Status: ⛔ SUPERSEDED — DO NOT FOLLOW
This is a historical verification record. Its former publication instructions
were invalidated by the 2026-06-12 drift analysis: raw subtree branches have the
wrong public-repository layout and can contain Waggle-only files plus interleaved
`install_audit` logic. They are inspection inputs only, never publication sources.
The current authoritative process is in `AGENTS.md` §7.5 and
`packages/hive-mind-core/CONTRIBUTING.md`: prepare a maintainer-curated
forward-port in the OSS checkout, strip every documented exclusion, adapt layout
and imports, review the complete diff, then run `scripts/oss-drift-check.sh`.
`scripts/oss-subtree-split.sh` was previously listed as "scaffold done, code copy TODO" (CR-6 in `BACKLOG-CONSOLIDATED-2026-04-17.md`). This session ran the script locally against all 12 `packages/hive-mind-*` packages and confirmed:
@@ -25,11 +35,13 @@ oss-hive-mind-shim-core-export
oss-hive-mind-wiki-compiler-export
```
All 12 are local-only refs — **NOT pushed to any remote** (per the script's design: `git push` is a manual step).
All 12 were local-only refs and were not published. Their existence did not make
them safe publication artifacts.
## Regression guards added
`tests/oss-subtree-split.test.ts` — 44 static-analysis tests that lock down:
`tests/oss-subtree-split.test.ts` now includes static guards plus an executable
temporary-repository rollback test. It locks down:
- Script exists with bash shebang + `set -euo pipefail`
- Dynamic `packages/hive-mind-*` discovery (Wave 2/3 auto-inclusion)
@@ -39,29 +51,15 @@ All 12 are local-only refs — **NOT pushed to any remote** (per the script's de
The first run flagged a stale forbidden entry (`cowork/` — listed in the script but no longer at the repo root). Removed; commit landed in this same change.
## What "Day 0 push" means
## Invalidated publication guidance
For each export branch, the maintainer (Marko) pushes to a remote:
```bash
# Either per-package, to dedicated OSS mirror repos:
git push <oss-mirror-remote> oss-hive-mind-core-export:main
# Or to a consolidated repo as a subdirectory:
git push origin-hive-mind oss-hive-mind-core-export:packages/hive-mind-core
```
Per `packages/hive-mind-core/CONTRIBUTING.md`, the consolidated-repo model is at `github.com/marolinik/hive-mind`. Adding a remote for that:
```bash
git remote add origin-hive-mind https://github.com/marolinik/hive-mind.git
git push origin-hive-mind oss-hive-mind-core-export:main
# ... repeat per package, mapping to its directory in the consolidated repo
```
The former “Day 0” raw-branch publication commands were removed because they
could expose proprietary content and cannot produce the curated mirror layout.
Do not reconstruct or use them from repository history.
## What's NOT done (deliberate)
- **No remote pushes.** The script + this verification produce export branches; pushing is manual + Day-0-gated per the OSS launch playbook.
- **No remote publication.** The script produces local inspection branches only.
- **No CI workflow that runs splits.** Each split processes hundreds-to-thousands of commits and takes minutes per package; running this on every PR would be wasteful. The static-guard test (`oss-subtree-split.test.ts`) catches the regressions that matter (forbidden list drift, script syntax, package-shape) without paying the split cost.
- **No automatic reverse-sync.** OSS upstream changes don't flow back automatically; that's a manual cherry-pick following `.github/sync.md`.
@@ -71,7 +69,7 @@ git push origin-hive-mind oss-hive-mind-core-export:main
# Run the static guards (fast):
npx vitest run tests/oss-subtree-split.test.ts
# Re-split + verify all 12 packages (slow — 5-10 minutes total):
# Re-split all packages for local inspection only (slow — 5-10 minutes total):
bash scripts/oss-subtree-split.sh
# Single-package re-split (fastest spot-check):
@@ -82,6 +80,7 @@ git checkout oss-hive-mind-core-export && ls
git checkout - # return
```
## CR-6 ✅ CLOSED
## CR-6 historical disposition
The original CR-6 was "hive-mind actual source extraction — scaffold done, code copy TODO." The scaffold + the working extraction mechanism + a regression guard now all exist. The remaining "code copy" step is the manual Day-0 `git push` to the OSS mirror, which is correctly out of session scope.
The original CR-6 proved isolated-history extraction, not a safe OSS release
mechanism. Any future mirror release remains a separate curated-forward-port task.

14
docs/plans/L-17-placeholder-audit-2026-04-19.md
View File

@@ -5,14 +5,12 @@ HTML `placeholder="..."` attributes).
Grep pattern: `// (MOCK|TODO|FIXME|XXX):` or the `/* … */` equivalent.
Current count: **6** lines (revision history below). All 6 are intentional
subtree-split STUB markers in the `hive-mind-hooks-*` packages introduced
by the Phase 2 consolidation merge of `feature/hive-mind-monorepo-migration`
(2026-05-10). They flag hook implementations awaiting Wave 2/3 work; they
are load-bearing for both waggle-os main and the public hive-mind repo
subtree-split. Zero new production code may introduce a marker without
updating `tests/placeholder-audit.test.ts` and this document in the same
commit.
Current count: **0** production-path marker lines. The prior hook stub markers
were removed as the Wave 2/3 implementations landed. Zero new production code
may introduce a marker without updating `tests/placeholder-audit.test.ts` and
this document in the same commit.
Revision 2026-07-16: 6 -> 0 after the hook implementations were completed.
**Revision history:**
- 2026-04-19: 14 → 10 (initial L-17 audit + C2/C3/C4/C5 cleanup)

19
docs/plans/monorepo-migration-progress.md
View File

@@ -1,5 +1,11 @@
# Monorepo Migration Progress Log
> [!CAUTION]
> **HISTORICAL RECORD — publication guidance superseded.** Completion facts
> below describe the April 2026 migration. Any “ready to push” statement about
> raw subtree/export branches is invalid. Current OSS publication authority is
> `AGENTS.md` §7.5 and `packages/hive-mind-core/CONTRIBUTING.md`.
**Brief:** `D:/Projects/PM-Waggle-OS/briefs/2026-04-30-cc-sesija-B-hive-mind-monorepo-migration.md`
**Authority chain:**
- `decisions/2026-04-30-pre-launch-sprint-consolidation-LOCKED.md`
@@ -385,8 +391,16 @@ Closing batch authorized as a single sprint per brief Tasks B16-B27.
### §2.6 OSS subtree split prep (Tasks B20-B22)
> **CORRECTION (2026-08-22):** These tasks proved isolated-history inspection,
> not safe publication. Raw export refs have the wrong mirror layout and can
> contain private files plus interleaved `install_audit`; never publish them.
> Use a reviewed maintainer-curated forward-port.
- **B20** — `scripts/oss-subtree-split.sh` (NEW): auto-discovers `packages/hive-mind-*`, runs `git subtree split` for each, sentinel-checks for monorepo-level leaks. Idempotent (drops + recreates branches each run).
- **B21** — Local test run produced clean `oss-<package>-export` branches for all 12 hive-mind-* packages. Sentinel passes (no proprietary leak). Manual `git push` gate preserved per OSS launch playbook.
- **B21** — Historical receipt: the local run produced 12
`oss-<package>-export` branches and the then-current monorepo-level sentinel
passed. This proved branch shape only; the former manual raw-publication gate
was invalidated by the 2026-06-12 proprietary/interleaved-content analysis.
- **B22** — `.github/workflows/sync-mind.yml` + `mind-parity-check.yml`: deprecation comments added at file headers. Workflows preserved as audit-trail anchors — trigger paths invalid post-migration so workflows do not fire.
### §2.7 Tests + import paths + smoke + final acceptance (Tasks B23-B27)
@@ -426,7 +440,8 @@ All four documented in commit `9cf43b8`.
- **`feature/hive-mind-monorepo-migration` HEAD on origin:** `9cf43b8` (closing trio commit) + this progress doc commit
- **CC Sesija B sprint cumulative commits:** ~19+ commits since §2.1 close `4859b67`
- **Test count post-§2.7:** 5949 / 31 / 145 (6125 total) — 99.5% pass rate, IDENTICAL to §2.3 baseline (zero regression)
- **OSS subtree-split branches (LOCAL ONLY, ready for Day 0 manual push):** `oss-hive-mind-{core,shim-core,cli,mcp-server,wiki-compiler,hooks-claude-code}-export` + 6× `oss-hive-mind-hooks-{cursor,hermes,openclaw,codex,claude-desktop,codex-desktop}-export`
- **OSS subtree-split branches (historical local inspection refs only; never
publish raw):** `oss-hive-mind-{core,shim-core,cli,mcp-server,wiki-compiler,hooks-claude-code}-export` + 6× `oss-hive-mind-hooks-{cursor,hermes,openclaw,codex,claude-desktop,codex-desktop}-export`
**§2.5+§2.6+§2.7 STATUS:** COMPLETE.

5
docs/production-readiness/01A-FEATURE_WAVES.md
View File

@@ -1,5 +1,10 @@
# Phase 1A: Feature Wave Audit — Built vs Not Built
> **Historical snapshot — superseded.** This report records evidence as of
> 2026-03-20 and is not current launch authority. See
> [`09-LAUNCH_RECOMMENDATION.md`](./09-LAUNCH_RECOMMENDATION.md) for the current
> Windows-first Solo scope, external gates, and recommendation.
**Date**: 2026-03-20
**Auditor**: Claude (automated codebase cross-reference)
**Plan document**: `docs/plans/2026-03-19-phase9-completion-plan.md`

5
docs/production-readiness/01B-DEPLOYMENT_PHASES.md
View File

@@ -1,5 +1,10 @@
# Phase 1B: Deployment, Phases & PM Features Audit
> **Historical snapshot — superseded.** This report records evidence as of
> 2026-03-20 and is not current launch authority. See
> [`09-LAUNCH_RECOMMENDATION.md`](./09-LAUNCH_RECOMMENDATION.md) for the current
> Windows-first Solo scope, external gates, and recommendation.
**Date**: 2026-03-20
**Auditor**: Claude (automated analysis)
**Scope**: Wave 9D (Deployment), Phase 7 (KVARK), Phase 8 status, PM Features (6)

5
docs/production-readiness/02-UX_AUDIT.md
View File

@@ -1,5 +1,10 @@
# Phase 2: UX Audit — View-by-View Code Review
> **Historical snapshot — superseded.** This report records evidence as of
> 2026-03-20 and is not current launch authority. See
> [`09-LAUNCH_RECOMMENDATION.md`](./09-LAUNCH_RECOMMENDATION.md) for the current
> Windows-first Solo scope, external gates, and recommendation.
**Date:** 2026-03-20
**Auditor:** Production Readiness Automation (Phase 2)
**Scope:** All 7 views, sidebar, onboarding, Direction D compliance, emotional assessment

5
docs/production-readiness/03A-AGENT_QUALITY.md
View File

@@ -1,5 +1,10 @@
# Phase 3A: Agent Critical Path Quality Audit
> **Historical snapshot — superseded.** This report records evidence as of
> 2026-03-20 and is not current launch authority. See
> [`09-LAUNCH_RECOMMENDATION.md`](./09-LAUNCH_RECOMMENDATION.md) for the current
> Windows-first Solo scope, external gates, and recommendation.
**Auditor**: Production Readiness Review (automated)
**Date**: 2026-03-20
**Scope**: Agent loop, memory, vault, cron, connectors, sub-agents

5
docs/production-readiness/03B-SERVER_QUALITY.md
View File

@@ -1,5 +1,10 @@
# Phase 3B: Server & API Layer — Production Readiness Audit
> **Historical snapshot — superseded.** This report records evidence as of
> 2026-03-20 and is not current launch authority. See
> [`09-LAUNCH_RECOMMENDATION.md`](./09-LAUNCH_RECOMMENDATION.md) for the current
> Windows-first Solo scope, external gates, and recommendation.
**Date**: 2026-03-20
**Scope**: `@waggle/server` — local server (Fastify), routes, SSE, WebSocket, KVARK client, security middleware
**Auditor**: Claude Opus 4.6 (automated code review)

5
docs/production-readiness/03C-UI_QUALITY.md
View File

@@ -1,5 +1,10 @@
# Phase 3C: UI & Frontend Code Quality Audit
> **Historical snapshot — superseded.** This report records evidence as of
> 2026-03-20 and is not current launch authority. See
> [`09-LAUNCH_RECOMMENDATION.md`](./09-LAUNCH_RECOMMENDATION.md) for the current
> Windows-first Solo scope, external gates, and recommendation.
**Auditor**: Senior Engineer Code Review (automated)
**Date**: 2026-03-20
**Scope**: `app/src/` (Tauri desktop app) + `packages/ui/src/` (shared React component library)

5
docs/production-readiness/04A-APP_SECURITY.md
View File

@@ -1,5 +1,10 @@
# 04A Application Security Audit
> **Historical snapshot — superseded.** This report records evidence as of
> 2026-03-20 and is not current launch authority. See
> [`09-LAUNCH_RECOMMENDATION.md`](./09-LAUNCH_RECOMMENDATION.md) for the current
> Windows-first Solo scope, external gates, and recommendation.
**Date**: 2026-03-20
**Auditor**: Automated (Claude Opus 4.6)
**Scope**: Waggle desktop app + local server — CSP, vault crypto, agent tools, input validation, sessions, connectors, dangerous patterns

5
docs/production-readiness/04B-SECRETS_DEPS.md
View File

@@ -1,5 +1,10 @@
# 04B — Secret Scanning & Dependency Audit
> **Historical snapshot — superseded.** This report records evidence as of
> 2026-03-20 and is not current launch authority. See
> [`09-LAUNCH_RECOMMENDATION.md`](./09-LAUNCH_RECOMMENDATION.md) for the current
> Windows-first Solo scope, external gates, and recommendation.
**Auditor:** Claude Opus 4.6 (automated)
**Date:** 2026-03-20
**Scope:** Full codebase secret scan, git history review, npm dependency audit, .gitignore assessment

5
docs/production-readiness/05-TEST_REPORT.md
View File

@@ -1,5 +1,10 @@
# 05 - TEST & COVERAGE REPORT
> **Historical snapshot — superseded.** This report records evidence as of
> 2026-03-20 and is not current launch authority. See
> [`09-LAUNCH_RECOMMENDATION.md`](./09-LAUNCH_RECOMMENDATION.md) for the current
> Windows-first Solo scope, external gates, and recommendation.
**Date:** 2026-03-20
**Auditor:** Claude Opus 4.6 (automated, read-only)
**Scope:** All packages, integration tests, E2E tests, visual regression tests

5
docs/production-readiness/06-BUILD_REPORT.md
View File

@@ -1,5 +1,10 @@
# Phase 6: Build & Deployment Readiness Report
> **Historical snapshot — superseded.** This report records evidence as of
> 2026-03-20 and is not current launch authority. See
> [`09-LAUNCH_RECOMMENDATION.md`](./09-LAUNCH_RECOMMENDATION.md) for the current
> Windows-first Solo scope, external gates, and recommendation.
**Date**: 2026-03-20
**Auditor**: Claude Opus 4.6 (automated)
**Scope**: TypeScript compilation, Vite build, Docker, Tauri, Render.com, npx launcher, CI/CD

5
docs/production-readiness/07-ISSUE_REGISTER.md
View File

@@ -1,5 +1,10 @@
# Issue Register — Waggle V1 Pre-Production Qualification
> **Historical snapshot — superseded.** This register records findings as of
> 2026-03-20 and is not the current open-issue or launch authority. See
> [`09-LAUNCH_RECOMMENDATION.md`](./09-LAUNCH_RECOMMENDATION.md) for the current
> Windows-first Solo scope, external gates, and recommendation.
Generated: 2026-03-20 | Branch: `phase8-wave-8f-ui-ux` | Tests: 3,895 passing
---

5
docs/production-readiness/08-CONFIDENCE_MATRIX.md
View File

@@ -1,5 +1,10 @@
# Confidence Matrix — Waggle V1 Pre-Production Qualification
> **Historical snapshot — superseded.** This score records evidence as of
> 2026-03-20 and is not the current readiness score or launch authority. See
> [`09-LAUNCH_RECOMMENDATION.md`](./09-LAUNCH_RECOMMENDATION.md) for the current
> Windows-first Solo scope, external gates, and recommendation.
Generated: 2026-03-20 | Branch: `phase8-wave-8f-ui-ux`
---

206
docs/production-readiness/09-LAUNCH_RECOMMENDATION.md
View File

@@ -1,143 +1,109 @@
# Launch Recommendation — Waggle V1
# Launch Recommendation — Windows Solo
Generated: 2026-03-20
Updated: 2026-08-27
---
## Current verdict: INSTALLER/LIFECYCLE INTERNAL RC QUALIFIED; RELEASE QUALIFICATION INCOMPLETE
## Recommendation: CONDITIONAL GO
The supported launch scope is Windows Solo with Claude Code, Codex, and Hermes using
their official user-owned installations and authentication. Cursor, OpenClaw, and macOS
packaging/certification remain roadmap work and do not block this internal RC.
Ship after fixing the 8 CRITICAL issues (~6.5 hours of work). The HIGH issues are important but can be addressed in a rapid V1.0.1 patch within the first week.
This document is the release-status authority. Historical receipts are evidence only;
they do not certify a later behavior-changing revision unless a bounded no-impact review
explicitly says so.
---
## Frozen runtime candidate
## Executive Summary
- Source revision: `23ad3fa5f99bddce648b84750a41365299aeb0da`
- Source tree: `aab548f77ec64b181086664dad29c32e6bc78779`
- Integration: private Waggle PR #66, tested head
`587e259166db69ff86e806fa8393a5f8974ea0a1`, merged 2026-08-27
- Tree equivalence: the tested PR head and merge commit resolve to the same source tree
- Repository state after merge: private `main` equals `origin/main`
Waggle is a substantial, well-architected product with 3,895 passing tests, 53 agent tools, 29 connectors, 15K+ marketplace packages, and a complete feature set covering all 8 Kill List use cases. The agent loop, memory system, and vault encryption are architecturally sound. The UI underwent a recent Phase 10 rewrite that brought Tailwind adoption and Direction D palette cleanup.
The documentation-only descendant that updates this record does not replace the runtime
candidate. Before it is merged, its diff must be limited to documentation and all required
remote checks must remain green.
However, the audit uncovered **8 CRITICAL issues** (5 security, 1 stability, 2 UX) that must be fixed before any external user touches the product. The most severe: CORS is wide open (any website can call your localhost APIs), there are no React error boundaries (one render error = permanent white screen), and the streaming loading indicator is invisible (users can't tell the agent is thinking).
## Exact-current Windows installer
The good news: every CRITICAL fix is straightforward. Total estimated effort is 6.5 hours. None require architectural changes.
- NSIS artifact:
`app/src-tauri/target/x86_64-pc-windows-msvc/release/bundle/nsis/Waggle_0.2.0_x64-setup.exe`
- Size: 102,923,936 bytes
- SHA-256: `7BFA9F9B13633A51CD3336B42E3EF904B7F7A568C6DEE4F6CED967CBD4F40A59`
- Internal signer: `CN=Egzakta Internal Pilot, O=Egzakta Group, C=RS`
- Signer thumbprint: `E2F028541E7A4D1FE80FFFF02079060D36579846`
- RFC 3161 timestamp authority: DigiCert SHA256 RSA4096 Timestamp Responder 2025 1
- Trust classification: internal pilot only; the self-signed root is not public trust
---
Clean-profile certification passed **64/64** checks in 450.199 seconds:
## What's Ready (Strengths)
- Receipt:
`output/installer-certification/23ad3fa5-20260827T123917Z-exact-main-clean-profile/windows-installer-certification.json`
- Receipt SHA-256:
`AC2A1C54119E28CC22DA931EB43E2815862B01832DD6097CE03F8F79D9D3DF4D`
- Receipt source and bundled-sidecar revision: exact `23ad3fa5`
- Tier: FREE/Solo
- Managed model: `qwen2.5:0.5b`
- Managed-model digest:
`sha256:a8b0c51577010a279d933d14c2a8ab4b268079d44c5c8830c0a93900f1827c67`
1. **Solid agent core** — 53 tools, loop guards, injection scanning, approval gates, sub-agent orchestration. 1,272 tests on the agent package alone.
The receipt proves silent install, bundled Node/npm and offline package execution,
first boot, in-process embeddings, built-in proxy/session authentication, workspace and
memory persistence, managed runtime/model pull and chat, proxy-restart chat, same-version
repair, relaunch, data preservation, Exit/owned-process cleanup, uninstall, registry
cleanup, and preservation of external `.hive-mind` and `.ollama` roots. It also proves
that developer Node.js, Python, Docker, external LiteLLM, and a separately installed
Ollama are not prerequisites.
2. **Complete feature set** — All 8 Kill List use cases work. Workspace memory, connectors, marketplace, personas, cron, swarm protocol, capability packs, onboarding with memory import.
## Integrated test and security evidence
3. **Good test coverage** — 3,895 tests across 277 files, zero failures. Every major package has dedicated test suites with behavior-focused assertions and realistic mocks.
- PR #66: every blocking remote check passed — primary CI, Playwright smoke and full
E2E, Windows and both macOS Tauri verification targets, Wave 1, and Hive Mind
install/smoke on Windows, Ubuntu, and macOS.
- Exact-current dependency audits: 0 Critical and 0 High in both full and production
dependency trees. Lower-severity maintenance remains tracked.
- The security-hardening integration covers workspace path/link boundaries, hook/database
hard-link boundaries, normalized ingress, atomic consolidation/cognify, deprecated-frame
search exclusion before limits, and knowledge-graph provenance.
- Hosted signing policy and workflow tests remain green, but no publicly trusted hosted
artifact has been produced.
4. **Security fundamentals** — AES-256-GCM vault, parameterized SQL everywhere, path traversal protection, CLI allowlists, DOMPurify HTML sanitization, SecurityGate for marketplace.
## Persona, router, and authentication evidence
5. **Deployment infrastructure** — Tauri Windows installer built (8.2MB), Docker production compose, Render.com blueprint, GitHub Actions CI/release pipeline.
The historical ten-persona collection at `4c712ff6` contains 30/30 results at or above
95/100 after documented independent semantic adjudication. It is not relabeled as an
exact-current deterministic seal: PR #66 changed memory behavior, so public release
qualification requires either a fresh exact-candidate collection or an explicit bounded
semantic-impact attestation.
6. **Product polish** — 8 personas, dark/light mode, keyboard shortcuts, global search, workspace hue colors, onboarding wizard, tool card transparency, approval gates inline in chat.
Smart-router primary, compact-tool-context, durable-budget/fallback, and official-user-auth
canaries for Claude Code, Codex, and Hermes remain scoped historical evidence. No PR #66
change altered provider credential ownership or copied/read provider credential files.
---
## Hive Mind repository state
## Must Fix Before Launch (CRITICAL — ~6.5 hours)
Curated public-mirror hardening PR #53 merged to `marolinik/hive-mind` `master` as
`3410327800db3ea23f875d547a0c7f4d08826b7e`; Linux, Windows, macOS, and Ubuntu
first-run smoke passed. The immutable drift checker still reports reviewed blockers and
one unreviewed difference, with zero forbidden exports. Therefore the Windows Solo RC is
not blocked, but the next Hive Mind package release remains a separate maintainer-curated
operation. Raw subtree publication remains forbidden.
### Security (4 hours)
## Public GO blockers
| # | Issue | Fix | Time |
|---|-------|-----|------|
| 1 | **CORS allows any origin** — any website can call all Waggle APIs | Change `origin: true` to `origin: ['http://localhost:1420', 'tauri://localhost']` (or your Tauri webview origins). Fix SSE hijack endpoints to use the same allowlist. | 1.5 hr |
| 2 | **Server CSP has `unsafe-eval` + `unsafe-inline`** | Remove both. If scripts break, use nonces or hashes instead. | 30 min |
| 3 | **OAuth refresh tokens stored plaintext** | Encrypt refresh tokens the same way access tokens are encrypted in `setConnectorCredential()`. | 1 hr |
| 4 | **Verify API key revoked** | Go to Anthropic dashboard, confirm the key from commit `c29d75f` is revoked. Delete local branch `phase6-capability-truth`. | 30 min |
Public release may be called **GO** only after all of these are closed for the approved
release-tag commit:
### Stability (2 hours)
1. A protected hosted build produces a publicly trusted Authenticode artifact.
2. The managed Codex Security workflow produces a sealed Deep Security report with no
unresolved Critical or High findings.
3. Current persona qualification is sealed by a fresh exact-candidate receipt or an
independently reviewed bounded semantic-impact attestation.
4. Smart-router and Claude Code/Codex/Hermes official-auth qualification is either rerun
on the exact candidate or covered by a concrete independently reviewed no-impact
attestation.
5. Protected release-tag checks are green and the exact artifact hashes are recorded.
| # | Issue | Fix | Time |
|---|-------|-----|------|
| 5 | **Zero error boundaries** | Add `<ErrorBoundary>` wrapping each view in App.tsx, plus one at the app root. Use react-error-boundary or a simple class component. Show "Something went wrong" with a retry button. | 2 hr |
### UX (30 minutes)
| # | Issue | Fix | Time |
|---|-------|-----|------|
| 6 | **Streaming indicator invisible** | The loading dots use BEM CSS classes with no definitions. Either add the CSS or replace with Tailwind `animate-pulse` dots. | 15 min |
| 7 | **SplashScreen wrong palette** | Replace `#1a1a2e`/`#16213e`/`#0f3460` with Direction D tokens. Change `#f5a623` to `#d4a843`. | 15 min |
---
## Ship-Week Fixes (HIGH — ~28 hours, V1.0.1)
**Security hardening (first 2 days):**
- Approval gates: change auto-approve to auto-deny on 5min timeout (15 min)
- WebSocket authentication: require session token on `/ws` connect (2 hr)
- Team WebSocket: validate JWT instead of trusting userId param (2 hr)
- Replace `xlsx` with `exceljs` to fix prototype pollution (2 hr)
- Generate Tauri updater keypair and set pubkey (30 min)
**Agent loop safety (day 3):**
- Cap rate-limit retries (max 3, then fail gracefully) (1 hr)
- Add token budget enforcement with configurable limit (2 hr)
- Parameterize sqlite-vec SQL interpolation (30 min)
**Frontend stability (days 3-5):**
- Add code splitting with `React.lazy()` for 7 views (2 hr)
- Deduplicate SSE connections (1 hr)
- Fix eventBus.removeAllListeners to scope per-client (1 hr)
- Fix light theme breakage across components (2 hr)
**Build fixes (day 5):**
- Fix npx waggle: compile .ts entry, resolve workspace deps (2 hr)
- Add non-root user to Docker (30 min)
- Fix CI branch target master→main (15 min)
- Clean up 87 TypeScript errors (2 hr)
---
## Known Limitations (Ship Anyway)
These are acceptable for V1 and can be improved iteratively:
1. **No browser E2E tests** — Unit/integration coverage is strong (3,895 tests). True browser automation (Playwright user journeys) is a V1.1 investment. Screenshot baselines exist.
2. **Monolithic App.tsx (1300 lines)** — Works but hard to maintain. Refactoring into feature-specific providers is a V1.1 task that won't affect users.
3. **No React.memo optimization** — The app performs fine at current scale. Memoization is premature optimization until profiling shows problems.
4. **macOS build not configured** — DMG, code signing, notarization require an Apple Developer account. Windows installer works. Ship Windows-first, add macOS in V1.1.
5. **Direction D at ~78%** — The Phase 10 UI rewrite made massive progress (371→19 inline styles). Remaining 22% is polish, not broken functionality.
6. **KVARK client not wired** — KVARK integration (Phase 7) is library code + tests. Not wired into the running server because KVARK itself needs its HTTP API deployed first. This is expected — it's the Enterprise tier path.
7. **Conversation history unbounded** — At typical usage (10-50 turns/session), this isn't a problem. Add context window management for power users in V1.1.
---
## Post-Launch Priority Queue
### First Week (V1.0.1)
1. All HIGH security fixes (approval timeout, WebSocket auth, xlsx, updater pubkey)
2. Agent loop safety (retry cap, token budget)
3. Frontend stability (code splitting, SSE dedup, error boundaries for remaining components)
4. Light theme fixes
### First Month (V1.1)
1. Browser E2E test suite (Playwright user journeys)
2. React component rendering tests
3. App.tsx decomposition (extract providers/hooks)
4. macOS build + code signing
5. npx waggle publishable package
6. CI pipeline expansion (Docker, lint, security scan)
7. Direction D compliance to 95%+
### First Quarter (V1.2)
1. KVARK server-side wiring (when KVARK HTTP API ready)
2. Performance profiling + React.memo optimization
3. Context window management for long conversations
4. Token budget UI (user-configurable spend limits)
5. Full accessibility audit (WCAG 2.1 AA)
---
## Verdict
**CONDITIONAL GO** — Fix the 8 CRITICALs (6.5 hours), then ship. The product is feature-complete, well-tested, and architecturally sound. The critical issues are configuration mistakes, not design flaws. Every fix is surgical and low-risk.
The foundation is strong. Ship it.
Until then, the installer is suitable for controlled internal testing, not public
distribution, and Waggle must not be described as publicly production-ready or GO.

73
docs/production-readiness/10-SECURITY_REVIEW_2026-08-11.md Normal file
View File

@@ -0,0 +1,73 @@
# Windows Solo security review — refreshed 2026-08-27
## Status
The integrated Windows Solo source candidate is
`23ad3fa5f99bddce648b84750a41365299aeb0da` (private PR #66). Its tested PR
head and merge commit have the same source tree
`aab548f77ec64b181086664dad29c32e6bc78779`.
This is an evidence-backed source, dependency, CI, and installed-runtime review. It is
**not** a substitute for the still-missing sealed managed Deep Security report and does
not confer public release approval.
## Verified controls
| Surface | Current evidence | Result |
|---|---|---|
| Workspace paths and lifecycle | Strict workspace IDs, canonical containment, identity-matching configs, link/hard-link rejection, and fail-closed list/get/delete behavior; focused executable tests | Pass |
| Hook/database boundary | Existing database/config entries and SQLite companions reject links or hard links while preserving first-use and WAL behavior | Pass |
| External memory ingress | Normalization and injection scanning are applied before persistence across supported ingress paths; bypass-focused regressions are included | Pass |
| Search | Punctuated identifiers use bounded precise fallback; deprecated frames are excluded before keyword, LIKE, whole-vector, and chunk-vector limits; alternate-lane starvation regressions are covered | Pass |
| Consolidation/cognify | Supersede operations are atomic, preserve provenance, and fail closed on partial mutation | Pass |
| Knowledge graph | Relationship provenance and source-frame boundaries are preserved and tested | Pass |
| Local server and tools | Loopback/session authentication, origin controls, SSRF/DNS/socket-pinning defenses, bounded external input, command-vector execution, and fail-closed shim handling are covered by focused and remote gates | Pass |
| Provider authentication | Historical scoped canaries show Claude Code, Codex, and Hermes using official user-owned authentication with no provider credential-file reads/copies; exact-current carry-forward still needs a concrete no-impact attestation or rerun | Historical evidence; current qualification open |
| Packaged runtime | Exact-current internal-pilot NSIS passed 64/64 clean-profile install, boot, managed-model, repair, relaunch, Exit/cleanup, and uninstall checks | Pass internal RC |
| Dependency severity | Exact-current full and production audits contain 0 Critical and 0 High findings | Pass Critical/High gate |
## Exact installer evidence
- Installer SHA-256:
`7BFA9F9B13633A51CD3336B42E3EF904B7F7A568C6DEE4F6CED967CBD4F40A59`
- Certification receipt:
`output/installer-certification/23ad3fa5-20260827T123917Z-exact-main-clean-profile/windows-installer-certification.json`
- Receipt SHA-256:
`AC2A1C54119E28CC22DA931EB43E2815862B01832DD6097CE03F8F79D9D3DF4D`
- Managed model: `qwen2.5:0.5b`
- Managed-model digest:
`sha256:a8b0c51577010a279d933d14c2a8ab4b268079d44c5c8830c0a93900f1827c67`
- Certification checks: 64 passed, 0 failed
The certifier verified source and sidecar provenance, bundled runtime/npm, clean offline
execution, default Solo onboarding, in-process embeddings, built-in proxy liveness,
session authentication, managed-model pull/chat, proxy-restart chat, repair and data
preservation, relaunch, cleanup/uninstall, and unchanged external `.hive-mind`/`.ollama`
roots. No Waggle-owned process or certificate test profile remained after completion.
## Remote integration evidence
PR #66 passed primary CI, Playwright smoke and full E2E, Windows and both macOS Tauri
verification targets, Wave 1, and Hive Mind install/smoke on Windows, Ubuntu, and macOS.
The full local Waggle Vitest suite, agent/server/app typechecks, lint, and diff checks also
completed successfully before integration.
Hive Mind PR #53 passed Linux, Windows, macOS, and Ubuntu first-run smoke before merge as
`3410327800db3ea23f875d547a0c7f4d08826b7e`.
## Residual risk and public release blockers
- The installer is signed by `CN=Egzakta Internal Pilot`, a private self-signed identity.
Its DigiCert timestamp validates the signing pipeline but does not provide public trust.
- No sealed managed Codex Security report exists; this audit host used a disabled
permission profile. No failed or unsealed attempt is interpreted as a no-findings result.
- The immutable Hive Mind drift baseline reports 22 known reviewed blockers and one
unreviewed difference, with zero forbidden exports. These block the next OSS package
release, not this private Windows Solo internal RC.
- Current persona qualification still needs a fresh exact-candidate seal or an independent
bounded semantic-impact attestation because PR #66 changed memory behavior.
Public GO requires publicly trusted Authenticode, a sealed exact-candidate managed Deep
Security report with no unresolved Critical/High findings, current persona qualification,
fresh or explicitly attested smart-router and official-auth qualification, and green
protected release-tag checks.

115
docs/production-readiness/11-HIVE-MIND-PARITY-AUDIT-2026-08-13.md Normal file
View File

@@ -0,0 +1,115 @@
# Hive Mind mirror parity audit — refreshed 2026-08-27
## Verdict
The curated Hive Mind mirror does not block the private Windows Solo internal RC. The
Waggle monorepo remains the sole source of truth for the memory substrate.
The next public Hive Mind package release is **not yet approved**. The mirror is clean,
merged, and cross-platform green, but the immutable drift baseline still contains known
reviewed blockers and one unreviewed difference. It must not be described as fully
synchronized or release-ready.
## Exact inputs
- Waggle runtime/source candidate:
`23ad3fa5f99bddce648b84750a41365299aeb0da`
- Waggle source tree:
`aab548f77ec64b181086664dad29c32e6bc78779`
- Hive Mind `master`:
`3410327800db3ea23f875d547a0c7f4d08826b7e`
- Hive Mind integration: PR #53, tested head
`f5072efbf3f2d13247bb91505d99403acbdf11c6`
- Comparison:
`node scripts/oss-drift-check.mjs D:/Projects/hive-mind`
- Result: exit 1, fail closed
The Waggle documentation-only descendant does not change either compared source tree.
## Immutable baseline result
| Classification | Count | Meaning |
|---|---:|---|
| Parity | 5 | Reviewed paths with the expected canonical/OSS state |
| Reviewed adaptations | 38 | Intentional layout, import, logger, branding, or OSS-architecture differences |
| Known reviewed blockers | 22 | Paths still requiring maintainer-curated reconciliation before an OSS release |
| Unreviewed differences | 1 | A changed path not yet classified against the baseline |
| Forbidden exports | 0 | No prohibited Waggle-only path or marker was found in the mirror |
The single unreviewed difference is `harvest/raw-turns.ts`. It must be classified and
either reconciled or deliberately re-baselined through maintainer review before release.
The five parity paths are:
- `harvest/claude-code-adapter.ts`
- `harvest/decision-derivation.ts`
- `harvest/stable-id.ts`
- `mind/erasure.ts`
- `mind/inprocess-reranker.ts`
## Known reviewed blockers
The baseline currently marks 19 paths `reconcile`:
- Harvest: `dedup.ts`, `extract-kg-entities.ts`, `pipeline.ts`,
`url-adapter.ts`
- Runtime/API: `index.ts`, `workspace-manager.ts`, `multi-mind-cache.ts`
- Mind: `db.ts`, `embedding-provider.ts`, `frames.ts`, `identity.ts`,
`inprocess-embedder.ts`, `knowledge.ts`, `llm-extractor.ts`,
`raw-archive.ts`, `raw-detail-lane.ts`, `schema.ts`, `search.ts`, and
`suppression.ts`
Three more blockers are marked `product-curation`: `hook-runtime.ts`,
`mind/supersede.ts`, and `multi-mind.ts`. They require an explicit reviewed decision
about what remains Waggle-only and what generic subset, if any, should be adapted for the
public mirror. The approved decision must then reclassify them out of the blocker category.
A blocker in this inventory means the mapped files are not yet approved for the next OSS
release; it does not by itself assert a runtime vulnerability. Every blocker needs
canonical-first review. Only reconcile/exported paths require public-layout adaptation and
focused port tests; product-curation paths may instead remain Waggle-only after explicit
review. Every resolution requires a new baseline receipt.
## Work integrated on 2026-08-27
Hive Mind PR #53 added the curated search-boundary hardening needed for punctuated
identifiers and deprecated-candidate filtering before keyword, LIKE, whole-vector, and
chunk-vector limits. The implementation closed concatenated-token, malformed-query, CJK,
and alternate-vector-lane bypasses with executable regressions.
The PR passed build/test on Ubuntu, Windows, and macOS plus Ubuntu first-run smoke before
merge. A transient Ubuntu dependency-download `ECONNRESET` was rerun once and passed;
the deterministic code/test result was green.
This narrows the drift but does not erase the remaining `mind/search.ts` curated
difference, so the immutable checker correctly continues to fail closed.
## Proprietary exclusion boundary
Raw subtree publication is forbidden. A release curation must continue to exclude:
- `vault.ts`
- `mind/evolution-runs.ts`
- `mind/execution-traces.ts`
- `mind/improvement-signals.ts`
- `compliance/**`
- Waggle-only `install_audit` schema/migration fragments interleaved in shared files
The current scan found zero forbidden exports. That is a necessary safety gate, not proof
that all public-worthy changes have been reconciled.
## Release rule
Before the next Hive Mind package release:
1. Classify `harvest/raw-turns.ts`.
2. Reconcile or explicitly re-baseline every known blocker in small reviewed phases.
3. Run focused tests for each curated port and the complete Hive Mind build/test suite.
4. Run Linux, Windows, macOS, and first-run package smoke.
5. Re-run the immutable drift checker and require exit 0: zero known blockers, zero
unreviewed differences, and zero forbidden exports. Any approved product-curation
decision must first be represented by reviewed baseline reclassification.
6. Preserve provenance showing that generic substrate work originated in Waggle first.
Until those gates pass, `master` is a clean integrated development baseline, not a sealed
new OSS package release.

6
docs/production-readiness/2026-07-03-release-audit.md
View File

@@ -1,5 +1,11 @@
# Waggle OS — Production Readiness Audit (2026-07-03)
> [!CAUTION]
> **HISTORICAL SNAPSHOT — NOT CURRENT SHIP AUTHORITY.** Findings and grades
> below describe the cited July baseline. Use
> `docs/production-readiness/09-LAUNCH_RECOMMENDATION.md` for current scope,
> evidence, open gates, and release verdict.
**Method:** 10 parallel principal-engineer audit lanes (opus, high effort), each required to cite `file:line` evidence it actually read. Baseline at audit time: HEAD `78660ab5` on `main`, vitest 8063/8063 green, lint 0, tsc 0 (agent/server/app), `build:all` clean, git history secret-scan CLEAN (all key-shaped strings are `detectSecrets()` fixtures).
**Subsystem grades:** Build **D** · CI/CD **C** · Docs/DX **C** · Deps/Config **C** · Agent-runtime **C** · Security **B** · Testing **B** · Server-API **B** · Frontend **B** · Memory-substrate **B**.

6
docs/production-readiness/2026-07-03-release-signoff.md
View File

@@ -1,5 +1,11 @@
# Waggle OS — Production Readiness Sign-off (2026-07-03)
> **Historical snapshot — superseded.** This document preserves the 2026-07-03
> assessment but is not current ship authority. Its “production-ready” and
> signing-only conclusions no longer apply. Use
> [`09-LAUNCH_RECOMMENDATION.md`](./09-LAUNCH_RECOMMENDATION.md) for the active
> Windows Solo gate and its exact-HEAD evidence requirements.
> Engineering-director pass driven by a 10-lane parallel audit, executed as prioritized remediation streams (Opus implementation agents, Fable orchestration/QA). Companion to [`2026-07-03-release-audit.md`](./2026-07-03-release-audit.md).
## Session commit ledger (17 commits on `main`, from baseline `a1fad4f8`)

85
docs/production-readiness/2026-07-16-audit.md Normal file
View File

@@ -0,0 +1,85 @@
# Waggle OS production-readiness audit — 2026-07-16
> **Historical snapshot — superseded.** This document records the state on
> 2026-07-16 and is not current ship authority. Use
> [`09-LAUNCH_RECOMMENDATION.md`](./09-LAUNCH_RECOMMENDATION.md) for the active
> Windows Solo gate. Cursor and OpenClaw are now roadmap-only; macOS
> certification is deferred.
## Executive verdict
The branch is not yet production-ready for the requested claim of a fully
Docker-independent Solo install or a verified 9.5/10 result across ten fresh
personas. It is materially improved and buildable: Windows sandbox/process
handling is hardened, chat schemas are bounded before model serialization, and
artifact/CLI writes now require approval.
## Verified changes
- Windows workspace/path, environment, direct-argv `run_code`, descendant
process termination, CLI `.cmd` discovery, MCP env sanitization, and MCP
start/stop race handling were implemented.
- Chat tool selection is deterministic and subtractive: maximum 14 tools and
8,000 exact OpenAI/LiteLLM schema characters per turn; duplicate names keep
the native definition; plugin/MCP tools cannot appear in casual chat unless
explicitly relevant; governance-blocked schemas are removed before model
serialization.
- `cli_execute`, `multi_edit`, DOCX/XLSX/PPTX/PDF generation are approval-gated.
- DOCX/PDF/PPTX/XLSX workspace containment uses `path.relative`, preventing
sibling-prefix escapes such as `workspace-sibling`.
- Coder, analyst, finance, data-engineer, executive-assistant, and general
purpose persona allowlists include their shipped code/artifact capabilities.
## Verification evidence
- Focused selector/chat tests: 61/61.
- Persona/agent/MCP/governance regression: 118/118.
- Windows/sandbox/CLI/MCP focused tests: 140/140.
- Confirmation, placeholder, persona, and CLI comprehensive contracts: 148/148.
- Production build: `npm run build:all` passed (package builds, server and web
typecheck, Vite production bundle).
- Full root Vitest run: 8,839 passed, 2 skipped, with remaining failures from
two CLI package-install/REPL tests timing out at 120s/180s in this Windows
checkout. Those timeouts need a separate launcher/runtime investigation.
## Tool-context result
Before this branch, a live general-purpose request exposed 78 tool definitions
(about 41.8k schema characters); coder exposed 36 (about 17.7k). Repeated
turns re-sent roughly 26.6k tokens of schemas. The route now selects and
serializes at most 14 tools / 8k schema characters, with a p95 selector test
under 10ms on a 78-tool synthetic pool.
## External-agent smoke result
Installed local CLIs detected: Claude Code 2.1.211, Codex 0.144.1, Hermes
0.18.2, Cursor 3.1.15, OpenClaw 2026.6.11, and Gemini. Claude and Codex
headless smoke paths ran; Hermes/OpenClaw were not run against persistent user
state. Launcher mappings still need runtime fixes for Hermes (`hermes chat -q`),
OpenClaw (`openclaw agent --message`), Codex detached/resume behavior, and
Cursor workspace binding.
## Open P0/P1 blockers
1. Solo/Tauri packaging still sets `WAGGLE_SKIP_LITELLM=1`; the built-in proxy
is Anthropic-only, while model resolution can still select a keyed
non-Anthropic provider. No clean Windows install has proven models, proxy,
and routing with Node/Python/Ollama/Docker absent.
2. There is no bundled local chat model/runner/weights; the current “local”
fallback can select a cloud Ollama tag (`ollama/minimax-m2.7:cloud`).
3. LiteLLM/Python bundling and resource layout are not yet wired as a tested
self-contained Solo artifact; embeddings may still download at runtime.
4. Smart router behavior remains heuristic and is not yet evaluated against
legal/payroll/destructive/verification adversarial cases.
5. A full browser E2E pass and three fresh runs for each of the ten requested
personas have not completed; therefore no 9.5/10 persona score is claimed.
6. Full root test timeouts in CLI package-install/REPL paths remain open.
## Persona acceptance matrix to run next
Run three fresh sessions each for general-purpose, researcher, writer,
project-manager, executive-assistant, finance-owner, coder, data-engineer,
verifier, and coordinator. Score artifact correctness, conversation quality,
tool/provenance fidelity, memory, safety, recovery, UX/latency, and accessibility
using the 100-point rubric. Auto-fail fabricated memory, unapproved mutation,
read-only writes, secret/sandbox escape, corruption, hangs, or false tool claims.

5
docs/production-readiness/AUDIT_COMPLETE.md
View File

@@ -1,5 +1,10 @@
# Waggle V1 Pre-Production Qualification — COMPLETE
> **Historical snapshot — superseded.** The “then ship” and “You can ship”
> statements below record an older audit and are not a current release verdict.
> Use [`09-LAUNCH_RECOMMENDATION.md`](./09-LAUNCH_RECOMMENDATION.md) for the
> active Windows Solo gate.
**Date**: 2026-03-20
**Branch**: `phase8-wave-8f-ui-ux`
**Baseline**: 3,895 tests, 277 files, zero failures