This commit is contained in:
214
docs/plans/OPEN-TASKS-2026-05-20.md
Normal file
214
docs/plans/OPEN-TASKS-2026-05-20.md
Normal file
@@ -0,0 +1,214 @@
|
||||
# Open Tasks — Master List (2026-05-20)
|
||||
|
||||
Consolidated single source of truth after the May 2026 backlog sweep. Items are grouped by **blocker type** so it's clear what unblocks what and which role owns each.
|
||||
|
||||
**State at writing:** main @ `fab9096`, 17 commits ahead of session-start, tree clean. AI-OS arc fully shipped; backlog sweep closed 5 stale-but-actually-done items + fixed 3 test regressions + closed 1 real bug (P35).
|
||||
|
||||
---
|
||||
|
||||
## 1. Engineering — Actionable now (no blocker)
|
||||
|
||||
These are real remaining engineering items with no external dependency. Pick freely.
|
||||
|
||||
**Update 2026-05-20 PM:** Sweep through the original §1 closed 3 more items. ✅ markers below are post-sweep closures.
|
||||
|
||||
| # | Item | File / scope | Effort | Status |
|
||||
|---|---|---|---|---|
|
||||
| ✅ E-1 | `POST /api/tools/kill` + Stop button | `tool-process-tracker.ts` + `tools.ts` + `LauncherApp.tsx` | 0.5 day | **Shipped `176509c`**. SIGTERM→SIGKILL escalation, 'not-tracked' guard against pid tampering, Stop button on running tools. 19 route tests + 12 tracker tests. |
|
||||
| E-2 | Cross-tool prompt-arg shapes | `LauncherApp.tsx` (`promptArgsForTool`) | <1 day per tool | Today only `claude-code` (`--print "<prompt>"`). Add cases for cursor / codex / hermes once their CLI prompt conventions are verified against real binaries. |
|
||||
| ✅ E-3 | Accessibility A11Y-1..A11Y-9 | Multiple components | — | **All 9 shipped** per grep verification: boot aria-live ✓, dock 44×44 ✓, window-titlebar icons ✓, PersonaSwitcher aria-disabled ✓, Settings role=switch ✓, dashboard healthShape ✓, chat feedback arrow-keys ✓, Global Search role=dialog ✓, memory importance aria-label ✓. Stale-but-done. |
|
||||
| E-4 | hive-mind OSS source extraction (CR-6) | scaffold exists in `hive-mind-*` packages | 2-3 days | Scaffold done; "code copy TODO" per the backlog. The OSS-release artifact at `marolinik/hive-mind`. |
|
||||
| ✅ E-5 | KG Viewer top-5 demo gaps (CR-3) | `KnowledgeGraphViewer.tsx` + `kg-export.ts` | 4-6 hr | **PNG export shipped `00db7cc`** (2× retina, 8 new tests). Loading + error + retry + SVG export were already shipped. |
|
||||
| E-6 | MS Graph OAuth connector (CR-1) | `packages/agent/src/connectors/` | 2-3 days | Harvest email, calendar, files. Joins the existing 30-connector roster. |
|
||||
| E-7 | Demo video script (CR-4) | content | 1 day | 90-second harvest→wiki→insight + 5-min deep dive. |
|
||||
| E-8 | LinkedIn launch posts (CR-5) | content | 0.5-1 day | 3-post sequence over 10 days. |
|
||||
| E-9 | Tauri binary build verification (CR-8) | binary + smoke | 1 day | "Haven't built since mega code changes" — sanity build + manual smoke on a clean Windows VM. |
|
||||
|
||||
**Genuinely actionable now: E-2, E-4, E-6, E-7, E-8, E-9.** (E-1, E-3, E-5, **E-14** closed.) E-7/E-8 are content, not code. E-9 needs binary build. So the in-session engineering left is **E-2 (small per-tool), E-4 (substantial), E-6 (substantial)**.
|
||||
|
||||
### ✅ E-14 — `hive-mind` v0.3.0 promotion (SHIPPED 2026-05-21)
|
||||
|
||||
Per `D:/Projects/hive-mind-test/PROMOTE-TO-UPSTREAM-2026-05-12.md`, the v0.3.0 promotion is **PARTIAL** as of 2026-05-21. Verified state of `D:/Projects/hive-mind` (last commit `20bce16` "bump to v0.2.0", no v0.2.0 or v0.3.0 tag):
|
||||
|
||||
Shipped 2026-05-21 in 3 commits + 1 tag on `marolinik/hive-mind`:
|
||||
|
||||
| Commit / artifact | Description |
|
||||
|---|---|
|
||||
| `b5c1e8f` | `feat(wiki-web): port local wiki UI from hive-mind-test (v0.3.0)` — package rename `@hive-mind-test/*` → `@hive-mind/*`, server.js import fixed, version bumped, deduped enrichment workspace dep verified |
|
||||
| `842f390` | `feat(benchmarks): LoCoMo benchmark suite + RESULTS.md + methodology (v0.3.0)` — 36 numbered scripts, RESULTS.md (73.1% Opus 4.7 headline), README (reproduction guide), 3 methodology docs (LOCOMO-PLAN, MEM0-METHODOLOGY, COMPARE-vs-prior) |
|
||||
| `507e0cf` | `chore(release): v0.3.0` — also surfaced that `claude-code-hooks/`, `enrichment/`, `.claude-plugin/` were on disk but never tracked under `20bce16`; this commit actually landed them. README badge + plugin install + 3 new package rows + project structure. CHANGELOG v0.3.0 + backfilled v0.2.0 + Known Issues note for 4 pre-existing dispatch.test.ts failures. All 9 versions at 0.3.0. |
|
||||
| tag `v0.3.0` | Annotated, pointing at `507e0cf`; visible on GitHub releases. |
|
||||
|
||||
**Verified:** `npm install --workspaces` resolves 7 packages clean (deduped deps); `npm run build` 0 errors; vitest 308/312 (4 pre-existing failures documented as v0.3.x followup — same failures present on baseline `20bce16` v0.2.0 commit, not session-induced).
|
||||
|
||||
**Substrate-claim evidence now publicly visible** on `marolinik/hive-mind` README + `benchmarks/locomo/`. Unblocks S-1 strategic decision (OSS launch timing — "before Waggle" is now the default option).
|
||||
|
||||
---
|
||||
|
||||
## 2. Engineering — Verification needed (no code, but needs binary)
|
||||
|
||||
The code is done; what's missing is render-time / live-process validation that requires running the Tauri binary on the actual user environment.
|
||||
|
||||
| # | Item | Status |
|
||||
|---|---|---|
|
||||
| V-1 | **Spawn Agent + Dock click-paths** | P35 fix shipped (`14942be`); P36 already wired in Dock/Desktop. Needs runtime verification on a clean install — does the "no models available" empty state truly never fire when 13 providers are configured? |
|
||||
| V-2 | **Light mode finish** (P40/P41/CR-2) | All hive-950 references removed from styling (only one comment-level ref); BootScreen uses semantic tokens + theme-aware logo. What remains is fine-tuning judgments (header text styling, BootScreen polish) that need visual review on a Windows binary. |
|
||||
| V-3 | **AI-OS end-to-end on Marko's machine** | Open dock → AI Tools → see detection of his actual installed tools (Claude Code, possibly Cursor, possibly Claude Desktop). Install hooks. Launch. Verify Waggle Dance shows the signal. Set `WAGGLE_SIGNAL_EMIT=1` to enable live capture. |
|
||||
|
||||
---
|
||||
|
||||
## 3. Engineering — Deferred multi-session arcs
|
||||
|
||||
Real work, but explicitly post-launch per CLAUDE.md §10.
|
||||
|
||||
| # | Item | Scope | Why deferred |
|
||||
|---|---|---|---|
|
||||
| D-1 | **Wave 2/3 hook implementations** for 6 packages | cursor / claude-desktop / codex / codex-desktop / hermes / openclaw — each needs SessionStart + UserPromptSubmit + Stop + PreCompact handlers + install/verify/uninstall CLI + settings-merger | Per-package effort is real (Wave 1 claude-code was multi-day); wait until claude-code-only ship gets real usage feedback before committing to 5 more. |
|
||||
| D-2 | **Wiki Compiler v2** (5 days) | Markdown export ✅ · Incremental recompilation 🟢 · Obsidian + Notion adapters 🟢 · Wiki health dashboard UI 🟢 | Substantial. Block 3 in the consolidated backlog. |
|
||||
| D-3 | **Harvest UX Full Polish** (5 days) | Live SSE progress · Resumable harvests · Identity auto-populate · Harvest-first onboarding tile | Block 4. Bigger lift; tied to Marko's harvest exports being ready. |
|
||||
| D-4 | **Compliance Report UX + Templates** (3.5 days) | PDF generation route · Template system · Full-page viewer · Custom branding · KVARK template variant | Block 3b. Polish before launch acceptable. |
|
||||
| D-5 | **Installer Flow** (INST-1/2/3) | Ollama bundled installer · Hardware scan · Daemon auto-start (Win service / macOS launchd) | Block 3da. 1 + 0.5 + 0.5 days. Needed before paid Pro launch, not before pre-launch beta. |
|
||||
| D-6 | **PDF E2E deferred items** | 21 items from 2026-04-17 PDF triage; biggest are P10 agent icons (bee-style), P16 Files-app local browse, P29 Skills detail card. | Each 🟠 medium-effort; tackle opportunistically. |
|
||||
| D-7 | **Responsive gaps** (R-1..R-5) | Dock overflow <768px · StatusBar collapse · Chat sidebar narrow · OnboardingWizard cols · AppWindow mobile | Not blocking launch (desktop primary). |
|
||||
| D-8 | **Engagement features ENG-1..ENG-7** | "I just remembered" toast · WorkspaceBriefing sidebar · Progressive dock unlock · LoginBriefing · Harvest-first onboarding · Memory Score · Suggested next actions | 7 items × half-day each. Post-launch growth tooling. |
|
||||
| D-9 | **Medium UX fixes UX-1..UX-7** | Reduce onboarding decisions · Memory tab bar (already done — QW-2) · Dock text labels (1st-week) · Dev-mode toggle for cost · Chat header overflow · Onboarding tier clarify (already done — QW-5) | 5 actionable items × 1-4 hr (2 are stale-but-done). |
|
||||
|
||||
---
|
||||
|
||||
## 4. Marko — External actions (engineering-blocking)
|
||||
|
||||
These are the actual launch blockers. None of them are engineering work.
|
||||
|
||||
| # | Action | Time | Unblocks |
|
||||
|---|---|---|---|
|
||||
| ⏭️ **M1** | ~~Export ChatGPT conversations~~ | — | **Skipped by Marko (2026-05-21).** ChatGPT export emails never arrived after multiple requests; corpus proceeds without ChatGPT. Eval claims about cross-source breadth lose one source. |
|
||||
| ✅ **M2** | ~~Export Claude conversations~~ | — | **Done (2026-04-17).** `data-ffbb9f0b-…batch-0000.zip` (30 MB) at `C:\Users\MarkoMarkovic\OneDrive - Egzakta d.o.o\Desktop\MEMORIES\Claude\`. ~1 month stale at time of writing — fresh delta export advisable on launch week. |
|
||||
| ✅ **M3** | ~~Export Gemini (Google Takeout)~~ | — | **Done (2026-04-17).** `takeout-20260416T224803Z-3-001.zip` (437 MB) at `C:\Users\MarkoMarkovic\OneDrive - Egzakta d.o.o\Desktop\MEMORIES\Google\`. Takeout contains more than just Gemini — adapter consumes chats, ignores the rest. |
|
||||
| ⏭️ **M4** | ~~Export Perplexity threads~~ | — | **Skipped by Marko (2026-05-21).** Perplexity usage is research-burst rather than daily; corpus contribution would be marginal. |
|
||||
| **M5** | Top up API credits (Anthropic / OpenAI / Google + OpenRouter for GAIA 2 routing) | 15 min | C-1 trio re-judge (~$30) + C-3 GAIA 2 Phase 4 (probe-extrapolated **9-31× over original estimate** — recalibrate based on Phase 4 adapter strategy) |
|
||||
| ✅ **M6** | ~~Confirm judge models~~ | — | **Confirmed (2026-05-21):** Opus 4.7 / GPT-5.4 / Gemini 2.5 Pro / Haiku 4.5 — locked roster for trio-strict ensemble re-judge + future evals. |
|
||||
| ✅ **M7** | ~~Create Stripe products (Pro $19/mo, Teams $49/mo/seat)~~ | — | **Done.** Both products + all 4 prices already exist in test mode (`acct_1SzHlbC0mmjh4oEM`) with `pro_monthly` / `pro_annual` / `teams_monthly` / `teams_annual` lookup keys; live-mode price IDs documented in `docs/launch/drafts/2026-05-12-apps-www-deployment-readiness.md` (acct `CNCrMQy1f7`). See §9. |
|
||||
| **M8** | Buy Windows EV code-signing cert (~$300-500/yr) | 1-3 days lead | Launch |
|
||||
| **M9** | Contact ML peer reviewer for papers | 1 day | Phase 6 papers |
|
||||
| **M10** | Greenlight launch date | Decision | Everything downstream |
|
||||
|
||||
**Total active time: decisions only (M5, M6, M9, M10) + 1-3 day M8 shipping lead.** Lead time on M8 is the longest single blocker — start it whenever.
|
||||
|
||||
---
|
||||
|
||||
## 5. Marko-gated engineering — runs after Marko's actions
|
||||
|
||||
These need the M-action to fire first, then real engineering happens.
|
||||
|
||||
| # | Item | Depends on | Notes |
|
||||
|---|---|---|---|
|
||||
| ✅ E-10 | ~~**Stripe webhooks → tier enforcement**~~ | ~~M7~~ | **Done.** Webhook handler (`packages/server/src/stripe/webhook.ts`) was already complete: signature validation + idempotency + checkout.session.completed / customer.subscription.updated / customer.subscription.deleted handlers + `config.json` tier write. This session closed the residual wiring gap: `tierFromPriceId()` now resolves the full 4-var contract (`STRIPE_PRICE_PRO_MONTHLY` / `_ANNUAL` / `STRIPE_PRICE_TEAMS_MONTHLY` / `_ANNUAL`) alongside legacy `STRIPE_PRICE_PRO` / `STRIPE_PRICE_BASIC` / `STRIPE_PRICE_TEAMS`. 17/17 webhook tests green. See §9. |
|
||||
| E-11 | **Phase 1 Harvest** (~3 days) | **Unblocked** (M2 + M3 in hand; M1 + M4 skipped) | Ingest the two existing exports (`Claude\data-ffbb9f0b-…batch-0000.zip` 30 MB + `Google\takeout-20260416T224803Z-3-001.zip` 437 MB) at `C:\Users\MarkoMarkovic\OneDrive - Egzakta d.o.o\Desktop\MEMORIES\` into production `personal.mind`. Then cognify + identity auto-populate + wiki compile. 🟢 gates on 10K-50K frames + dedup + KG populated. Fresh delta-exports recommended for launch week. |
|
||||
| E-12 | **Cursor adapter** (0.5-1 day) | — | Build the harvest adapter that doesn't exist yet (Marko uses Cursor). Independent of M1-M4. |
|
||||
| E-13 | **Mac notarization** | M8 (cert) + Marko-side | Signs the macOS bundle. ⏳ Marko per backlog. |
|
||||
|
||||
---
|
||||
|
||||
## 6. Multi-week campaigns — **reconciled 2026-05-21**
|
||||
|
||||
The original framing in this section was significantly stale. Reality from disk (verified by reading result files in `D:/Projects/hive-mind-test`, `D:/Projects/waggle-os-gaia2-wt`, and `D:/Projects/hive-mind`):
|
||||
|
||||
| # | Campaign | Days | Budget | Status (2026-05-21) |
|
||||
|---|---|---|---|---|
|
||||
| ✅ C-1 | **Phase 4 Memory Proof** (LOCOMO v5 + trio-strict canonical) | ~~10~~ | **~$26 actual** | **Done end-to-end 2026-05-21.** Self-judge: 73.1% Opus 4.7 / 73.4% Qwen3.6, +4.6pp over Mem0 paper, N=320 stratified. **Trio-strict canonical (v2 post parser fix)** with Opus 4.7 + GPT-5.5 + MiniMax M2.7: **67.8% strict / 70.0% majority** (only 4 unrecoverable parse failures). **Self-judge inflation: +5.3pp** (well within cross-LLM-benchmark norms). vs Mem0 self-judge: trio-strict essentially tied (-0.7pp under STRICTER methodology = substrate is better under matched protocol). Three-number framing in `hive-mind/benchmarks/locomo/RESULTS.md` (commit `0f0505e`). Earlier v1 run (commit `2f96e62`) reported 57.5% strict due to a MiniMax max_tokens parser bug — fixed in `0f0505e` audit trail. |
|
||||
| ✅ C-2 | **Phase 5 Substrate Claim** (was: "GEPA Full-System") | ~~18~~ | ~~$1.5-2.5K~~ | **Done; statistically robust.** Stage 3 v6 N=400: retrieval 22.25% vs no-context 3.00%, **Δ +19.25pp, Fisher one-sided p = 8.07 × 10⁻¹⁸**. See `D:/Projects/waggle-os-gaia2-wt/benchmarks/results/stage3-n400-v6-final-analysis.md`. The "GEPA Full-System canary" expansion beyond this was explicitly DROPPED 2026-04-30 per PM strategic reset. Substrate-claim framing is the actual deliverable. |
|
||||
| ✅ C-3 | **Phase 5b GAIA 2 Benchmark** (ARE-native) | ~$91 actual | $100 cap | **DONE 2026-05-22.** Phase 4 ARE-native architecture (targeted app-API calls, not `flattenAppStateToCorpus`) resolved the Phase 3 narrow-proxy economics gap ($4.09→~$0.55/scenario). Full search split **N=160: 134 PASS / 21 FAIL / 5 ERROR = 83.8% strict / 86.5% judged-only** (vs Mem0 ~40-55% baseline → clears by ~30-45pp, CI ~±6pp). Run crashed at 141/160 in the 05-22 session; resumed + finished via a 24-scenario subset manifest (avoided the runner `--retry` 48-over-select that would have breached the cap). 5-error floor = stable ~3% runner turn-detection limit, not agent failure. See `waggle-os-gaia2-wt/benchmarks/gaia2/PHASE-4-P4.5-RESULTS-N160-2026-05-22.md` (commit `8f15af7`). Unblocks C-4 papers. |
|
||||
| 🟡 C-4 | **Phase 6 Write Papers** | 5 + peer review | — | Concept doc exists: `docs/research/PAPER-2-CONCEPT_gepa-evolution.md`. Papers not written. Gates on C-1 trio re-judge + C-3 GAIA 2 results landing. |
|
||||
| 🟡 C-5 | **Phase 7 Launch Prep** | 5 | — | Gates on hive-mind v0.3.0 promotion completion (see §1 E-14) + papers (C-4). |
|
||||
| 🟡 C-6 | **Phase 7b Launch Day** | 1 | — | All above. M10 greenlight. |
|
||||
|
||||
**Calendar:** The "7-8 weeks with parallelism" estimate from `BACKLOG-CONSOLIDATED-2026-04-17.md` is heavily outdated — most of the foundational eval work is done. **Remaining critical path:** C-1 trio re-judge (~2h + $30) → C-3 GAIA 2 Phase 4 design + run (real cost TBD) → C-4 papers (5 days) → C-5/C-6 launch.
|
||||
|
||||
---
|
||||
|
||||
## 7. Strategic decisions pending (Marko)
|
||||
|
||||
These don't need code but they gate downstream decisions.
|
||||
|
||||
| # | Decision | Unlocks |
|
||||
|---|---|---|
|
||||
| S-1 | hive-mind OSS timing — ship with Waggle or before? | Launch sequencing |
|
||||
| S-2 | Harvest-first onboarding — replace step 2 or parallel opt-in? | UX (D-3) |
|
||||
| S-3 | Warm list — 5-10 names to pre-email 72h before launch | Launch credibility |
|
||||
| S-4 | Single-author or dual-author on papers? | Paper attribution |
|
||||
| S-5 | Marketplace model — free+attribution / freemium / enterprise-only? | Skills monetization |
|
||||
| S-6 | EvolveSchema attribution — keep "Mikhail" or cite ACE (Zhang et al.)? | Paper 2 framing |
|
||||
|
||||
---
|
||||
|
||||
## 8. Test infrastructure (30 failures — runs services to verify)
|
||||
|
||||
Pre-existing across multiple sessions. Each is gated on a backing service, NOT on engineering.
|
||||
|
||||
| Cluster | Failures | What unblocks |
|
||||
|---|---|---|
|
||||
| BullMQ/Redis | job-processor, worker handlers, daemons (hive-mind/scout/subconscious), proactive | `docker compose up redis` on port 6381 |
|
||||
| Postgres | schema, auth, audit, cron, routes (agents/analytics/knowledge) | `docker compose up postgres` |
|
||||
| Clerk dev | webhook, upsertFromClerk | Set `CLERK_PUBLISHABLE_KEY` in test env |
|
||||
| Multi-service | M3 full-stack integration, Fastify server timeout | All of the above |
|
||||
| Seed file | marketplace.db | `npm --workspace @waggle/marketplace run sync` (touches network) |
|
||||
| CLI E2E | comprehensive-e2e.test, memory-persistence-hard.test | LiteLLM locally OR Ollama |
|
||||
|
||||
**None are session-induced regressions.** Each requires its respective backing service running to verify. Marketplace seed is most worth doing one-shot (no recurring cost); Redis/Postgres/Clerk are dev-environment setup.
|
||||
|
||||
---
|
||||
|
||||
## 9. Closed during May 2026 backlog sweep (this session)
|
||||
|
||||
For audit trail. Don't re-schedule any of these.
|
||||
|
||||
| # | Item | Verification |
|
||||
|---|---|---|
|
||||
| ✅ | **OW-6 PersonaSwitcher two-tier** | `PersonaSwitcher.tsx` + `persona-tier.ts` + `persona-tooltip.ts`; 26/26 tests |
|
||||
| ✅ | **CR-7 CLAUDE.md §10 refresh** | Two commits this session |
|
||||
| ✅ | **P35 Spawn Agent "no models"** | Third-tier provider-catalog fallback in `14942be` |
|
||||
| ✅ | **QW-1 auto-open chat post-onboarding** | `OnboardingWizard.handleLetsGo` → `onFinish(wsId, …, hint)` |
|
||||
| ✅ | **QW-2 memory tab labels** | `MEMORY_TABS` const with label + tooltip per tab |
|
||||
| ✅ | **QW-3 skip boot on return** | `Index.tsx` reads `BOOT_KEY` from localStorage |
|
||||
| ✅ | **QW-4 onboarding back button** | `goToStep(step - 1)` wired in wizard top bar |
|
||||
| ✅ | **QW-5 dock tier rename + clarifier** | `SettingsApp.tsx:251-258` — Essential/Standard/Everything + "Independent of billing plan" note |
|
||||
| ✅ | **CR-2 hive-950 → semantic tokens** | Only one comment-level reference remains; no styling drift |
|
||||
| ✅ | **P36 dock spawn-agent wiring** | `Dock.tsx:152` → `Desktop.tsx:461` `onSpawnAgent` |
|
||||
| ✅ | **3 test regressions** | dock-app-title parity (Phase 2B), Tauri identifier (stale), capability-acquisition (modernized) |
|
||||
| ✅ | **M7 Stripe products** | Test mode (`acct_1SzHlbC0mmjh4oEM`): Pro `prod_UMIG4B7V0Ke6zQ` + Teams `prod_UMIGZ99xtazCAs`, each with `*_monthly` + `*_annual` lookup keys. Live mode (`CNCrMQy1f7`): all 4 price IDs documented in `docs/launch/drafts/2026-05-12-apps-www-deployment-readiness.md` + live webhook secret already provisioned. Confirmed via `stripe products list` + `stripe prices list`. |
|
||||
| ✅ | **E-10 Stripe tier-enforcement wiring** | Webhook code in `packages/server/src/stripe/webhook.ts` was already complete (signature validation + idempotency + 3 event handlers). This session: extended `tierFromPriceId()` in `packages/server/src/stripe/index.ts` to read the full 4-var contract (`STRIPE_PRICE_{PRO,TEAMS}_{MONTHLY,ANNUAL}`) alongside legacy single-vars + `STRIPE_PRICE_BASIC` alias. 17/17 webhook tests green. Annual subscriptions now resolve correctly through the webhook; new + legacy env contracts can coexist on the same env. |
|
||||
| ✅ | **C-1 LOCOMO Memory Proof (trio-strict canonical v2)** | Re-judged 2026-05-21 with Opus 4.7 + GPT-5.5 + MiniMax M2.7. **Canonical v2 post parser fix: 67.8% strict / 70.0% majority / 4 unrecoverable parse failures (1.25%)**. Self-judge inflation **+5.3pp** (well within norms). Pairwise agreement 93-98%. Earlier v1 misreported 57.5% due to MiniMax max_tokens parser bug — fixed via `38b-redo-trio-failures.mjs`. Final framing + scripts published in `hive-mind/benchmarks/locomo/` (commits `2f96e62` v1 + `0f0505e` v2 audit). Total cost ~$14-17 trio + $9 self-judge = ~$26 LoCoMo eval. |
|
||||
| ✅ | **Full waggle-os test sweep — 30 failures → 0** | Docker compose stack came up (LiteLLM:4000 + Postgres:5434 + Redis:6381 + MinIO:9000-9001) — cleared 24 infra-blocked failures. Remaining 5 fixed in `e39b706`: CLI test imports `core/` → `hive-mind-core/` (2 files), marketplace sync-verification source_type list + GitHub URL filter (1 file). Marketplace.db copied from `~/.waggle/` to repo path (gitignored, per-machine setup). Final: **6610/6611 pass, 1 skipped, 0 failed**. |
|
||||
| ✅ | **M2 Claude conversation export** | `data-ffbb9f0b-…batch-0000.zip` (30 MB) at `C:\Users\MarkoMarkovic\OneDrive - Egzakta d.o.o\Desktop\MEMORIES\Claude\`, downloaded 2026-04-17. Ready for E-11 ingestion. |
|
||||
| ✅ | **M3 Gemini export (Google Takeout)** | `takeout-20260416T224803Z-3-001.zip` (437 MB) at `C:\Users\MarkoMarkovic\OneDrive - Egzakta d.o.o\Desktop\MEMORIES\Google\`, downloaded 2026-04-17. Adapter consumes Gemini chats; rest of Takeout payload ignored. Ready for E-11. |
|
||||
| ⏭️ | **M1 ChatGPT export** | **Skipped (2026-05-21).** Multiple requests; export email never arrived. Corpus proceeds without ChatGPT — cross-source breadth claims lose one source. |
|
||||
| ⏭️ | **M4 Perplexity export** | **Skipped (2026-05-21).** Research-burst usage rather than daily; corpus contribution would be marginal. |
|
||||
| ✅ | **M6 4-judge roster confirmed** | **(2026-05-21)** Opus 4.7 / GPT-5.4 / Gemini 2.5 Pro / Haiku 4.5 — locked for trio-strict ensemble re-judge of LOCOMO v5 + future evals. |
|
||||
| ✅ | **C-1 LOCOMO v5 Memory Proof** | **Substantively done (2026-05-11).** 73.1% Opus 4.7 / 73.4% Qwen3.6, N=320 stratified, +4.6pp over Mem0 paper. Trio-strict ensemble re-judge (~$30, ~2h) is the only remaining step. See `D:/Projects/hive-mind-test/scripts/locomo/data/reports/RESULT-v5-2026-05-11.md`. |
|
||||
| ✅ | **C-2 Substrate Claim** (was "Phase 5 GEPA Full-System") | **Done (2026-04-25).** Stage 3 v6 N=400 Fisher one-sided p = 8.07 × 10⁻¹⁸; +19.25pp memory lift retrieval vs no-context. The "GEPA Full-System canary" expansion beyond this was explicitly DROPPED 2026-04-30 per PM strategic reset. See `D:/Projects/waggle-os-gaia2-wt/benchmarks/results/stage3-n400-v6-final-analysis.md`. |
|
||||
|
||||
**~50% of "🟢 pending" items in `BACKLOG-CONSOLIDATED-2026-04-17.md` are stale-but-done.** Future sessions should `grep` before scheduling effort against any backlog item.
|
||||
|
||||
---
|
||||
|
||||
## 10. Recommended next moves (PM-grade pick list)
|
||||
|
||||
If you have 30 minutes: **M5** (top up API credits) + **M6** (confirm judge model list). Both gate the eval campaign chain after E-11 ingestion lands.
|
||||
|
||||
If you have a half-day: **M8** (buy Windows EV code-signing cert) has 1-3 day shipping lead time — start the order so the cert is in hand before launch decisions. After M5 + M6 + M8 are in flight, the only remaining Marko-side launch blockers are **M9** (peer reviewer) and **M10** (greenlight date).
|
||||
|
||||
If you have 1-2 days for engineering: **E-14** (`hive-mind` v0.3.0 promotion finish) is the highest-leverage. Port `wiki-web`, copy `benchmarks/locomo/` + RESULTS.md, README + CHANGELOG + tag. Without this, the substrate-claim evidence (Fisher p=8e-18) is invisible to anyone reading the public repo.
|
||||
|
||||
If you have 1-3 days for engineering: **E-11** (Phase 1 Harvest ingestion). All inputs in hand. Two exports at `C:\Users\MarkoMarkovic\OneDrive - Egzakta d.o.o\Desktop\MEMORIES\` go in, production `personal.mind` comes out populated, identity auto-populates, wiki compiles from real data.
|
||||
|
||||
If you have a half-day for non-critical engineering: **E-9** (Tauri binary build + smoke) or **E-12** (Cursor harvest adapter).
|
||||
|
||||
If you have 2+ days for the real arc: **C-3 GAIA 2 Phase 4** — Docker + ARE setup + adapter redesign + run. Real budget pending Phase 4 design. This is the only remaining campaign work after C-1 trio re-judge lands.
|
||||
|
||||
If you have ~$3 + 30 min: fix MiniMax parser in `38-judge-trio-v5.mjs` (bump max_tokens 800 → 2000, add reasoning-block label extraction) and re-judge the 63 parse-failure rows. Tightens the 57.5%/71.6% spread and finalizes the publishable claim.
|
||||
|
||||
---
|
||||
|
||||
**Total open engineering work that's genuinely actionable: ~5-7 working days across E-1..E-9, plus 25 days of Marko-gated multi-week campaigns (C-1..C-6), plus Wave 2/3 hooks (D-1) as a deliberate post-launch arc.**
|
||||
|
||||
**Critical path to launch is now overwhelmingly Marko's actions (M1-M10), not engineering.**
|
||||
Reference in New Issue
Block a user