oleg.m/waggle-os
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

moving
Some checks failed
Installer Smoke / installer-smoke (push) Has been cancelled
Details

Browse Source
This commit is contained in:
Oleg Maslov
2026-09-02 10:10:29 +02:00
commit 0c3e2ead3b
3841 changed files with 970576 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

428
docs/research/01-oss-memory-packaging-strategy.md Normal file
View File

@@ -0,0 +1,428 @@
# Open-Source Packaging Strategy — `hive-mind` Memory System
**Author:** Waggle OS research series (1 of 7)
**Drafted:** 2026-04-15 (overnight batch)
**Scope:** Strategy + action plan for open-sourcing Waggle's memory system as a standalone project tentatively named `hive-mind`. License selection, moat preservation, competitive positioning, launch tactics, governance model, success criteria.
---
## TL;DR
**Ship `hive-mind` as a permissively-licensed OSS project** containing the memory primitives (FrameStore, HybridSearch, KnowledgeGraph, IdentityLayer, AwarenessLayer, CognifyPipeline, MemoryWeaver, Wiki Compiler skeleton, harvest adapters). **Keep Waggle's orchestration, UI, multi-mind enterprise features, evolution stack, and compliance reporting proprietary.**
**License recommendation: Apache 2.0 for the OSS layer** (max ecosystem compatibility, patent grant, contributor-friendly), with **BSL (Business Source License) or a narrow `Commons Clause` wrapper on the specific subsystems most at risk of cloud-vendor capture** (team sync server, enterprise admin console) — though those aren't part of the OSS split anyway, so Apache 2.0 across the split is the clean choice.
**Positioning angle:** the ONLY OSS memory system designed around *compliance-by-default* (EU AI Act logging baked in) with a *wiki-compilation layer* (structured knowledge pages over the frame corpus) and a *provable temporal frame model* (not just key-value memory, not just turn-blob storage).
**Timing:** publish ~30-60 days *after* the v2 hypothesis reveal (see report 4). The memory OSS announcement benefits from the attention the hypothesis reveal will generate.
---
## 1. Why open source at all?
Four reasons, in priority order:
### 1.1 Set the terms of the ecosystem
The memory-for-agents layer is in an early-standardization moment. mem0, Letta (née MemGPT), Zep, Cognee, GraphRAG (Microsoft), Mastra, and a handful of smaller projects are all attempting to become the default. If **hive-mind** ships as the most-rigorous, best-documented, and most-complete OSS memory system right now, it has a real shot at becoming the reference.
### 1.2 Inbound talent + credibility
An actively-maintained OSS project with clear docs and a coherent vision attracts:
- ML engineers who want to contribute to "the real" memory system
- Researchers who need a citable artifact
- Technical buyers who do due diligence on their vendors' open-source footprint before signing enterprise deals
### 1.3 Enterprise sales lubricant
"This runs on open-source primitives that you can inspect, fork, and audit" is a meaningful closer in enterprise sales cycles (for KVARK conversations especially). The more OSS-friendly Waggle appears, the easier those conversations are.
### 1.4 Harvest corpus expansion
OSS users of `hive-mind` who later adopt Waggle/KVARK commercially bring their existing harvest corpus with them. The corpus grows the industry network effect, even if the specific user doesn't pay.
### 1.5 What open source is NOT for
- **Not** for building a VC-backed company (Waggle is part of Egzakta — profitable / strategic, not VC-growth-at-all-costs)
- **Not** for competing with itself (hive-mind is a complement, not a substitute, for Waggle the product)
- **Not** for moat generation (moats come from the closed-source evolution stack, compliance reporting, multi-mind orchestration, and the product itself)
---
## 2. What goes OSS and what stays proprietary
### 2.1 In scope for hive-mind OSS
From `packages/core/`:
- **FrameStore** + schema (I/P/B frame model)
- **SessionStore**
- **HybridSearch** with RRF fusion
- **KnowledgeGraph** with SCD-2 temporal validity
- **IdentityLayer**
- **AwarenessLayer**
- **CognifyPipeline** (ingest + entity extraction + KG linkage)
From `packages/weaver/`:
- **MemoryWeaver** (consolidation, decay, strengthen, link)
From `packages/core/src/harvest/`:
- The adapter interface + universal-import types
- All 11 current adapters (chatgpt, claude, claude-code, claude-desktop, gemini, perplexity, markdown, plaintext, pdf, url, universal)
- Distillation pipeline skeleton (pluggable LLM, no hard-coded provider)
From `packages/wiki-compiler/`:
- The compiler skeleton (page types, incremental compilation, dedup)
Bindings:
- SQLite schema (documented)
- Embedder interface (pluggable: Ollama, LiteLLM, OpenAI, HuggingFace)
- Reference CLI for testing + harvesting
### 2.2 Kept proprietary
- **`packages/agent/`** — agent runtime, personas, tool surface, hooks, approval gates, cost tracker, subagents
- **Evolution stack** — 10 files in `packages/agent/src/evolution*` + `judge.ts` + `evolve-schema.ts` + `iterative-optimizer.ts`
- **Compliance reporting** — `packages/core/src/compliance/` + the AuditReport PDF generator (`compliance-pdf.ts`)
- **MultiMindCache + team sync + MCP catalog** — `packages/shared/src/mcp-catalog.ts`, `packages/core/src/mind/team-sync.ts`
- **UI + Tauri shell** — all of `apps/web/` and `app/`
- **KVARK integration** — `packages/agent/src/kvark-tools.ts`
- **Skill promotion gates + marketplace** — `packages/marketplace/`, the promote_skill tool, trust-model.ts
- **Marketplace + Stripe** — `packages/server/src/stripe/`
### 2.3 Why this split works
The OSS layer is the *primitives* — an experienced engineer could build their own agent on top of `hive-mind` if they wanted to. But they'd re-invent:
- Persona management (13 built-in personas + tool-pool filtering)
- Skill lifecycle (auto-extract + promotion + retirement + marketplace)
- Approval gates + autonomy tiers
- Cost tracking + budget caps
- Compliance reporting (AuditReport + PDF)
- Wiki app UI
- Harvest UI
- Evolution loop + eval + judging
That's roughly 80% of Waggle's value delivered in the product layer. The OSS layer is the foundation; Waggle is the building.
### 2.4 Repository structure
Two options:
**Option A: Separate repo** (`waggle-os/hive-mind`)
- Clean mental model
- Easier for OSS contributors (no Waggle-product PRs to navigate)
- Costs: dual CI, cross-repo testing, subtree-merge friction
**Option B: Monorepo subset exposed** (`waggle-os/waggle-os`, with `packages/core` + `packages/weaver` + `packages/wiki-compiler` under Apache 2.0 and rest under proprietary license)
- Operationally simpler
- Reader confusion: "is this project open or not?"
- OSS contributors would need to tiptoe around proprietary dirs
**Recommendation: Option A (separate repo).** The cognitive clarity is worth the CI cost. Use git subtree or a publish-to-mirror workflow to keep hive-mind in sync with waggle-os packages/core.
---
## 3. License selection
### 3.1 Options surveyed
| License | Permissive? | Patent grant? | Cloud-vendor capture risk | Recommendation fit |
|---|---|---|---|---|
| MIT | Yes | No explicit | High | Widely loved; risky if a hyperscaler forks it for managed service |
| Apache 2.0 | Yes | Yes | High but patent grant is defensive | **Top candidate** |
| BSD-3 | Yes | No explicit | High | Similar to MIT |
| MPL 2.0 | Partial (file-level copyleft) | Yes | Medium | Interesting but confusing for users |
| LGPL 3 | File-level copyleft | Yes | Medium | Unfriendly for embedded uses |
| AGPL 3 | Strong copyleft over network | Yes | Low | Contributors allergic to it; poison pill for many enterprise adopters |
| BSL (Business Source License) | Time-gated proprietary → Apache | Yes | Low | Great for monetization protection but OSS community skeptical |
| SSPL (MongoDB) | Not OSI-approved | Yes | Low | Permanent stigma post-MongoDB episode; avoid |
| Commons Clause overlay | Adds "can't sell" restriction to any base | Depends on base | Low | Niche; creates confusion |
### 3.2 Recommendation: Apache 2.0 (refined: consider Mastra's dual-license pattern)
After research-agent validation: the strongest license strategy is **Apache-2.0 for the `hive-mind` primitives PLUS a source-available `ee/` directory pattern** (copy Mastra's playbook) for any future enterprise-only subsystems that stay in the same repo. Mastra ships Apache-2.0 `packages/core` and source-available `packages/ee/` governed by an Enterprise License — it's functionally the best of both worlds and the community has accepted it. If Waggle later wants a team-server or enterprise feature inside `hive-mind`, the `ee/` pattern is the clean answer. For the initial OSS split the `ee/` dir can just not exist yet.
Reasoning for Apache-2.0 on the core:
- **Patent grant is critical.** MIT/BSD don't have it; Apache 2.0 does. For a memory system that may touch patentable territory (temporal frame model, hybrid RRF, wiki compilation), the patent grant is defensive armor.
- **Cloud-vendor capture risk is real but manageable.** If AWS/Azure/GCP fork hive-mind and launch "Managed Hive Mind as a Service," that's painful but not existential — Waggle's moat is in the agent/evolution/compliance layer, not the memory primitives themselves. Apache 2.0 lets Waggle keep iterating ahead.
- **Community goodwill.** Apache 2.0 is neutral-positive in every community discussion. AGPL or BSL would earn criticism and limit adoption. Waggle wants adoption more than it wants to prevent forks.
- **Enterprise compatibility.** Every Fortune 500 legal team has already approved Apache 2.0. BSL or AGPL require fresh legal review and often get blocked.
### 3.3 What about the historical cases?
| Case | Context | Lesson |
|---|---|---|
| **Elastic → SSPL (2021)** | AWS launched OpenSearch managed service; Elastic relicensed to SSPL. OpenSearch fork thrives, Elastic stock tanked then recovered. | Relicensing post-adoption is painful. License choice upfront matters. |
| **Hashicorp → BSL (2023)** | Terraform fork (OpenTofu) emerged; community split. | BSL is monetization-friendly but splits communities. |
| **MongoDB → SSPL** | Launched MongoDB Atlas; licensed to block cloud-vendor reselling. | Worked commercially but ended OSI-approved status; academic + some government users left. |
| **Redis → dual-license (2024)** | Similar AWS motivation; complex licensing move. | Sends confusing signals to users. |
| **CockroachDB → dual-license** | BSL + enterprise commercial. | Kept community; monetization clean. |
**Pattern:** relicensing *from* permissive *to* restrictive is always painful. Starting permissive (Apache 2.0) and keeping commercial product distinct is cleaner. **Waggle's model has the commercial product already separate from the OSS candidate, so this is the natural path.**
### 3.4 Contributor agreements
Recommend **Developer Certificate of Origin (DCO)** — simple, no paperwork burden, used by Linux, Docker, GitLab. Requires committers to sign off on commits. Apache Corporate Contributor License Agreement (CCLA) is overkill for this project and a friction point for contributors.
---
## 4. Positioning vs the OSS competition
*(Specific star counts and funding details to be refined when Agent A's research returns. Framing below based on public knowledge as of early 2026.)*
### 4.1 The neighboring projects (verified April 2026)
| Project | Stars | License | Funding / Backer | Positioning |
|---|---|---|---|---|
| **mem0** (ex-Embedchain) | ~48k | Apache-2.0 | $24M Series A (Basis Set, Kindred, Peak XV, YC) | Universal memory layer; fact extraction + vector recall. Paper claims ~26% accuracy gain on LOCOMO vs full-context. |
| **Letta** (ex-MemGPT) | ~13k | Apache-2.0 | $10M seed @ $70M post (Felicis, YC); UC Berkeley Sky Lab | OS-inspired tiered memory (core/recall/archival); highest benchmark scores (~83% LongMemEval) |
| **Graphiti** (ex-Zep OSS) | ~24.5k (plus 4.4k zep) | Apache-2.0 | VC-backed | Bi-temporal KG memory. **Zep Community Edition deprecated April 2025** — only Graphiti OSS remains. |
| **GraphRAG** (Microsoft) | ~31k | MIT | Microsoft Research | Entity-community graph over doc corpora. Retrieval-oriented research artifact. |
| **Mastra** | ~22k | **Apache-2.0 core + source-available `ee/` (Enterprise License)** | YC; ex-Gatsby team | TypeScript agent framework with 4 built-in memory types. **Pattern Waggle should copy.** |
| **Cognee** | ~14.2k | Apache-2.0 | Independent/seed | "Knowledge engine" — vector + graph unified memory, cognitive-science framing. Graduated GitHub Secure OSS. |
| **LangChain memory** | part of 100k+ LangChain | MIT | LangChain (VC) | Baseline table-stakes: Buffer/Window/Summary/Entity/KG memory classes. |
| **Anthropic / OpenAI Memory** | closed | proprietary | — | Surface-scoped (per-product), not portable. Anthropic's "Auto Dream" consolidation cycle is notable. |
**License consensus is striking:** 5 of the 6 top OSS memory projects picked Apache-2.0 (GraphRAG is the MIT outlier — and Microsoft doesn't monetize it). None has gone BSL/SSPL/AGPL yet. The field is still in land-grab; permissive license wins adoption.
### 4.2 Where hive-mind would differentiate
The strongest differentiators, from most to least impactful:
1. **Compliance-by-default** — every frame has provenance (source, timestamp, agent/user/tool origin), every interaction can be logged in an audit-ready format. No other OSS memory system ships EU AI Act Art 12/14/19/26/50 status checking out of the box. This is the headline.
2. **Temporal frame model** — I/P/B frames with consolidation/decay/strengthen. Conceptually novel, patentable, and practically valuable (solves the "my memory is full of outdated stuff" problem that plagues key-value memory systems).
3. **Wiki compilation** — no other memory system produces structured entity/concept/synthesis pages automatically. This turns "I have a mind" into "I have a navigable second brain." Huge UX win.
4. **Write-path contradiction detection** — Gap K shipped this session. Ships detection of contradictory memories on save, flags them, emits a correction-category improvement signal. Other systems accept contradictions silently.
5. **Multi-mind isolation + sanctioned cross-access** — first-class concept of personal vs workspace vs team vs enterprise scope, with approval-gated cross-mind reads. Designed for multi-tenant from day one.
6. **Harvest adapters for 11 consumer AI tools** — no other OSS memory system ingests from ChatGPT / Claude / Claude-code / Gemini / Perplexity / etc. Most are API-first; hive-mind is "bring your existing AI life."
7. **SQLite + sqlite-vec baseline** — no infrastructure to deploy. Competitors typically require postgres + pgvector + redis; hive-mind runs on a single file. Lowering setup friction by 10x.
### 4.3 What hive-mind should NOT claim
- Not the fastest (we don't benchmark against top-performance vector DBs)
- Not the most scaled (single-user / small-team volumes; not billion-frame operations)
- Not the "simplest" — our model is richer than key-value; that's a feature, not a bug, but we should acknowledge the conceptual overhead in onboarding
---
## 5. Brand, naming, identity
### 5.1 Name: `hive-mind`
Rationale:
- Already referenced in existing memory (`project_wiki_compiler_spec.md` and elsewhere)
- Continues the Hive DS brand family (honey/hive/waggle/bee semantic)
- Memorable + culturally loaded (sci-fi resonance without being silly)
- Verb-able: "hive-mind it" as shorthand for "persist this to memory"
- Available on GitHub (tentatively — needs check)
- Distinct enough from "memory" that people know they're adopting a specific model
### 5.2 Tagline options
- "Persistent memory for AI agents. Compliance-first. Local-first."
- "The second-brain kernel for autonomous AI."
- "Open-source memory for AI. Built for trust."
First is the most accurate; second is punchiest; third is strategically clearest on what makes us different.
### 5.3 Visual identity
- Hexagonal honeycomb logo variant (lighter weight than Waggle's waggle-bee logo)
- Same Hive DS palette (honey, hive-950)
- GitHub README: hero screenshot of a wiki page compiled from frames (proof visual)
### 5.4 Relationship to Waggle brand
"hive-mind is the memory kernel that powers Waggle OS. Waggle is the full product (UI + agents + skills + compliance + evolution + marketplace). hive-mind is the open-source foundation you can use standalone."
Clear enough that technical users understand the relationship immediately. Avoids the "is Waggle secretly closed now?" confusion.
---
## 6. Launch tactics
### 6.1 Pre-launch (T-60 days)
- Repo in draft (private on GitHub org, Apache 2.0 license committed)
- README polished to publishable quality
- 5-10 starter tutorials (cookbook-style: "harvest ChatGPT", "build a custom adapter", "use in a Python app", "wire to your own LLM", etc.)
- Quickstart docs on `hive-mind.dev` or `docs.waggle-os.ai/hive-mind`
- Reproducibility: an example notebook that ingests a small sample + shows search, KG, wiki compilation
- Soft-launch to 20-50 friendly reviewers (the warm list from report 4)
- Gather feedback; polish for 2-4 weeks
### 6.2 Launch day (T+0)
- Publish repo public on GitHub
- Blog post announcing: "Open-sourcing the memory system behind Waggle OS"
- HN Show HN post
- Twitter thread (Marko + Waggle org accounts)
- Reddit posts to r/MachineLearning, r/LocalLLaMA, r/ChatGPTCoding
- Pin in Waggle Discord / community channels
- Reach out to 5-10 AI newsletter writers (Ben's Bites, The Rundown, Import AI, Alpha Signal) with a personalized note
### 6.3 Launch +7 days
- Engage every substantive issue/PR/comment
- Ship 2-3 small contributor-friendly PRs as "good first issue"
- Write a follow-up post: "What we learned in the first week"
- Podcast circuit if interest
### 6.4 Launch +30 days
- First contributed adapter merged (Cursor / Notion / Obsidian — whichever lands first)
- First third-party integration announced (someone using hive-mind in their own project)
- Cold-weather evaluation: how many stars, how many PRs, how many discussions?
- Go / no-go on a first community call
### 6.5 Launch +90 days
- v0.1 → v0.2 release with community-driven features
- Roadmap published (inside hive-mind repo, not Waggle's)
- Relationship with a research group codified (paper citation, joint blog post, conference co-presence)
---
## 7. Governance
### 7.1 Maintainer model
- **Waggle org core maintainers** (2-3 Waggle team members) — decide roadmap, merge final PRs, own releases
- **Trusted committers** (community members with merged PRs + reputation) — review code, label issues, resolve disputes
- **Contributors** (anyone with a merged PR) — visible in CONTRIBUTORS.md
No BDFL drama — roadmap decisions happen in public issues.
### 7.2 Feature-contribution policy
- **Additive features accepted** (new adapters, new integrations, new embedder types)
- **Core schema changes require RFC** — open an issue, discuss, vote, merge
- **Breaking changes in minor versions acceptable pre-1.0, forbidden post-1.0**
- **No features that duplicate Waggle-product surface** (e.g., we won't accept a PR that adds the full skill-promotion flow from `packages/agent/src/skill-tools.ts`; those stay product-side)
### 7.3 Handling "enterprise wants X feature" in OSS
If a KVARK customer pays for a feature in Waggle's proprietary layer that *could* also benefit OSS (e.g., "distillation with constitutional AI"), default: the constitutional-AI adapter goes to Waggle, the *hook point* that accepts a pluggable constitutional-AI adapter goes to `hive-mind`.
This is the standard commercial-OSS split: the extension point is open, the specific implementation may be commercial.
---
## 8. Metrics
### 8.1 Vanity metrics (track but don't optimize for)
- GitHub stars — target: 2k in 3 months, 5k in 12 months
- PyPI/npm downloads — target: 10k/month in 6 months (if we publish language-specific packages)
### 8.2 Leading indicators (actually matter)
- Active contributors/month — target: 10+ by month 3
- Merged PRs from non-Waggle authors / total merged PRs — target: 30%+ by month 6
- Issues opened by non-Waggle users — proxy for engagement
- Third-party projects citing hive-mind — target: 5 in 6 months
- Conference/meetup talks mentioning hive-mind — target: 3 in 6 months
### 8.3 Strategic (compound)
- KVARK enterprise conversations citing OSS — target: 2+ per month mentioning hive-mind awareness by month 12
- Waggle product signups from hive-mind users — target: measurable conversion funnel
- Research paper citations — target: 10+ by month 12
---
## 9. Risks + mitigations
| Risk | Mitigation |
|---|---|
| Nobody cares (low star count → demoralizing) | Pre-launch warm list for quality reception; accept slow organic growth if genuine |
| Massive star count but no real users (GitHub vanity without adoption) | Track *engagement* metrics above stars; surface them in a README badge |
| A cloud-vendor forks and builds managed service | Apache 2.0 allows this; our moat is the product layer, not the primitives. Don't optimize for "preventing fork"; optimize for "Waggle is still the best home" |
| Community PRs break scope (people try to land agent logic in hive-mind) | Clear CONTRIBUTING.md with scope; maintainer discipline |
| Technical debt accumulates because we treat hive-mind as "done" | Quarterly investment commitment: N hours per quarter from Waggle team in hive-mind maintenance |
| Community split (hostile fork) | Make contribution easy and welcome; respond to constructive critique; preserve right-to-fork as a feature, not a failure |
| Dependency drift (sqlite-vec breaks, better-sqlite3 API changes) | Test matrix in CI; automated dependency updates via Dependabot; pin minor versions |
| License decision creates future regret | Permissive (Apache 2.0) decisions are reversible into proprietary much harder than the reverse; start permissive, narrow only if absolutely required |
---
## 10. Interaction with Waggle commercial strategy
### 10.1 Freemium funnel
- **hive-mind** (OSS) — developer's entry point. Free forever.
- **Waggle Free tier** — product's entry point. Free forever.
- **Waggle Pro / Teams** — monetized product.
- **KVARK / Enterprise** — sovereign enterprise deal.
Developers who discover hive-mind → build a prototype → realize they need the full product → signup Waggle Free → upgrade. Also: enterprise buyers evaluating hive-mind → realize they need commercial support + advanced features → KVARK conversation.
**Critical: the OSS version must be genuinely useful on its own, not a crippleware teaser.** A developer shouldn't hit a wall in hive-mind and need to install Waggle to do a basic thing. The split is along *product layer*, not *feature flag*.
### 10.2 Cross-promotion
- Waggle README links to hive-mind
- hive-mind README links to Waggle *only* on the last "Want the full product?" line, not front-loaded
- Shared design system (Hive DS) links both visually
- Case studies that involve both reference both
### 10.3 Support / SLA
- OSS: issues + PRs, community Discord, best-effort response
- Waggle Pro / Teams: email support, business-day SLA
- KVARK: dedicated support engineer, 24/7 on Enterprise plans
Clear tier of support is itself a Waggle-product selling point over "just use OSS."
---
## 11. Open questions for Marko
1. **Timing relative to v2 hypothesis reveal** — recommend publish ~30-60 days after the reveal (compounds the attention). Is this compatible with your priorities?
2. **Resource allocation** — 1 eng allocated 1 day/week to hive-mind maintenance is enough initially. Ok with that commitment?
3. **Cross-team (Egzakta) alignment** — OSS strategy needs Egzakta board / leadership buy-in. Who do we need to preview this with?
4. **Name confirmation** — `hive-mind` the working name; any objection from Egzakta marketing / trademark? Check `hive-mind.org` and GitHub availability.
5. **Partial-open vs full-open on harvest adapters** — the 11 adapters are the gateway for user corpus ingestion. Open-source them entirely, or keep perplexity/claude-desktop/cursor (strategic targets) proprietary to drive Waggle-product adoption? My recommendation: **all 11 open**, no gatekeeping. If a competitor forks and builds a better harvest UI, that's fine — our moat is elsewhere.
6. **Spin-off company?** — does hive-mind ever become its own entity (spinout with VC backing), or always stay a Waggle-led OSS project? Recommend: stay Waggle-led unless a compelling offer emerges; spinouts are expensive distractions.
---
## 12. Action plan — 12-week sprint
| Week | Deliverable |
|---|---|
| 1 | License + repo scaffolding; Apache 2.0 committed; skeleton README + CONTRIBUTING + CODE_OF_CONDUCT |
| 2 | Extract `packages/core/src/frames.ts`, `search.ts`, `knowledge.ts`, `identity.ts`, `awareness.ts` into hive-mind; rewrite imports; standalone tests pass |
| 3 | Extract `packages/weaver/` + `packages/core/src/harvest/` adapters; standalone tests pass |
| 4 | Extract `packages/wiki-compiler/` skeleton; standalone tests pass |
| 5 | Docs polish: quickstart, 5 cookbook tutorials, architecture overview |
| 6 | Private beta with 20-30 warm-list reviewers |
| 7 | Feedback incorporation round 1 |
| 8 | Private beta round 2 (broader, 50+ reviewers) |
| 9 | Feedback round 2; last polish |
| 10 | Launch-week content prep: blog post, Twitter thread, HN draft, newsletter outreach |
| 11 | Launch! + first-week engagement |
| 12 | Retrospective + v0.2 roadmap |
Compressible to 8 weeks if resources allow; realistic at 12.
---
## Closing
`hive-mind` is a strategic ecosystem play, not a product in its own right. It positions Waggle as the adult in the memory-for-agents room, attracts talent and credibility, lubricates enterprise sales cycles, and gives Egzakta's KVARK story a compelling "open foundation" narrative. The license choice (Apache 2.0), the scope choice (primitives only, not product), the timing choice (post-hypothesis-reveal), and the governance choice (Waggle-led, community-welcoming) all reinforce each other.
Done right, in 12 months: 5k stars, 50+ contributors, 3+ research citations, 10+ named third-party integrations, and a measurable KVARK enterprise pipeline expansion attributable to the OSS presence. Done wrong: a ghost-town repo and a distraction from product work. The difference is largely about maintainer commitment and the quality of the launch narrative — both of which Waggle is uniquely equipped to get right.
*Competitor-specific data (stars, funding, activity metrics) will be folded in when Agent A's OSS memory landscape research returns.*

445
docs/research/02-memory-system-scientific-draft.md Normal file
View File

@@ -0,0 +1,445 @@
# Hive-Mind: Persistent, Multi-Tenant, Compliance-First Memory for Autonomous AI Agents
**Authors:** Marko Markovic (Egzakta Group), Waggle OS team
**Date:** Draft — 2026-04-15
**Status:** Working paper. Numbers and external evaluation pending the v2 hypothesis run + agent-architecture verification. This is the structural + conceptual draft for internal review.
**Target venue:** arXiv cs.AI preprint; potential conference workshop (ACL / NeurIPS Agents / ACL NLP-for-agents).
---
## Abstract
We present Hive-Mind, the memory system powering Waggle OS, a workspace-native AI agent platform for single-user through enterprise deployments. Hive-Mind departs from the dominant key-value and turn-blob memory designs in three ways. First, it models memory as a temporal **frame graph** with three frame types (Independent/Predicted/Bidirectional) inspired by video-compression encoding, allowing consolidation, decay, and link formation to operate over semantic clusters rather than individual turns. Second, it treats **multi-mind isolation** (personal, workspace, team, enterprise scope) as a first-class concept rather than an add-on, with provenance tracking and approval-gated cross-mind reads. Third, it ships **compliance-by-default** — every interaction is logged in a format that maps directly to EU AI Act Articles 10, 12, 14, 19, 26, and 50, producing audit reports as a first-class output rather than a post-hoc integration. On top of this foundation, Hive-Mind compiles a living **wiki** of entity, concept, and synthesis pages from the frame corpus, giving users a navigable second-brain over their unstructured conversational history. We describe the system architecture, the frame model's formal properties, the multi-mind consistency protocol, the compilation pipeline, and the integration with Waggle's closed-loop prompt-evolution subsystem. We report preliminary results on write-path contradiction detection and skill auto-extraction, and outline a roadmap for empirical evaluation against existing memory libraries.
---
## 1. Introduction
Autonomous AI agents that maintain long-term state are an active area of research and deployment [Letta/MemGPT; mem0; Zep; GraphRAG]. The dominant designs fall in three categories:
- **Turn-blob / scrollback memory:** store full conversation history; retrieve by vector similarity. Exemplified by naive chatbot memory and early ChatGPT custom-instructions.
- **Key-value memory:** structured facts with keys; updated on explicit user signal. Exemplified by mem0's API-first design and OpenAI's "Memory" feature.
- **Graph-augmented memory:** entity-relation graphs over conversational content; retrieval combines graph walks with vector search. Exemplified by Zep, GraphRAG.
Each design solves specific problems and sidesteps others. **Turn-blob** is high-fidelity but has catastrophic recall-cost growth. **Key-value** is efficient but loses context and struggles with temporal reasoning ("what did we decide when we talked about X?"). **Graph-augmented** combines well but requires either expensive LLM calls for every write (for entity extraction) or heavy upfront schema work.
Hive-Mind takes a **fourth position:** memory is best modeled as a temporal graph of frames with explicit continuation semantics. Combined with multi-mind isolation and compliance-grade auditing, this gives a memory system that is simultaneously rich, performant, multi-tenant, and enterprise-ready.
Our contributions:
1. The **I/P/B frame model** for AI memory, borrowed from video compression, with formal decay/consolidation semantics
2. A **multi-mind protocol** for isolating personal, workspace, team, and enterprise memory while supporting sanctioned cross-mind access
3. **Write-path contradiction detection** that emits actionable improvement signals rather than silently accepting conflicting data
4. **Skill promotion** as an organizational mechanism on top of memory: scope-graded extraction and distribution of re-usable prompt modules, with tier gates and audit trails
5. A **wiki compilation layer** that produces structured entity/concept/synthesis pages from the frame corpus incrementally
6. A **compliance-by-default** interaction log with direct mapping to EU AI Act articles, producing audit-ready PDFs as a primary output
The system is implemented in a dual-license configuration: the memory primitives are slated for Apache 2.0 release as `hive-mind`; the product layer (agent runtime, evolution stack, UI, compliance reporting) remains in Waggle's commercial repository.
### Verified scope numbers (from code inventory, 2026-04-15)
- **10 harvest adapters** (chatgpt, claude, claude-code, gemini, perplexity, markdown, plaintext, pdf, url, universal)
- **18 canonical personas** with tool allowlists + failure patterns
- **13 SQL tables** across 8 schema layers; sqlite-vec virtual table at 1024-dim embeddings; WAL mode
- **RRF_K = 60** for hybrid search fusion
- **4 scoring profiles** (balanced / recent / important / connected) with explicit weight vectors
- **60+ native agent tools** across memory / filesystem / git / web / planning / workflow / compliance categories
---
## 2. System architecture
### 2.1 High-level
Hive-Mind is a library, not a service. It runs in-process with the agent runtime. Storage is SQLite (via better-sqlite3) with `sqlite-vec` for embedded vector search. A single deployment is a single `.mind` file; multi-mind deployments have multiple `.mind` files coordinated by a cache + multiplex layer.
```
┌────────────────────────────────────────────────────┐
│ Agent Runtime (Waggle or user-provided) │
├────────────────────────────────────────────────────┤
│ Hive-Mind API surface │
│ FrameStore HybridSearch KnowledgeGraph │
│ IdentityLayer AwarenessLayer CognifyPipeline │
│ MemoryWeaver WikiCompiler │
├────────────────────────────────────────────────────┤
│ SQLite + sqlite-vec │
│ memory_frames knowledge_entities … │
└────────────────────────────────────────────────────┘
```
### 2.2 The frame model
A frame is the atomic storage unit. Every frame has:
```
{
id, gop_id, frame_type ∈ {I, P, B},
content,
importance ∈ {critical, important, normal, temporary, deprecated},
source ∈ {user_stated, tool_verified, agent_inferred},
confidence ∈ {high, medium, low, unverified},
access_count, created_at, updated_at
}
```
Where:
- **I-frame (Independent):** a keyframe; complete, self-contained content.
- **P-frame (Predicted):** a continuation delta with a reference to an earlier I-frame in the same GOP. Represents "this adds to / refines / extends" the I-frame's content.
- **B-frame (Bidirectional):** a link between two frames expressing a structural relation (shared entity, causal link, contradiction flag).
A *Group of Pictures* (GOP, identified by `gop_id`) corresponds roughly to a session or thread. Within a GOP, there is typically one I-frame followed by N P-frames; B-frames may link across GOPs.
#### 2.2.1 Why this model
The design choice encodes a bet: **most useful memory is not a flat sequence of assertions but a branching temporal graph with explicit grouping and refinement semantics.** A conversation about "the Acme contract" produces one I-frame (the core facts) and many P-frames over time (amendments, clarifications, decisions). Consolidation (§2.3) periodically merges P-frames back into the I-frame, upgrading its richness while preserving the decay-friendly property that *old unused P-frames can be discarded without losing the core memory*.
This mirrors the intuition behind MPEG video encoding: the expensive keyframes carry structure; the cheap predicted frames carry deltas; the bidirectional frames carry cross-reference structure.
#### 2.2.2 Formal properties
Let frames be $F_i$ with I/P/B tags. Let $\text{gop}(F_i)$ denote the GOP, $\text{parent}(F_i)$ the referenced I-frame for a P-frame.
- **Containment:** for each GOP $g$, there exists at least one $F_i$ with $\text{gop}(F_i) = g$ and $\text{frame\_type}(F_i) = I$.
- **Delta referential integrity:** for every P-frame $F_p$, $\exists F_I$ with $\text{gop}(F_I) = \text{gop}(F_p) \land \text{frame\_type}(F_I) = I \land \text{id}(F_I) = \text{parent}(F_p)$.
- **B-frame validity:** every B-frame references two valid non-deprecated frames.
These invariants are maintained as DB constraints + application-level checks in `FrameStore`.
### 2.3 MemoryWeaver — consolidation, decay, strengthening
On a scheduled cadence (hourly consolidation, daily decay, weekly strengthening):
- **`consolidateGop(gop_id)`:** gather all P-frames under the GOP's current I-frame; merge content; emit a new I-frame; mark old P-frames `deprecated`.
- **`decayFrames()`:** delete `deprecated` frames with `access_count == 0`. (Non-destructive decay: frames that were read recently survive longer.)
- **`strengthenFrames()`:** promote `temporary → normal → important` for frames with high access counts.
- **`linkRelatedFrames(kg)`:** for each entity in the knowledge graph, find frames that mention it; create B-frames for pairs not yet linked.
This gives the memory a self-maintaining property: the useful content densifies, the irrelevant content dissolves, and cross-cutting connections emerge automatically.
### 2.4 HybridSearch with RRF
Query pipeline:
1. Embed the query with the configured embedder (Ollama / LiteLLM / API).
2. Vector search via `sqlite-vec` with cosine similarity.
3. Keyword search via SQLite FTS5.
4. Fuse via **Reciprocal Rank Fusion** (RRF_K = 60):
$\text{score}(d) = \sum_{r \in \{\text{vec}, \text{kw}\}} \frac{1}{K + \text{rank}_r(d)}$
5. Apply scoring profile (balanced / recent / important / connected) — weighted reranking by timestamp, importance enum, or connection-count signal.
Scoring profiles are important: the same query may want different results depending on the user's task (`recent` for "what did we discuss yesterday?", `connected` for "what's relevant to this entity?"). Making the profile explicit in the API surface lets agents request the appropriate prior.
### 2.5 Knowledge graph with SCD-2
Entities + relations have `valid_from` / `valid_to` columns. When an entity is updated, the old row is closed and a new row opened. This gives temporal reasoning: "what did we believe about X on date D?" is answerable as a point-in-time query.
Auto-population via `CognifyPipeline`: on every frame save, an LLM extracts entities and relations; new ones are inserted; existing ones matched by name (with fuzzy match) are strengthened in access_count.
### 2.6 Identity and awareness layers
Two lightweight layers orthogonal to the frame graph:
- **IdentityLayer** stores stable user facts — name, role, organizations, preferences, communication style. Auto-derived during onboarding from the first N harvested conversations via a one-shot LLM extraction. User-editable.
- **AwarenessLayer** stores volatile state — active tasks, recent tool uses, pending approvals, flags. Refreshed every agent turn.
Both are used at prompt construction time to give the agent context without searching memory for basics.
### 2.7 Harvest pipeline
Ingest foreign conversational data from 11 source types via adapter interface:
```
SourceAdapter {
sourceType: ImportSourceType,
parse(input: unknown): UniversalImportItem[]
}
```
Adapters include chatgpt, claude, claude-code, claude-desktop, gemini, perplexity, markdown, plaintext, pdf, url, universal. Each normalizes its source-specific format to a common `UniversalImportItem` which then flows through classification → extraction → distillation → frame save. Multi-layer deduplication (exact match + normalized string match + embedding cosine similarity > 0.95 + cross-mind check) prevents re-imports from double-writing.
Harvest provenance is preserved: every frame from a harvest carries `originalSource`, `originalId`, `importedAt`.
### 2.8 Write-path contradiction detection
Introduced as of this work's 2026-04 revision, every `save_memory` call runs `detectContradiction(newContent, recentFrames)` before writing. When detected:
1. The frame is still written (contradictions are data, not errors).
2. The return string is annotated `[flag: contradicts_existing (excerpt)]` so the caller / user can see the conflict.
3. A `correction`-category `ImprovementSignal` is emitted with pattern key `write-conflict:<normalized>` and metadata capturing both sides.
Downstream, these signals flow into the closed-loop evolution subsystem (§4).
### 2.9 Compliance-by-default interaction store
Independent of the frame store, every interaction (LLM call, tool call, approval event) is logged with:
```
{ id, timestamp, workspaceId, sessionId, model, provider,
inputTokens, outputTokens, costUsd, toolsCalled,
humanAction ∈ {approved, denied, modified, none},
riskContext, importedFrom, persona }
```
This maps to AI Act Article 12 (logging of AI system operation). The `humanAction` field maps to Article 14 (human oversight). The `riskContext` field maps to Article 26 (risk classification). The log is append-only and retained per configurable retention policy, maps to Article 19 (retention).
A `ComplianceStatusChecker` and `ReportGenerator` produce structured audit reports with per-article pass/warn/fail status, model inventory, oversight log, harvest provenance, and interaction counts. A separate renderer produces a boardroom-grade PDF (styled with Hive DS tokens, locale-pinned to en-US for cross-jurisdiction portability).
### 2.10 Multi-mind protocol
Personal, workspace, team, and enterprise minds are separate `.mind` files. The `MultiMindCache` opens and closes them on demand via LRU. The agent's orchestrator injects `WorkspaceLayers` into tool dependencies so read/write tools route to the right mind.
Cross-mind access requires approval: the `read_other_workspace` tool triggers an approval gate (which can be granted per-session, permanent, or denied). This makes cross-mind read an *event* that's auditable, rather than a silent capability.
Team minds are synchronized via a `TeamSync` layer that pushes/pulls frames (currently frame-only; skill and KG sync are roadmap).
### 2.11 Wiki compiler
On demand or on cadence, the compiler produces structured Markdown pages from the frame corpus:
- **Entity pages** (one per high-connectivity entity): who/what is this, timeline of mentions, related entities, relevant frames
- **Concept pages** (thematic clusters): what is this topic, who's mentioned it, what's been decided
- **Synthesis pages** (query-driven): given a question, compile an answer from relevant frames with inline citations
Incremental: only entities/concepts with new frames since last compilation get rebuilt. The compilation is an LLM call; for 1k-frame minds with ~50 entity pages, full compile typically completes in minutes.
The wiki output is browsable in Waggle's Memory app and exportable as a static site.
---
## 3. Skill promotion: organizational memory
Adjacent to the memory core, we ship a skill-promotion mechanism. A *skill* is a versioned Markdown file with YAML frontmatter (name, description, scope, permissions) plus a body describing how to accomplish a specific task well.
Scopes: **personal → workspace → team → enterprise**. A skill is promoted one rung at a time with tier gating (team requires `teamSkillLibrary` capability; enterprise requires ENTERPRISE tier). Each promotion:
1. Rewrites frontmatter (scope + appends to `promoted_from` history)
2. Moves the file to the scope-appropriate directory
3. Emits a `skill_promotion`-category `ImprovementSignal`
4. Is rollback-safe (old copy preserved on target-collision)
This mechanism transforms skills from a personal-tool artifact into a **governed organizational knowledge asset**. A senior legal counsel authors the `nda-review` skill for personal use, promotes to team after vetting, promotes to enterprise after compliance review. Every version of every skill has an author trail and a deployment trail.
Skills also auto-extract from conversation history: when the same tool-sequence repeats 2+ times, `autoExtractAndCreateSkill` produces a draft SKILL.md from the detected pattern. Users accept/reject the suggestion; accepted skills enter the personal scope and can then be promoted.
Decay: skills unused for ≥90 days are archived to a recoverable `skills-archive/` directory — keeps the library lean without data loss.
---
## 4. Closed-loop prompt evolution integration
Hive-Mind's companion evolution subsystem (open-source roadmap pending but not in the initial `hive-mind` OSS split) closes the loop from memory → improvement signals → prompt mutations → judged candidates → deployed prompts.
Specifically: `ExecutionTraceStore` records every chat turn with input/output/score. The eval-dataset builder curates held-out subsets from these traces. `IterativeOptimizer` runs a GEPA-style rank-by-score mutation loop. `EvolveSchema` (integration targeting Mikhail's 2026 paper) runs structural schema-level evolution. `ComposeEvolution` combines signals from both. `EvolutionGates` enforce constraints. `EvolutionOrchestrator` coordinates end-to-end runs with accept/reject gates presented to the user.
Preliminary result: an internal v1 evaluation (n=10 coder questions, 4 blind multi-vendor judges) measured Gemma 4 31B (Google, released April 2026 under Apache 2.0, Arena #3 open at 1452 Elo) with a Waggle-evolved prompt at **108.8 % of raw Claude Opus 4.6** per-judge mean (v2 scaling to n=60 × 3 domains × 3 baselines with hard train/test split is pending execution). This is discussed in a companion paper and is not the focus of this work; mentioned here because the evolution subsystem is architecturally dependent on the memory subsystem described above.
The evolution subsystem integrates **GEPA** — short for *Genetic-Pareto* per Agrawal et al. (arXiv:2507.19457, ICLR 2026 Oral), which uses reflective prompt evolution with a Pareto frontier of candidates across multiple objectives, outperforming RL baselines (GRPO) by +6 % average and +20 % max with up to 35× fewer rollouts, and the previous best prompt optimizer (MIPROv2, Opsahl-Ong et al., arXiv:2406.11695) by >10 % — and a structural-schema evolution layer whose closest public analog is **ACE** (Agentic Context Engineering, Zhang et al., arXiv:2510.04618), a Stanford/SambaNova line of work on incremental playbook evolution that avoids context collapse in long-running agents. Waggle's contribution is the integration of these signals against a persistent memory corpus + LLM-as-judge scoring + constraint-gated deployment — not a novel core algorithm. Prior related work: DSPy (Khattab, arXiv:2310.03714, ~28k GitHub stars, ~160k monthly PyPI downloads); TextGrad (Yuksekgonul et al., Nature 2025); OPRO (Yang et al., arXiv:2309.03409); Promptbreeder (DeepMind, arXiv:2309.16797); APE (Zhou et al., arXiv:2211.01910).
---
## 5. Evaluation
*(To be completed with real numbers post-implementation validation and post-benchmark suite.)*
### 5.1 Evaluation plan
We propose three evaluation axes:
#### 5.1.1 Retrieval quality
- **Dataset:** a synthesized 10,000-frame corpus covering a synthetic multi-month conversation history across 5 domains (work, family, health, finance, hobby).
- **Queries:** 500 questions with known-answer frames (oracle provides ground truth).
- **Baselines:** mem0 (API-mapped equivalent), Zep, Letta archival-vs-recall, LangChain's ConversationBufferMemory + vector store.
- **Metrics:** recall@5, recall@20, MRR, latency p50/p95.
- **Hypothesis:** Hive-Mind's RRF hybrid + frame-model recency signal outperforms pure vector baselines by ≥15% on recall@5 for ambiguous queries.
#### 5.1.2 Consolidation quality
- **Dataset:** a 30-day conversation simulation with 10-20 turns per session.
- **Metric:** After 30 days of simulated use + MemoryWeaver running on schedule, what's the delta between pre-consolidation frame count and post? What's the loss on a 100-query held-out test of the pre-consolidation content?
- **Hypothesis:** 50-70% frame-count reduction with ≤ 5% answer-quality loss.
#### 5.1.3 Compliance report fidelity
- **Method:** a mock AI-Act audit scenario (50 regulator-style questions against a 30-day interaction log).
- **Metric:** does the `AuditReport` answer each question correctly from the structured log?
- **Hypothesis:** 100% structured-answerable; 0% requiring manual data archaeology.
### 5.2 Preliminary numbers
*From the Waggle production deployment (personal mind seeded with 156 real Claude Code frames per prior telemetry):*
- Median query latency over 1,200-frame mind: under 100ms for balanced-profile search *(preliminary; requires instrumented benchmarking)*.
- Write-path contradiction detection: 3 unit-test cases shipped, production-path integration verified. Field validation pending larger corpora.
- Skill auto-extract: 7 unit-test cases shipped; triggers on ≥3-tool-sequence + ≥2 repetitions.
- Compliance PDF generation: 11 unit-test cases shipped covering all 5 article status blocks + empty-state fallbacks + 50-cap oversight log truncation.
*Full evaluation suite is pending. The claim of this paper is the architecture and the integration, not the benchmark dominance — that will require its own follow-up.*
---
## 6. Related work
**Memory for agents.** Letta/MemGPT [Packer et al. 2023] introduced archival vs recall memory paging as an LLM-native OS analogue. mem0 [Mem0 Team, 2024+] packages memory as a dev API. Zep [Zep Team, 2024+] combines knowledge graphs and temporal recency. Cognee focuses on graph reasoning + memory. GraphRAG [Microsoft, 2024] applies graph retrieval to multi-hop queries. LangChain's memory modules provide per-turn aggregation.
**Differentiators.** Hive-Mind departs from these by: (a) the frame model, which models continuation / refinement explicitly rather than as flat turn-list; (b) write-path contradiction detection + signal emission, which treats contradictions as improvement opportunities rather than silent conflicts; (c) multi-mind isolation with sanctioned cross-access as first-class; (d) compliance-by-default auditing; (e) wiki compilation for navigable memory.
**Video-compression analogy.** The I/P/B frame model borrows the vocabulary from MPEG and H.264 [Wiegand et al.] but not the compression mathematics. The analogy is conceptual: keyframes carry structure, predicted frames carry deltas, bidirectional frames carry cross-reference. We don't claim a coding-theoretic optimality result for memory; we claim the conceptual analogue is useful and solves real problems that flat-turn designs struggle with.
**Prompt evolution and self-improvement.** Integration with the GEPA + EvolveSchema [to be cited] stack is the companion work; see §4.
**Compliance and governance.** The AI-Act mapping is informed by EU AI Office guidance and NIST AI RMF 2.0. We are not aware of other OSS memory systems shipping compliance-by-default audit generation as a first-class feature.
---
## 7. Limitations
- **Single-node.** SQLite-backed; we do not yet claim support for multi-node replication or horizontal scaling. The multi-mind protocol anticipates multi-tenant isolation but not distributed consensus.
- **LLM dependency for ingest.** CognifyPipeline and the distillation path require an LLM for entity extraction and summarization. Offline-only deployments fall back to raw frame storage without KG enrichment.
- **Adapter coverage.** 11 harvest sources today; ecosystem gaps include Cursor, Copilot, Notion, Obsidian, voice-transcript services. Adapter addition is straightforward but each requires source-specific parsing.
- **Compliance scope.** We map to EU AI Act today; HIPAA, SOC2, ISO 27001 mappings are conceptual and require bespoke configuration per deployment.
- **Benchmark gap.** The evaluation section above is a proposal rather than completed empiricism. A full multi-memory-system benchmark is a follow-up effort.
- **Skill promotion model.** Four-scope model (personal / workspace / team / enterprise) may not fit all organizational structures. Flat team-member-level permissions may be desired for some users.
- **Frame size.** Frames today are single-content strings; richer content (images, code, video references) is tagged metadata rather than first-class frame content. Multi-modal memory is future work.
---
## 8. Conclusion
Hive-Mind is a memory system for AI agents that takes four architectural bets: (1) temporal frames with I/P/B typing are a better primitive than flat turn-blobs or key-value; (2) multi-mind isolation should be first-class, not retrofitted; (3) contradictions and skill-learning should trigger structured improvement signals, not silent overwrites; (4) compliance should be the default, not the upsell.
We believe each bet is independently defensible and jointly position the system to serve a range of users from solo prosumers to regulated enterprises. We ship the core memory primitives as Apache 2.0 open-source via the `hive-mind` project and integrate them commercially in Waggle OS with additional skill promotion, evolution, and UI layers.
The empirical work is ahead of us. The architecture is available now. We invite the research and practitioner community to build on, critique, and extend it.
---
## 9. Code availability
- **hive-mind** (Apache 2.0): `github.com/waggle-os/hive-mind` *(publication pending; 12-week sprint planned per report 01)*
- **Waggle OS** (commercial + open product): `waggle-os.ai`
- **Sovereign enterprise tier (KVARK):** `www.kvark.ai`
- **v2 hypothesis reproducibility repo** (companion work): `github.com/waggle-os/evolution-hypothesis-v2` *(forthcoming with v2 run)*
---
## Acknowledgments
Marko Markovic leads Waggle OS as part of Egzakta Group's AI portfolio. The evolution subsystem builds on GEPA (Agrawal et al., ICLR 2026 Oral) and ACE (Zhang et al., 2025) as the closest public analogs for reflective prompt evolution and context-playbook evolution respectively. The prior "Mikhail's EvolveSchema" internal reference in our project memory could not be definitively tied to a specific public paper; we cite ACE as the closest published analog and flag this for correction if the original reference is located. The compliance mapping draws on EU AI Office interpretive guidance and NIST AI RMF 2.0. Contributors to the Waggle OS codebase are listed in the repository.
---
## References
*(Placeholder — to be populated with proper citations during final editorial pass. Confirmed-needed-citations below.)*
- Packer et al., MemGPT: Towards LLMs as Operating Systems, 2023
- Microsoft Research, GraphRAG: Unlocking LLM discovery on narrative private data, 2024
- Wiegand et al., Overview of the H.264/AVC Video Coding Standard, IEEE 2003
- Khattab et al., DSPy: Compiling Declarative Language Model Calls, 2023
- Yuksekgonul et al., TextGrad, 2024
- Yang et al., OPRO: Large Language Models as Optimizers, 2023
- Fernando et al., Promptbreeder, 2023
- Zhou et al., APE: Automatic Prompt Engineer, 2022
- GEPA paper — citation pending from overnight research
- EvolveSchema paper (Mikhail et al., 2026) — citation pending from overnight research
- Regulation (EU) 2024/1689 (AI Act)
- NIST AI Risk Management Framework 2.0
---
## Appendix A — SQL schema excerpt (illustrative)
*(Illustrative schema. See `packages/core/src/mind/schema.ts` for the canonical definitions. Agent D's overnight architectural map will replace this appendix with the authoritative schema listing.)*
```sql
CREATE TABLE memory_frames (
id INTEGER PRIMARY KEY AUTOINCREMENT,
gop_id TEXT NOT NULL,
frame_type TEXT NOT NULL CHECK (frame_type IN ('I','P','B')),
content TEXT NOT NULL,
parent_id INTEGER,
importance TEXT NOT NULL DEFAULT 'normal'
CHECK (importance IN ('critical','important','normal','temporary','deprecated')),
source TEXT DEFAULT 'agent_inferred',
confidence TEXT,
access_count INTEGER NOT NULL DEFAULT 0,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX idx_frames_gop ON memory_frames(gop_id);
CREATE TABLE knowledge_entities (
id INTEGER PRIMARY KEY AUTOINCREMENT,
name TEXT NOT NULL,
type TEXT NOT NULL,
valid_from TEXT NOT NULL DEFAULT (datetime('now')),
valid_to TEXT
);
CREATE TABLE knowledge_relations (
id INTEGER PRIMARY KEY AUTOINCREMENT,
source_id INTEGER NOT NULL,
target_id INTEGER NOT NULL,
relation_type TEXT NOT NULL,
confidence REAL,
valid_from TEXT NOT NULL DEFAULT (datetime('now')),
valid_to TEXT,
FOREIGN KEY (source_id) REFERENCES knowledge_entities(id),
FOREIGN KEY (target_id) REFERENCES knowledge_entities(id)
);
CREATE TABLE improvement_signals (
id INTEGER PRIMARY KEY AUTOINCREMENT,
category TEXT NOT NULL CHECK (category IN (
'capability_gap','correction','workflow_pattern','skill_promotion'
)),
pattern_key TEXT NOT NULL,
detail TEXT NOT NULL DEFAULT '',
count INTEGER NOT NULL DEFAULT 1,
first_seen TEXT NOT NULL DEFAULT (datetime('now')),
last_seen TEXT NOT NULL DEFAULT (datetime('now')),
surfaced INTEGER NOT NULL DEFAULT 0,
surfaced_at TEXT,
metadata TEXT NOT NULL DEFAULT '{}'
);
CREATE UNIQUE INDEX idx_signals_category_key ON improvement_signals(category, pattern_key);
```
*Note: FTS5 tables and sqlite-vec virtual tables are declared alongside — see the authoritative schema in the source tree.*
---
## Appendix B — Hive-Mind public API sketch (TypeScript)
```ts
import { MindDB, FrameStore, HybridSearch, KnowledgeGraph,
IdentityLayer, AwarenessLayer } from 'hive-mind';
const db = new MindDB(path.join(home, '.mind'));
const frames = new FrameStore(db);
const search = new HybridSearch(db, embedder);
const kg = new KnowledgeGraph(db);
// Write
const session = sessions.create();
frames.createIFrame(session.gop_id, 'Alice suggested we use Postgres.', 'normal');
// Query
const results = await search.search('what did Alice suggest', { profile: 'recent', limit: 10 });
// KG walk
const alice = kg.searchEntities('Alice')[0];
const outRelations = kg.getRelationsFrom(alice.id);
// Consolidation
const weaver = new MemoryWeaver(db, frames, sessions);
setInterval(() => weaver.consolidateGop(session.gop_id), 3600_000);
```
The commercial Waggle product wraps this API with its agent runtime, skill system, UI, and evolution stack.
---
*End of working draft. Architectural details, citations, and empirical numbers to be refined when overnight research agents return and when the v2 hypothesis run completes. This draft exists so the structural narrative is reviewable before the details are finalized.*

281
docs/research/03-memory-harvesting-strategy.md Normal file
View File

@@ -0,0 +1,281 @@
# Memory Harvesting — Strategy, Action Plan, and UX
**Author:** Waggle OS research series (3 of 7)
**Drafted:** 2026-04-15 (overnight batch)
**Scope:** End-to-end strategy for memory harvesting in Waggle OS — what it is, why it's a P0 moat, how users experience it today, how they *should* experience it, and the sequenced action plan to get there.
---
## TL;DR
Memory Harvest is Waggle's **lock-in moat.** Free forever as a Free-tier feature because the value compounds the longer someone harvests: their assistant gets *smarter about them*, and that smartness cannot be migrated to ChatGPT/Claude without leaving it behind. Per memory: "Memory + Harvest is free forever (lock-in moat). Agents are free (they generate memory). Skills and connectors are the upgrade trigger."
Today Waggle ships **9 adapters** (chatgpt, claude, claude-code, claude-desktop, gemini, plus markdown / plaintext / pdf / url / universal), with **perplexity just shipped in session S3**, and 156+ real Claude Code frames harvested in production per prior memory. The core pipeline is solid — `packages/core/src/harvest/pipeline.ts` handles universal-import → classification → extraction → distillation → frame save with dedup. What's missing is **UX polish that makes harvest feel like a superpower rather than an import chore.** This document lays out the current state, the target state, the six UX principles that should govern every harvest surface, and the phased action plan to get there across Q2 and Q3 2026.
---
## 1. Why Memory Harvest is strategic, not just a feature
### 1.1 The lock-in thesis
Every AI user has a shadow-corpus of conversations — hundreds to thousands of exchanges — scattered across ChatGPT, Claude, Gemini, Perplexity, Cursor, and (for serious users) local markdown notes. Today that corpus is an asset that benefits *the tool vendor*, not the user. When the user switches tools, they leave it behind.
Waggle's bet: **the user owns their corpus**, Waggle makes it portable and searchable, and the assistant that uses it gets smarter at a compounding rate. Once a user has harvested 1,000+ frames into Waggle, switching back is psychologically expensive even if the competitor is free — because those 1,000 frames become part of the user's working identity.
This is the same lock-in mechanic that made Gmail, Evernote, and Notion sticky. The difference: Waggle's memory is structured (entities + relations + sessions + conflicts), not just a pile of messages.
### 1.2 Adjacent strategic wins
- **Memory Harvest defends the EU AI Act compliance positioning.** Every harvested frame has provenance (source, importedAt, originalId). That's exactly the data trail Art 10 (data quality) and Art 12 (logging) auditors ask for.
- **Memory Harvest is the demand-gen wedge for KVARK.** Enterprise buyers see the "free, local, harvest from everywhere" motion in Pro/Teams, then when they realize they can do this on-prem at firm scale with full audit, that's the KVARK conversation.
- **Memory Harvest is the training data moat for evolution.** The harvested corpus is the input to EvolveSchema — the more a user harvests, the better their personalized prompts evolve.
---
## 2. Current state (April 2026)
### 2.1 What's shipped
| Adapter | Source export format | Production-tested? |
|---|---|---|
| chatgpt | JSON (OpenAI data export) | ✅ via prior sessions |
| claude | JSON (Anthropic data export) | ✅ |
| claude-code | CLI session logs | ✅ 156 frames in production |
| claude-desktop | Claude desktop app export | ✅ |
| gemini | JSON (Takeout + API history) | ✅ |
| perplexity | JSON (threads/messages) | ✅ shipped S3 |
| markdown | Any .md tree | ✅ |
| plaintext | Any .txt tree | ✅ |
| pdf | pdf-parse | ✅ |
| url | fetch + readability | ✅ |
| universal | Generic conversation arrays | ✅ |
### 2.2 What's missing
- **Cursor adapter** — IDE-scoped export with code-context attachments. Estimated 0.5-1 day. On the backlog.
- **Copilot adapter** — blocked on GitHub shipping a first-class export API for Copilot Workspace / Copilot Chat.
- **Grok adapter** — nice-to-have; XAI export format is stable enough to implement.
- **Notion / Evernote / Bear / Obsidian (beyond raw md)** — structured note exports with backlinks.
- **Voice / video transcripts** — Fathom, Fireflies, Granola. Different data shape (speaker turns with timestamps).
- **Email / calendar** — Gmail takeout, ICS feeds. Harder privacy surface.
### 2.3 Pipeline state
- `packages/core/src/harvest/pipeline.ts` orchestrates: adapter.parse → classifier (work/personal/technical, high/medium/low/skip) → extractor (decisions/preferences/facts/knowledge/entities/relations) → distillation → frame save → KG linkage
- Dedup: multi-layer — exact content match, normalized match, embedding cosine similarity (>0.95 threshold), cross-mind check (personal ↔ workspace)
- Provenance: every frame carries `originalSource`, `originalId`, `importedAt`, `distillationModel`, `confidence`, `pass`
- Auto-sync: poll stale sources every 30 minutes (per `packages/server/src/local/index.ts` runHarvestAutoSync)
### 2.4 UX state today
From the onboarding wizard (`apps/web/src/components/os/overlays/OnboardingWizard.tsx`) and the MemoryApp:
- Users can import via a file-picker after onboarding
- There's a harvest manager page
- Status visibility: frames-imported count per source, last-synced timestamp
**What's missing from the UX:**
- No first-run "harvest-first" onboarding. New users land in an empty Waggle and have to go *find* the import tool.
- No progress indicator during a large import (imagine 10,000-frame ChatGPT export — users see a spinner and wonder if it's hung)
- No deduplication-preview ("found 156 new frames, 40 already known") — just a done message
- No live cognify streaming — frames arrive in batches after full parse completes
- No recovery from partial failures ("imported 847/1200 then network dropped — here's where to resume")
- No identity-auto-populate during first harvest ("I see you mentioned your team is 5 people — is that still right?")
---
## 3. Target state — six UX principles
These are the design principles every harvest surface should honor:
### 3.1 Principle 1: **Harvest-first onboarding.**
First-run experience should be: *connect your first source, see the first insight, be hooked.* Not "click around the empty app for 10 minutes first."
Concrete: onboarding step 2 (after tier selection) is "Where does your AI life live today?" with 6 big tiles — ChatGPT, Claude, Gemini, Perplexity, Cursor, Markdown files. Users click one, drop a file, see frames stream in live.
### 3.2 Principle 2: **Privacy is the headline.**
Every harvest surface repeats the same four words above the fold: **"Your data stays here."** With a one-click "where does this go?" that explains: local SQLite file, not uploaded, not shared, you can export or wipe anytime. Screenshot-ready.
### 3.3 Principle 3: **Dedup is a feature, not a footnote.**
Users should *see* their assistant getting smarter. After an import: "Found 847 frames. 203 were already known (deduplicated). 644 new. Of those, 89 mention people you already know — linking now…" That paragraph is worth 10× the marketing impact of "import successful."
### 3.4 Principle 4: **Progress is tactile.**
No silent spinners. Every harvest shows:
- Phase (reading / classifying / extracting / distilling / saving / linking)
- Progress bar per phase
- Live frame count climbing
- Current item title ("processing: '2025 Q3 strategy call with Samantha'")
- Cumulative ETA based on per-item rate
### 3.5 Principle 5: **Recovery is resumable.**
If a 10,000-frame import dies at 7,423, the next retry picks up at 7,424. Idempotent dedup makes this safe. Status: "Resuming from frame 7,424 of 10,000…" not "restart from scratch."
### 3.6 Principle 6: **Identity and awareness auto-populate.**
During first harvest, Cognify extracts people/projects/preferences. The wizard's next step is "Here's what I learned about you. Correct anything wrong." — inline editable entity list.
---
## 4. User journeys
### 4.1 First-time user, 10-minute onboarding
1. **0:00** — download + install Waggle
2. **0:30** — "Welcome. Your AI remembers you. Where does your AI life live today?" → ChatGPT tile selected → user drags in `conversations.json`
3. **1:00** — live-view: "Reading… 1,247 conversations. Classifying…" — frames stream into the Memory app visible on the right
4. **2:00** — first frame card lights up. Confetti micro-interaction (subtle, not gauche).
5. **2:30** — "While that finishes, let's add a second source" → Claude tile, drop → same flow, merged into the same mind
6. **5:00** — both imports complete. "Found 2,134 frames. 487 duplicates removed. Linked 143 people, 87 projects, 52 technologies."
7. **5:30** — "Here's what I know about you so far. Correct anything." → editable entity grid: "Your company: Egzakta (✏️), Your role: CEO (✏️), Projects: Waggle OS, KVARK (✏️)…"
8. **7:00** — user corrects a couple of things, clicks "this looks right"
9. **7:30** — "Now try asking me something you've discussed before. I'll pull the context." → chat opens with a prefilled example: "What did we decide about the tier pricing?"
10. **8:00** — assistant responds with a real answer citing real frames from the harvest. User's face: 😮
That's the hook. The emotional beat at 8:00 is the whole game.
### 4.2 Ongoing power user — weekly drift
- Every 30 min, background auto-sync polls each registered source for new content
- When new content arrives, it's cognified silently
- A "memory grew" badge appears on the MemoryApp with a count: "+17 frames this week"
- Click → see the new frames with source attribution + any conflicts flagged (write-path contradiction detection from Gap K fires here)
- One-click "dismiss" marks them normal; "promote" escalates importance
### 4.3 Team knowledge worker — shared mind
- User invites 4 colleagues to the workspace
- Each colleague harvests their own ChatGPT/Claude
- Team-scoped frames merge (with per-frame attribution — "Alice imported this from her ChatGPT on 2026-04-10")
- Shared KG: entities mentioned by 2+ team members light up as "canonical"
- Conflict detection (Gap K): if Alice's harvest says "we chose Postgres" and Bob's says "we chose SQLite," a flag surfaces on the next team-mind search
---
## 5. Privacy surface — the explanations users need
Every harvest action needs an accompanying explanation, not buried in a TOS. These should be inline UI copy:
**During import:**
> "This file is being read on your machine. The conversations are parsed into memory frames in your local `.waggle` folder. No data is uploaded to Waggle, Anthropic, OpenAI, or any third party. (Except the LLM you configured for distillation — by default the cognify step uses your local LLM if one is configured, else falls back to your configured cloud provider via LiteLLM.)"
**After import:**
> "Your memory is at `~/.waggle/minds/personal.mind`. It's a SQLite database. You can copy it, back it up, export as JSON, or delete it anytime. There's no cloud sync unless you explicitly enable a team workspace."
**For the privacy-paranoid user:**
- Settings → Privacy → "Show me exactly what leaves my machine" → opens a panel listing every network egress: LLM provider, time, token count, purpose.
- "Run entirely offline" toggle → disables all LLM providers, switches to Ollama/local-only mode.
### The trust chart
A single screen at `Settings → Privacy → How this works` with five rows:
| Data class | Stored | Left machine? |
|---|---|---|
| Raw harvest imports (conversations.json etc.) | `.waggle/imports/` | No |
| Distilled frames | `.waggle/minds/*.mind` | No |
| LLM calls for distillation + chat | routed via LiteLLM to your configured provider | Yes — to the LLM provider you chose |
| Telemetry / analytics | disabled by default | No unless opted in |
| Team sync (only if team workspace enabled) | team-scoped frames only | Yes — to the team server you configured |
Crisp, honest, shows both what happens on-device and what leaves. This single screen is worth its weight in trust.
---
## 6. Action plan — phased
### Phase A (1 week) — ship the first-run hook
- Harvest-first onboarding wizard (replaces step 2 of current wizard)
- Live-view streaming frame counter (wire `harvest/pipeline.ts` progress events to SSE → React)
- Post-harvest identity-auto-populate screen
- Dedup summary message ("Found N new, M duplicates removed")
- Privacy headline above every harvest button
**Dependencies:** Fastify SSE endpoint + React component. No new backend logic — pipeline already tracks counts.
### Phase B (1 week) — ship the reliability layer
- Resumable harvests (checkpoint every 100 frames, retry from last checkpoint on failure)
- Partial-failure UI ("imported 847/1,200 then failed. Resume? Skip the problematic item? Cancel?")
- Conflict-flag surface in live view (Gap K's write-path contradiction detection gets rendered inline)
- "Memory grew" badge on MemoryApp with weekly count
### Phase C (2 weeks) — ship the missing adapters
- **Cursor adapter** (0.5-1 day) — top ask from Developer persona
- **Notion adapter** (1 day) — serves Knowledge Worker + Researcher + Prosumer
- **Obsidian adapter** (0.5 day, trivial — vault is already markdown)
- **Grok adapter** (0.5 day) — XAI users are small but vocal
### Phase D (2-3 weeks) — ship the auto-sync advanced features
- Per-source auto-sync cadence controls (hourly / daily / weekly / manual)
- Smart conflict merge UI (Gap K flags → user reviews → accept/reject/merge)
- Identity drift detection ("You mentioned 'Egzakta' 40 times 3 months ago, 5 times last month — still active?")
- Harvest health report (a weekly email / dashboard: "Your memory grew 247 frames this week, 89 new entities, 34 new relations.")
### Phase E (ongoing) — harder data shapes
- Voice/video transcripts (Granola, Fathom, Fireflies) — different frame-per-turn mechanics
- Email / calendar — privacy surface is heavier, needs careful onboarding
- Slack / Teams chat history — per-channel selection
- Enterprise knowledge bases — Confluence, SharePoint, Google Drive (MCP connectors already cover this for query; harvest = bulk index)
### Phase F (Q3+) — export / portability
- "Export everything" button (JSON + raw source files)
- Import from another Waggle instance (user moves laptop)
- Backup to user-chosen cloud (Dropbox / Drive / S3) — user manages credentials
- Sync between two Waggle installs on different machines (same user) — harder, needs conflict protocol
---
## 7. Metrics that should drive roadmap priority
- **Activation rate** — % of new users who harvest ≥1 source in the first session (target ≥60%)
- **Retention differential** — 30-day retention for users with harvest vs without (hypothesis: 2-3x)
- **Frames per user** — median + p95 (track growth month-over-month)
- **Time-to-first-insight** — seconds from install to first relevant memory-backed answer (target ≤5 min)
- **Source diversity** — median number of distinct sources per active user (target 3+; correlates strongly with retention)
These are instrumented in `packages/core/src/mind/telemetry.ts`. The dashboard for these metrics is itself a P1 backlog item.
---
## 8. Competitive positioning of harvest specifically
| Competitor | Harvest story |
|---|---|
| ChatGPT | Can import custom instructions / files for grounding, but no cross-tool harvest. Memory stays inside ChatGPT. |
| Claude Projects | Project-scoped knowledge bases; no cross-app harvest. |
| Notion AI | Can ingest pages you already have in Notion. Doesn't ingest external AI conversations. |
| mem0 | API-first. Developers integrate into their app. No end-user harvest UI. |
| Letta | Agents-first. Harvest not the pitch. |
| Rewind AI | Records screen activity. Different data model (frames from OS activity) — interesting adjacency but different user. |
| **Waggle** | **Full cross-tool harvest from 11 sources (Cursor next), local-first, compliance-ready, auto-linking, provenance-aware.** |
**Positioning statement:** "Waggle is the only AI workspace that harvests from every AI you already use, stores it on your machine, and makes it searchable across all of them. Your AI life, unified."
---
## 9. Open questions / decisions for Marko
1. **Harvest-first onboarding vs current tier-first onboarding** — is Phase A a v1 reshuffle (replace current step 2) or a parallel opt-in ("new user? try harvest-first")? Risk of confusion for existing users upgrading.
2. **Anonymous telemetry for activation metrics** — required to drive section 7, but privacy-paranoid users will revolt if on by default. Recommend: off by default, opt-in with "help us make Waggle better" nudge *after* the hook.
3. **Email / calendar as a Pro-tier lock** — they require OAuth and heavier compliance surface. Recommend: Free tier gets 11 current adapters; Pro adds email+calendar+voice-transcripts.
4. **Waggle-to-Waggle sync for user portability** — Phase F is real work. Prioritize against Cursor+Notion adapters (Phase C).
5. **Telemetry exposure** — "Here's what the LLM saw about you when answering" is an explainability feature that 3/7 personas want. Is that a P1 UX item now or a v2?
If you give me directional takes on those 5, Phase A can start Monday.
---
## Closing
Memory Harvest is the wedge. Everything else Waggle does (skills, evolution, compliance, agents, wiki) is *better* because of the harvest corpus, but the harvest itself is the first thing a new user should feel delight from — ideally before the 8-minute mark of their first session. Everything in this plan is in service of shortening that time-to-delight and then compounding the value monthly so churn is structurally low.
The technical pipeline is already strong (9 adapters, dedup, provenance, auto-sync). The gap is entirely UX + 3-4 missing adapters. Both are 1-3 week efforts. Phase A alone would move the "new user hooked in first session" needle measurably.

328
docs/research/03-paper-skeleton-v2-2026-04-30.md Normal file
View File

@@ -0,0 +1,328 @@
# Arxiv Paper Skeleton v2 — Pre-Launch Submission Ready
**Date:** 2026-04-30
**Author:** PM
**Status:** SKELETON DRAFT — awaiting Marko 7 decision points ratifikacija → drafting full content
**Predecessor:** `00-paper-outline.md` (2026-04-26) + `02-section-5-refresh-2026-04-30.md`
**Target venue:** arxiv preprint (cs.AI primary, cs.CL secondary), Day 0 launch submission
**Authority:** `decisions/2026-04-30-pre-launch-sprint-consolidation-LOCKED.md`
**Wall-clock:** 7-9 dana drafting (PM) + 1-2 dana review (Marko) = ~10 dana to submission
---
## §1 — Marko 7 decision points — RATIFIED 2026-05-02
**1. Title — RATIFIED:** "Apples-to-Apples on LoCoMo: A Bitemporal Local-First Memory Substrate and a +27.35-Point Methodology Gap."
Marko rationale: methodology-led, dvodelno, konkretan broj kao hook. Najjači za arxiv discoverability jer "methodology gap" privlači citate iz svake naredne LoCoMo evaluacije. Marko REJECTED PM rec (d) jer cross-family generalization je sekundarna narativna linija, ne primary anchor; primary contribution je substrate vs Mem0 + +27.35pp self-judge bias methodology gap.
**Abstract refinement (Marko ratified):** prva rečenica metodološka, druga arhitekturna, treća broj. Reorder iz prethodne 4-claim strukture da odražava ovaj sequence.
**2. Co-author roster — RATIFIED 2026-05-02 final:**
- Lead/corresponding: Marko Marković, CEO Egzakta Group ✅
- Co-author 1 (Egzakta technical lead): **Michail (Mikhail) Pavlukhin** ✅
- LinkedIn (Marko-confirmed 2026-05-02): https://www.linkedin.com/in/mpavlukhin/ (rs.linkedin = Serbia)
- **Profile verified via WebSearch 2026-05-02:** Public output = DSPy + GEPA practitioner tutorials ("Improving performance of small LLMs on creative tasks with DSPy and GEPA"), Timeboat Adventures experimental narrative game, AI engineering content creation.
- **EVOLVESCHEMA author — VERIFIED per `memory/project_benchmark_strategy.md` LOCKED 2026-04-20:** Pavlukhin je first-author EVOLVESCHEMA paper "LLM-Guided Evolutionary Optimization of DSPY Signature Schemas" (Marko uploadovao PDF 2026-04-20). Paper demonstrates composition pipeline EVOLVESCHEMA→GEPA, najbolji javno objavljeni brojevi 0.925 FIRE NER, 0.903 SGD Hotels (per Pavlukhin Table 2). Teza: "schema structure matters more than instructions". 74% HotPotQA gain od jedne strukturne mutacije (replace_output_fields).
- **Strategic alignment:** Pavlukhin radi tačno u methodology orbiti našeg §5.4 GEPA finding + §2.4 Related Work EVOLVESCHEMA reference. Coauthorship strengthens cross-citation network and signals Egzakta team deep methodological investment u trojnoj kompoziciji optimization layer-a.
- **Arxiv presence:** EVOLVESCHEMA paper trenutno NIJE NA ARXIV-U (PDF was internal Egzakta material per Marko 2026-04-20 upload). WebSearch arxiv author profile 404 + zero hits potvrđuje no prior arxiv first-author submission **yet**. → cannot currently endorse, ALI strategic option below.
- **STRATEGIC OPTION (Pavlukhin EVOLVESCHEMA arxiv submission):** Pavlukhin submit EVOLVESCHEMA kao standalone short preprint (4-6 strana, cs.AI primary) u sledećih 3-5 dana. Three benefits: (1) Pavlukhin postaje arxiv author → Scenario A re-activated (auto-endorsement kroz coauthorship); (2) Cross-citation network: EVOLVESCHEMA u §2.4 + §5.4 composition pipeline reference; (3) Egzakta team javan akademski signal pre Day 0 launch. Marko 2026-05-02 to ask Pavlukhin if feasible.
- Co-author 2 (methodology consult): **DROPPED** per Marko ratification 2026-05-02 (Opcija C). Paper ide kao 3-author. Barać preuzima methodology cross-check rolu pored academic advisor.
- Co-author 3 (academic advisor): **Dušan Barać** ✅
- Scholar profile (primary, Marko-confirmed 2026-05-02): https://scholar.google.com/citations?user=WvzsdGkAAAAJ&hl=sr&oi=ao
- Scholar profile (alternative ID seen earlier, possibly stale): https://scholar.google.com/citations?user=wn7N_HsAAAAJ&hl=sr&oi=sra — confirm sa Markom da li je isti čovek dva profila ili greška
- **Profile verified via web search 2026-05-02:** Full Professor, Department of E-business, Faculty of Organizational Sciences (FON), University of Belgrade. 56 publikacija ukupno; 25+ u peer-reviewed international journals sa impact factor; 8 papers u SCI/SCIe journals. 18+ godina IT consulting experience. Primary research domains: digital transformation, e-commerce, IT project management, e-learning technologies, AI-based apps, blockchain.
- **Academic credibility:** STRONG — Full Professor + University of Belgrade (recognized institution) + SCI track record. Strong co-authorship credibility signal for peer reviewers.
- **Methodology cross-check capacity:** ADEQUATE — IT/e-business background covers general empirical research methodology rigor; možda not specifično cs.AI/judge ensemble expertise, ali general academic peer-review standards covered.
- **Arxiv endorsement potential:** ❌ WEAK — primary domain je e-business / IT ecosystems, NE cs.AI/cs.CL/cs.LG; plus Marko confirmed "nema na arxiv" → no prior arxiv first-author submission → cannot endorse cs.AI primary submission. Domain mismatch + zero arxiv presence = double pre-empt.
**3. Endorsement path — RESOLVED 2026-05-02 final via Path A + Path D combination.**
Verified results:
- Pavlukhin: NEMA arxiv first-author submission **yet** — but EVOLVESCHEMA standalone arxiv submission planned (Marko-side action sutra 2026-05-03 ponedeljak) → Path A reactivated
- Barać: NEMA arxiv first-author submission + e-business domain mismatch → cannot endorse
- LinkedIn 1st-degree network sweep (Bojan Djuric verified 2026-05-02): no arxiv-eligible candidate match without cold outreach
- Marko 2026-05-02 verdict: cold outreach is out of scope for current sprint
**Path A — PRIMARY (Marko-side action sutra):** Pavlukhin podnosi EVOLVESCHEMA na arxiv kao standalone short preprint (4-6 strana, cs.AI primary). Posle arxiv approval (~24h), Pavlukhin postaje cs.AI/cs.LG eligible endorser. **Critical insight:** Pavlukhin NE MORA biti coauthor našeg paper-a — endorses kao nezavisni researcher kroz EVOLVESCHEMA author credentials. Single email request to direct contact, NE cold outreach. Timeline: 3-5 dana Pavlukhin draft + 24h approval = ~5-6 dana to endorsement-ready. Compatible sa Day 0 ETA 6-10 dana.
**Path D — FALLBACK + DECOUPLING:** Day 0 launch ne treba biti gated na arxiv preprint timing. Landing trust signal "Published methodology — arxiv preprint" (Trust Band Card 4 placeholder + Sources reference) zamenjuje se sa "Open methodology — github.com/marolinik/waggle/docs/methodology" (markdown doc u OSS repo). Credibility signal ostaje (open methodology = open source rigor), arxiv timing više nije launch dependency. Posle arxiv preprint linkujemo retroactively kao news cycle update.
**Landing copy implications (apply paralelno sa Track D apps/www port):**
- Trust Band Card 4 swap: "Published methodology — arxiv preprint" → "Open methodology — github docs"
- Footer Research column: "arxiv preprint" link → defer to placeholder ili remove until arxiv preprint dođe
- Hero Variant D Sasha (developer): no copy change needed (no arxiv mention)
- Proof Card 1 GEPA: keep "Methodology in arxiv preprint" — to je forward reference, cleanly fulfilled when Path A succeeds OR replace sa "Methodology in companion docs" if delayed
**PM action queue (sutra 2026-05-03):**
- Marko 1-line Pavlukhin message za EVOLVESCHEMA arxiv submission timing
- PM update Track D apps/www CC sesija D §3 acceptance review sa Trust Band Card 4 copy swap
- PM draft methodology markdown doc skeleton za github (post Day 0 link target)
**3. Endorsement path — DEFERRED:** Marko asked "kako da proverim ko od mojih kontakata ima prior arxiv publications" — PM follow-up sa 3 actionable proverava metoda (vidi §1bis below).
**4. Multiplier section disposition — RATIFIED with sharper framing:**
**NOT** "Branch B prerequisites for re-test (deferred)" — that reads evasive after Zep/Mem0 dispute reviewer-instinct.
**INSTEAD:** Eksplicitna "Negative Result" subsekcija u §6 Discussion sa 5 elementa:
1. Hypothesis statement
2. N=12 protocol
3. Numbers verbatim: h2=1/3, h3=0/3, h4=0/3
4. Preconditions for re-test imenovane: B-frame compaction stability, harvest timestamp fix, minimum N
5. Qualification: "ne aplicira na primary contribution (substrate-vs-Mem0 + methodology gap)"
"Deferred" ide u footnote, NE u headline. Negative result se hvata u CV publikacije jer je honest, ne uprkos tome.
**5. GEPA scope u §5.4 — RATIFIED (a) sa companion paper signal:**
Zadržava ~1.5 page §5.4 sub-section. PLUS footnote: *"Extended cross-family treatment in companion paper, in preparation."*
Marko rationale: standalone §5 GEPA bi te koštao desk-reject rizika ("this is two papers stitched together") što ne želiš na prvoj submisiji. Cross-family generalization sa GEPA Faza 1 je dovoljno jako da nosi sopstveni arxiv preprint za 6-8 nedelja sa ERL methodology framing. Footnote daje opciju za drugi launch ciklus bez current commitment. Ako kapaciteta za companion paper nema, footnote se izostavi pre submission.
**6. §5.5 framing — RATIFIED with active-voice rewrite:**
NOT "methodology maturity demonstration" — chita se defanzivno, apologetičko.
INSTEAD: **"Bias-detection guardrails functioning as designed"** — aktivan claim sa proverim outcome:
- "GPT selection bias **detected and filtered** by held-out validation at N=..."
- "Qwen-non-thinking decoupling probe **revealed** effect Z magnitude"
- "Calibration **evolved** across Amendments 7-11 with adjustments documented in Appendix"
Aktivni glagoli protiv pasivnih: "guard-rail caught and filtered" vs "bilo je pažljivo." Druga formulacija nosi metodološku težinu, prva zvuči apologetski.
**7. Phase 5 forward reference — RATIFIED with two terminological refinements:**
REVISED forward reference statement:
*"Production traffic Pass II rates, p95 latency, and recall@K on production traffic distribution from Phase 5 deployment will be reported in v2 of this preprint, scheduled within 60 days post-publication."*
Refinement 1: "expected ~6 weeks" → "scheduled within 60 days" (operational discipline signaling + 14-day margin if Phase 5 slips two weeks; missed v2 datum bi bio credibility liability za naredni paper).
Refinement 2: imenovati konkretne metrike (Pass II rates + p95 latency + recall@K) — vague forward reference izgleda kao vaporware; specifična obećana metrika izgleda kao discipline.
---
**Marko net evaluation note (2026-05-02):** PM intencije su sve ispravne; tri od četiri framing-a (4, 6, 7) treba da se pomere stepenicu ka aktivnijem, ne dodatak na sadržaj. Cilj nije da paper deluje skromno ili pažljivo — cilj je da deluje **tačno**.
Posle ovih ratifikacija, drafting kick-off može da krene paralelno sa rešavanjem Co-author 2 + Endorsement path.
---
## §1bis — Endorsement path actionables (PM follow-up za Decision 3)
Marko question: "kako da proverim ko od mojih kontakata ima prior arxiv publications". 3 metoda po brzini:
**Method A — arxiv search (5 min per name):**
1. Otvori https://arxiv.org/search
2. Unesi "FirstName LastName" u Author search
3. Filter: cs.AI, cs.CL, cs.LG (relevant categories)
4. Ako se pojavi makar 1 paper kao first author → endorsement candidate. Ako appears samo kao co-author → weaker, endorsement may still work ako paper je u cs.AI specifically (arxiv requires first-author publication u target category).
**Method B — Google Scholar (3 min per name):**
1. Otvori https://scholar.google.com
2. Search "FirstName LastName" + relevant keyword (e.g., "memory" or "agent")
3. Ako ima Scholar profile → click → vidi publication list. cs.AI/cs.CL papers = endorsement potential.
4. Bonus: h-index ≥3 u relevant category = strong endorser.
**Method C — Brain dump → PM verifies:**
Marko mi da listu 10-20 imena (LinkedIn 1st-degree connections u academic + AI research). PM uradi Method A + B za svaki kroz web search. Output je rang-lista candidates sortirana po endorsement probability + first-message draft za top 3.
**PM preporuka:** Method C najbrži za Marka (sve što treba je brain-dump 20 imena), ja u 30-45 min vratim ranked list. Pavlukhin kao confirmed Co-author 1 — proveriti da li ima cs.AI prior publications (ako da, automatic auto-endorsement, NEMA need za external).
Plus za Co-author 2 methodology consult — kandidati koji dolaze iz EVOLVESCHEMA, GEPA (Agrawal), ACE (Zhang) author networks su strong (već su empirical AI rigor). Pavlukhin može introduce.
---
## §2 — Section structure (10 sekcija, ~10-13 strana)
### Abstract (≤ 250 words)
Three-claim structure (preserved iz v1):
1. Architectural separation of memory substrate from retrieval is novel and methodologically important.
2. Substrate beats peer-reviewed Mem0 on LoCoMo oracle ceiling: 74% vs 66.9%.
3. We document +27.35pp self-judge bias; we propose trio-strict ensemble as remedy.
NEW fourth claim (per §5.4 refresh):
4. Genetic-evolutionary prompt adaptation (GEPA) on Waggle harness produces +12.5pp Pass II uplift on held-out validation; method generalizes cross-family (Claude flagship + Qwen 35B open-source); on-prem Qwen achieves Opus-class quality on validated samples.
Word budget: 230-250 words.
### 1. Introduction (~1 page)
Subsections:
- 1.1 Problem framing (LLM agents lack persistent memory; products bundle layers; independent measurement impossible)
- 1.2 Why substrate-retrieval separation matters (no one knows which layer to improve)
- 1.3 Contributions (4 — architectural + empirical + methodological + GEPA cross-family + open)
- 1.4 What this paper is not (no system-level competitive claim, V1 retrieval acknowledged)
### 2. Related Work (~1 page)
Subsections (preserved iz v1):
- 2.1 Memory systems for LLMs (Mem0, MemGPT, Letta, LongMem, GraphRAG)
- 2.2 Long-term conversational memory benchmarks (LoCoMo, LongMemEval; Gaia2 + τ³ za future work mention)
- 2.3 Methodology critiques in LLM evaluation (self-judging bias, judge ensembles, reproducibility)
- 2.4 NEW: Genetic-evolutionary prompt adaptation (Agrawal et al. 2025 GEPA, Pavlukhin et al. 2025 EvolveSchema)
- 2.5 NEW: Experiential reflective learning (ERL — arxiv:2603.24639; Waggle bitemporal-KG-conditioned extension framing)
- 2.6 NEW: Closed-learning-loop agents (Hermes Agent positioning differentiation)
### 3. Architecture (~2 pages)
Subsections (preserved iz v1):
- 3.1 Substrate: bitemporal knowledge graph
- 3.2 Memory compression: I/P/B frame analogy (MPEG-4 inspired)
- 3.3 Retrieval interface (intentionally pluggable)
- 3.4 Audit + governance layer (EU AI Act compliance, sovereign deployment)
### 4. Methodology (~1 page)
Subsections (preserved iz v1):
- 4.1 LoCoMo-1540 evaluation protocol (5 cells, 5 question types)
- 4.2 Trio-strict judge ensemble (Opus + GPT + MiniMax, κ_trio = 0.79)
- 4.3 Apples-to-apples re-judging (GPT-4o-mini self-judge methodology re-eval)
- 4.4 Pre-registration & reproducibility (manifest v6 frozen, git SHA, seeds locked)
- 4.5 NEW: GEPA evolution methodology (manifest v7 + 11 amendments, held-out validation, §F gates)
### 5. Experiments & Results (~4-5 pages)
#### 5.1 Substrate quality (oracle ceiling)
- Claim: 74% trio-strict (N=400)
- Compared: Mem0 peer-reviewed 66.9% basic / 68.4% graph
- Headline: +7.1pp over peer-reviewed baseline
- Statistical test: Fisher one-sided p < 8.07e-18 vs no-context (H1 PASS)
#### 5.2 Methodology bias quantification
- Self-judge: aggregate +X.X% inflation
- Trio-strict: true performance band
- Measured gap: **+27.35pp**
- Implication: published "91.6% LoCoMo SOTA" claims likely 18-30pp inflated
#### 5.3 V1 retrieval performance
- Retrieval cell: 48% trio-strict
- Honest: 26pp below substrate ceiling
- Direction: 5 specific improvements identified (community contribution invitation)
#### 5.4 GEPA Evolution + Cross-Family Generalization (NEW, ~1.5 page)
Subsections:
- 5.4.1 GEPA evolution methodology summary (~0.3 page)
- 5.4.2 Cross-family generalization results (~0.5 page) — table:
- claude::gen1-v1: +12.5pp in-sample / +12.5pp held-out / 0pp gap
- qwen-thinking::gen1-v1: +12.5pp / +12.5pp / 0pp
- gpt::gen1-v2: +25.0pp / +5.0pp / 20pp gap (FAIL §F.5 cond_2)
- §F.1 PASS 3/3, §F.5 cond_2 PASS 2/3
- 5.4.3 Qwen 35B = Opus-class on out-of-distribution (~0.4 page)
- Retrieval engagement 2.231 = 96% Opus parity
- Pass II 100% on N=13 (8 in-sample + 5 held-out)
- Sovereign deployment without flagship-tier compromise
- 5.4.4 Phase 5 deployment status (~0.2 page) — both AUTHORIZED variants
- 5.4.5 Pilot 2026-04-26 disposition (~0.1 page) — multiplier conditional finding
#### 5.5 Scoping Discussion (NEW, ~1.2 page)
4 methodological findings:
- 5.5.1 gpt selection bias exposed by held-out methodology (~0.3 page)
- 5.5.2 qwen-non-thinking retrieval-quality decoupling (~0.3 page)
- 5.5.3 generic-simple necessary-but-not-sufficient retrieval (~0.2 page)
- 5.5.4 Calibration evolution Amendments 7-11 (~0.4 page)
#### 5.6 Phase 5 Production Validation Hook (NEW, ~0.2 page)
Forward reference za v2 preprint sa production traffic results.
### 6. Discussion (~1-2 pages)
Subsections:
- 6.1 Substrate-retrieval separation as paper contribution
- 6.2 Sovereignty axis (local-first + Apache-2.0; KVARK secondary downstream)
- 6.3 Cross-family generalization implications (NEW per §5.4)
- 6.4 Limitations (single benchmark + retrieval gap + multiplier conditional)
### 7. Future Work (~0.5 page)
- Retrieval V2 (entity-aware reranking, hybrid scoring)
- Cross-dataset replication (LongMemEval + Gaia2 + τ³-bench banking_knowledge — per benchmark portfolio Phase 3 + Phase 4 sprints)
- Faza 2 sprint (gpt N=16 re-validation + scoping investigations)
- Multiplier thesis at scale (Branch B prerequisites)
- ERL methodology extension (per arxiv:2603.24639) submission MemAgents Workshop
### 8. Conclusion (~0.25 page)
Restate 4 contributions. Restate Apache-2.0 community invitation.
### 9. Acknowledgments
- Anthropic (Claude AI assistance, explicit per ICML/NeurIPS/arxiv standards — Claude does NOT appear as author)
- LiteLLM gateway za multi-vendor judge orchestration
- LoCoMo dataset authors
### 10. References (~30-50 BibTeX entries)
Categories:
- Memory systems (Mem0, MemGPT, Letta, LongMem, GraphRAG): 8-10
- Conversational memory benchmarks (LoCoMo, LongMemEval): 5-7
- LLM evaluation methodology (judge bias, ensembles): 6-10
- Genetic-evolutionary prompt adaptation (GEPA, EvolveSchema): 3-4
- Experiential reflective learning (ERL): 2-3
- Bitemporal databases + KGs: 4-6
- MPEG-4 video compression analogy: 2-3
- Reproducibility & pre-registration: 3-5
---
## §3 — Claim trace (where each numerical claim comes from)
Preserved iz v1 sa NEW additions:
| Claim | Source artifact | Verification path |
|---|---|---|
| 74% oracle ceiling | `benchmarks/results/v6-trio-strict/oracle-context-*.summary.json` | Stage 3 v6 N=400 |
| 66.9% Mem0 baseline | Mem0 paper Table 2 | arxiv:2504.19413 |
| +7.1pp gap | direct subtraction | verified |
| Fisher p < 8.07e-18 | Stage 3 v6 H1 | `benchmarks/results/v6-trio-strict/h1-fisher-result.json` |
| +27.35pp self-judge bias | Apples-to-apples re-eval | `benchmarks/results/v6-self-judge-rebench/` |
| κ_trio = 0.79 | Stage 3 v6 calibration | `benchmarks/calibration/2026-04-24-trio-strict-recal.json` |
| +12.5pp held-out (claude::gen1-v1) | Faza 1 §F.1 PASS | `decisions/2026-04-29-gepa-faza1-results.md` |
| +12.5pp held-out (qwen-thinking::gen1-v1) | Faza 1 §F.1 PASS | same |
| 0pp held-out gap (claude + qwen-thinking) | §F.5 cond_2 PASS | same |
| 20pp gap (gpt::gen1-v2 FAIL §F.5 cond_2) | held-out vs in-sample | same |
| 96% retrieval parity (qwen-thinking) | retrieval 2.231 / Opus 2.33 | Faza 1 evidence |
| 100% Pass II combined N=13 | Faza 1 Checkpoint C | same |
| 11 amendments manifest v7 | Faza 1 manifest chain | `decisions/2026-04-29-gepa-faza1-results.md` §11-amendment chain |
Every numerical claim mora trace to JSON artifact + git SHA + manifest anchor. No hand-computed numbers u paper text.
---
## §4 — Authoring sequence
**Day 0 (today, 2026-04-30):** This skeleton authored. Pending Marko 7 decision points ratifikacija.
**Day 1 (post-ratifikacija):** PM autoring abstract + introduction (~1500 words).
**Day 2:** PM autoring related work + architecture (~3000 words).
**Day 3:** PM autoring methodology (~1500 words).
**Day 4-5:** PM autoring experiments & results — heavy lifting (~5000 words sa §5.4 + §5.5 + §5.6 detail).
**Day 6:** PM autoring discussion + future work + conclusion (~2000 words).
**Day 7:** PM autoring acknowledgments + references (BibTeX entries).
**Day 8-9:** Internal review (Marko + co-authors) + revisions.
**Day 10:** LaTeX formatting + final polish + arxiv submission.
ETA submit: Day 10 wall-clock from ratifikacija. Posle submission, preprint live within 1-2 days.
---
## §5 — Cross-references
- v1 outline: `00-paper-outline.md`
- §5 refresh: `02-section-5-refresh-2026-04-30.md`
- Faza 1 closure: `decisions/2026-04-29-gepa-faza1-results.md`
- Pre-launch sprint consolidation: `decisions/2026-04-30-pre-launch-sprint-consolidation-LOCKED.md`
- Benchmark portfolio refresh: `briefs/2026-04-29-benchmark-portfolio-refresh-2026-venues.md`
- This skeleton: `research/2026-04-26-arxiv-paper/03-paper-skeleton-v2-2026-04-30.md`
---
**End of skeleton. Awaiting Marko 7 decision points ratifikacija → drafting kick-off.**

198
docs/research/04-competitive-landscape.md Normal file
View File

@@ -0,0 +1,198 @@
# Competitive Landscape: hive-mind vs. the 2025-2026 AI Memory Market
**Date:** 2026-04-22 (initial); updated 2026-05-02 (Hermes Agent integration per `strategy/competitive/2026-04-30-hermes-agent-intel-update.md`)
**Scope:** Research-only competitive positioning for hive-mind (local-first AI memory, MCP server, SQLite, FTS5+vector hybrid, knowledge graph, I/P/B frames, wiki compiler).
**Naming disambiguation (added 2026-05-02):**
- "**Hermes**" in this document, when referenced as MCP client (Strengths §1.4, Opportunities, Tagline #3), refers to the Nous Research Hermes AI coding client — **consumer** of MCP servers, friendly to hive-mind (positive distribution channel).
- "**Hermes Agent**" (added below as §1.11) refers to the Nous Research **closed learning loop agent product** launched 25 February 2026 — direct competitor in the OSS knowledge worker agent space.
---
## 1. Competitor Profiles
### 1.1 mem0 (mem0.ai)
- **Architecture:** LLM-driven extract/consolidate/retrieve layer on top of pluggable stores (vector + optional graph + relational). Requires an LLM (default GPT-4.1-nano) and embedding model. Graph memory is a paid variant.
- **Benchmarks:** 91.6 on LoCoMo, 93.4 on LongMemEval, 64.1/48.6 on BEAM 1M/10M, ~7k tokens per retrieval (token-efficient algorithm, 2025). [mem0.ai/research]
- **Local-first?** Self-hostable (Apache-2.0), but fundamentally **cloud-shaped**: still calls OpenAI/embedding APIs by default; best features (graph memory) are managed-only.
- **MCP:** Yes, first-party `mem0` MCP server + OpenMemory MCP (local).
- **Pricing:** Hobby free (10k memories), Starter $19, Pro $249 (graph + analytics), Enterprise custom. [mem0.ai/pricing]
- **License:** Apache-2.0 (core) / proprietary (cloud).
- **Differentiator:** Market leader on benchmark headlines; 41k+ GitHub stars; $24M Series A (Oct 2025).
### 1.2 Letta (formerly MemGPT)
- **Architecture:** Block-based context memory. Three tiers: **Core Memory** (in-context RAM block the agent edits), **Recall Memory** (searchable conversation log), **Archival Memory** (tool-queried cold storage). Self-editing is the signature feature.
- **Benchmarks:** No LOCOMO headline; legacy MemGPT paper established the "LLM as OS" metaphor.
- **Local-first?** Self-hosted server runs locally; needs external LLM for agent reasoning. ADE connects to local or cloud.
- **MCP:** Yes — Letta consumes MCP servers as tool sources; third-party Letta-MCP-server exists.
- **Pricing:** Self-hosted free; cloud $20–200/mo. [letta.com]
- **License:** Apache-2.0.
- **Differentiator:** Stateful, self-editing agents — agent *writes its own memory blocks*.
### 1.3 Zep / Graphiti (getzep.com)
- **Architecture:** Graphiti temporal knowledge graph (Apache-2.0) backing Zep Cloud. Bi-temporal edges (valid-from / valid-to). Typically requires Neo4j/FalkorDB/Kuzu. Zep arXiv 2501.13956 is the paper.
- **Benchmarks:** Zep claimed 84% LoCoMo → mem0 re-evaluated as 58.44% → Zep counter-claimed 75.14% → Dec 2025 new claim of 80% at <200ms P95. Graphiti claims 94.8% on DMR and +18.5% on LongMemEval. Treat with salt — there is an open public benchmark dispute. [GitHub issue getzep/zep-papers#5]
- **Local-first?** Graphiti OSS is self-hostable but heavyweight (Neo4j). Zep Cloud is cloud-only; **Zep Community Edition was deprecated April 2025**.
- **MCP:** Yes — Graphiti MCP Server v1.0 shipped Nov 2025, 20k+ GitHub stars. [blog.getzep.com]
- **Pricing:** Free 1k credits/mo (prototype only), Flex $25/mo, custom enterprise.
- **License:** Apache-2.0 (Graphiti) / proprietary (Zep Cloud).
- **Differentiator:** Temporal knowledge graph done seriously; best-in-class MCP adoption.
### 1.4 Cognee (cognee.ai)
- **Architecture:** ECL pipeline (Extract → Cognify → Load). DataPoints = strongly-typed Pydantic objects acting as both node and edge schemas. Unifies relational + vector + graph. 38+ connectors. $7.5M seed.
- **Benchmarks:** HotPotQA: **0.93 human-like correctness** (Cognee 2025.1), beating LightRAG, Graphiti, mem0. Multi-hop strength.
- **Local-first?** Yes — self-hostable with local LLMs via Ollama. **MCP:** Yes, first-party. **Pricing:** OSS free; Core/Enterprise = contact sales. **License:** Apache-2.0.
- **Differentiator:** Strongest multi-hop graph reasoning; schema-typed knowledge.
### 1.5 Supermemory (supermemory.com)
- **Architecture:** Cloudflare Durable Objects, per-user MCP via URL-path isolation, SSE real-time. "Human-like decay" model. MemoryBench OSS benchmark suite.
- **Benchmarks:** 81.6% LongMemEval (GPT-4o); strong LoCoMo / ConvoMem. $3M Oct 2025.
- **Local-first?** No — **cloud-first SaaS**. MCP server OSS, engine cloud-resident. **MCP:** Yes, one of the most widely integrated (Claude Desktop, Cursor, Windsurf, VS Code, Claude Code). **Pricing:** Dev: Free / Pro $19 / Scale $399. Consumer: Free / $9. **License:** OSS client / proprietary engine.
- **Differentiator:** Best distribution across AI clients; consumer "second brain" app.
### 1.6 LlamaIndex Memory Modules
- **Architecture:** Library abstractions — `ChatMemoryBuffer` (FIFO), `ChatSummaryMemoryBuffer` (periodic summarization), `SimpleComposableMemory` (deprecated), and the newer unified `Memory` class with short-term FIFO + optional long-term extraction.
- **Benchmarks:** None — this is plumbing, not a memory system.
- **Local-first?** Runs where you host it; no storage opinion. **MCP:** No first-party memory server. **Pricing:** Free OSS. **License:** MIT.
- **Differentiator:** Embedded in the most popular RAG framework; low-ceremony for existing LlamaIndex users.
### 1.7 LangMem (LangChain)
- **Architecture:** SDK over LangGraph's `BaseStore` + Checkpointers. Semantic / episodic / procedural memory types. Background manager extracts and consolidates asynchronously.
- **Benchmarks:** No public LOCOMO numbers. **Local-first?** Library-level. **MCP:** Indirect only. **Pricing:** LangSmith tiers for managed observability. **License:** MIT.
- **Differentiator:** Tight LangGraph integration; opinionated episodic/procedural/semantic split.
### 1.8 OpenAI ChatGPT Memory
- **Architecture:** Proprietary; "saved memories" (facts) + "reference chat history" (retrieval over past conversations). Rolled to free tier June 2025.
- **Benchmarks:** None published. **Local-first?** No. **MCP:** No — and critically **not exposed via API**. [memobase.io blog] **Pricing:** Bundled. **License:** Proprietary.
- **Differentiator:** Default memory for ~800M consumer users; zero-setup.
### 1.9 Anthropic Claude Memory + Memory Tool
- **Architecture:** Two products. (a) **Consumer Claude Memory** — GA Team/Enterprise Sept 2025, Pro/Max Oct 23 2025, free tier Mar 2 2026. Project-scoped, work-pattern-focused. (b) **Memory Tool** (API, beta header `context-management-2025-06-27`) — client-side file directory the model CRUDs. Combined with context editing: **+39% vs baseline**.
- **Local-first?** Memory Tool is **client-side by design** — Anthropic gives the protocol, you host the files. Friendly to local storage. **MCP:** Anthropic authored MCP; Memory Tool is complementary. **Pricing:** Consumer bundled; Memory Tool = API tokens. **License:** Proprietary / open spec.
- **Differentiator:** The memory protocol standard-setter.
### 1.10 Basic Memory (basicmachines-co)
- **Architecture:** Persistent semantic graph stored as **plain Markdown files**, indexed in a local SQLite. Obsidian-compatible. MCP-native.
- **Benchmarks:** None. **Local-first?** **100% yes** — closest philosophical sibling to hive-mind. **MCP:** Yes, first-party. **Pricing:** Free OSS. **License:** AGPL-3.0 (more restrictive than hive-mind's Apache-2.0).
- **Differentiator:** Obsidian interoperability; markdown-as-source-of-truth. Lowest lock-in competitor.
### 1.11 Hermes Agent (Nous Research) — added 2026-05-02
- **Architecture:** Closed learning loop, prompt-augmented agentic workflows. Components: `MEMORY.md` + `USER.md` text files maintained per-project (prompt memory), SQLite FTS5 full-text-search storage of past sessions (episodic archive), auto-generated markdown skills capturing repeated workflows (procedural skills).
- **Launch date:** 25 February 2026.
- **Benchmarks:** Internal-only — claims **40% speedup on repeat tasks**, not peer-reviewed, no public methodology. Has not engaged any standardized public benchmark venue (LoCoMo, LongMemEval, Gaia2, τ³-bench).
- **Local-first?** Yes — files-on-disk + local SQLite. **MCP:** Self-published as agent harness (consumes MCP servers, plus emits its own memory protocol). **Pricing:** Free OSS. **License:** Apache 2.0 (permissive).
- **Adoption:** ~110K GitHub stars 10 weeks post-launch — strongest OSS adoption velocity in this category in 2026 to date.
- **Differentiator:** "Agent that gets better over time at your specific workflows" — **functionally identical positioning to Waggle's self-evolution narrative.** This is the only competitor in the local-first quadrant that explicitly markets the same architectural-philosophy story.
- **Threat level:** MEDIUM-HIGH (architectural philosophy overlap is non-trivial risk; mitigation strategies in §3 Threats below).
---
## 2. Positioning Matrix
### 2.1 Axes: Local-first ↔ Cloud-first vs. Flat (chat/vector) ↔ Graph/Structured
| | **Flat / Vector-only** | **Graph / Structured** |
| --------------------- | --------------------------------------------------- | ----------------------------------------------------------------------- |
| **Pure Local-first** | Basic Memory; Claude Memory Tool; **Hermes Agent** | **hive-mind**; Cognee (local mode); SuperLocalMemory |
| **Hybrid / Optional** | LlamaIndex; LangMem; Letta (OSS) | Cognee (hybrid); mem0 OSS (graph paid) |
| **Pure Cloud-first** | Supermemory; OpenAI Memory; LangMem Cloud | Zep Cloud; mem0 Cloud; ChatGPT Memory |
**Hermes Agent placement note (2026-05-02):** Hermes Agent occupies the **Pure Local-first + Flat/Vector** cell (alongside Basic Memory and Claude Memory Tool) because its episodic archive is flat SQLite FTS5, not a graph. It does not contest the **Pure Local-first + Graph/Structured** quadrant where hive-mind sits, but it competes for the same buyer through narrative overlap rather than architectural overlap.
### 2.2 Axes: Benchmark-chasing ↔ Opinionated epistemology
| | **Chases LOCOMO/LongMemEval** | **Has a point of view on *what memory is*** |
| -------------------------- | --------------------------------- | ------------------------------------------- |
| **Library / Framework** | LangMem, LlamaIndex | Letta (block self-edit) |
| **Graph-first system** | mem0, Zep/Graphiti | Cognee (ECL+DataPoints); **hive-mind (I/P/B)** |
| **Consumer/SaaS** | Supermemory | ChatGPT Memory, Claude Memory |
| **Markdown/file-native** | — | Basic Memory |
hive-mind sits almost alone in the **local-first + graph + opinionated epistemology** quadrant. Cognee is the only peer, but Cognee ships heavyweight pipelines and defaults to cloud LLMs; Basic Memory shares the local-first ethos but is flat markdown without bitemporal graphs or I/P/B framing.
---
## 3. SWOT — hive-mind
### Strengths
- **True single-file local-first**: SQLite workspace, sqlite-vec + FTS5 hybrid RRF, no cloud dependency, no Neo4j, no Cloudflare. Competitors either need an LLM server (mem0, Letta, Cognee) or a cloud (Zep, Supermemory, OpenAI, Anthropic consumer).
- **Opinionated epistemology (I/P/B frames)**: Intra=facts / Predicted=hypotheses / Bidirectional=corrections. No competitor has a comparable conceptual model — most collapse everything into "memories." This is a differentiating narrative.
- **Bitemporal knowledge graph**: parity with Graphiti's headline feature, but without the Neo4j dependency.
- **MCP-native with 21 tools + first-class Claude Code / Codex / Hermes support** — competitive with Supermemory (widest MCP distribution) and ahead of most (LangMem, LlamaIndex, OpenAI).
- **Harvest breadth**: 11 adapters (ChatGPT, Claude, Gemini, Perplexity exports, PDF, MD, URL) is unmatched — mem0 and Zep have nothing equivalent; Basic Memory relies on manual markdown.
- **Wiki compiler**: synthesis step nobody else offers as a first-class primitive.
- **Apache-2.0** — more permissive than Basic Memory's AGPL-3.0, matching mem0/Cognee/Letta/Graphiti.
### Weaknesses
- **No LOCOMO/LongMemEval number published.** In a market where the top-of-funnel conversation is benchmark X.Y%, this is a meaningful visibility gap. Even disputed numbers drive press (see Zep vs mem0 saga).
- **Smaller team / mindshare** vs. funded players (mem0 $24M, Zep, Supermemory $3M, Cognee $7.5M, Letta well-funded).
- **No managed cloud option** — some enterprises *want* a SaaS SKU. Currently 0% of that TAM.
- **Less mature SDK surface** compared to LangChain/LlamaIndex ecosystem gravity.
- **I/P/B metaphor needs evangelism** — video-codec framing is clever but requires education; mem0/Zep benefit from familiar vocabulary.
- **Extracted from a proprietary product (Waggle OS / Kvark)** — perception risk around "is this really open or a freemium funnel?" needs managed narrative.
### Opportunities
- **Local-first is the wedge.** The 87% consumer privacy concern (Cisco 2025) + EU AI Act + enterprise data-residency requirements = a growing slice that refuses cloud memory. Basic Memory and SuperLocalMemory prove demand; hive-mind can own the category with a richer feature set.
- **Claude Memory Tool alignment.** Anthropic's Memory Tool is client-side and storage-agnostic — hive-mind can position as **the recommended local backing store** for Claude Memory Tool. No competitor has staked that claim yet.
- **Benchmark publication.** Running MemoryBench (Supermemory's OSS suite) or LoCoMo and publishing a credible number — even if 5-10 points under mem0's headline — converts hive-mind from "interesting" to "comparable."
- **Developer "AI coding memory" niche.** Claude Code / Codex / Hermes MCP-first users skew technical and privacy-conscious. They don't want to send their codebase conversations to mem0 cloud. This is the highest-converting wedge.
- **Wiki compiler as a differentiator** — nobody synthesizes frames into interlinked pages. This is a content product hiding inside a memory product.
- **Acquihire / strategic interest.** Anthropic, GitHub, JetBrains, or Cursor could want a local memory layer. Being Apache-2.0 + single-file + MCP-native is acquirer-friendly.
### Threats
- **Anthropic or OpenAI ships a default local memory** bundled with their official client. ChatGPT/Claude desktop apps owning memory shrinks hive-mind's consumer TAM overnight.
- **mem0 or Supermemory ship a credible "local mode"**. Both have more eng resources; if mem0 bundles Ollama + sqlite-vec into a one-line install, the moat shrinks.
- **Graphiti / Zep pulls the graph-quality crown.** If Graphiti ships a lightweight embedded backend (Kuzu is already embeddable), Zep could compete in the local-first quadrant.
- **LangChain/LlamaIndex gravitate to opinionated defaults** — if either adopts mem0 or LangMem as the default and tutorials proliferate, greenfield devs never discover hive-mind.
- **Basic Memory or Cognee doubles down on local-first + graph** and outpaces hive-mind in that quadrant.
- **Benchmark-war optics.** The public Zep ↔ mem0 spat shows the category is noisy; entering without a number is risky, entering with a weak number is worse.
- **Hermes Agent (Nous Research) — added 2026-05-02.** Architectural-philosophy overlap is non-trivial: Hermes Agent markets the same "agent that gets better over time" narrative, has 110K stars 10 weeks post-launch, and ships closed learning loop with prompt memory + SQLite FTS5 episodic archive + auto-generated procedural skills. **However, six structural moats remain unaddressed by Hermes:** (1) bitemporal knowledge graph vs flat SQLite FTS5, (2) I/P/B frame model with importance weighting vs undifferentiated text, (3) MPEG-4 frame compression + wiki compiler (neither in Hermes), (4) modular Apache 2.0 npm packages vs Hermes monolithic distribution, (5) EU AI Act audit triggers built-in vs not addressed, (6) **peer-reviewed-style benchmark portfolio (LoCoMo apples-to-apples, GEPA cross-family, forthcoming Gaia2 + τ³-bench banking_knowledge) vs Hermes internal "40% speedup" only — this is the unbridgeable credibility moat for 2026.** Probability Hermes engages public benchmark venue u 2026: ~30%. Probability Hermes builds bitemporal-style memory: low (architectural rewrite). Probability Hermes targets regulated industries: low (Apache hobbyist/dev market, no compliance positioning). Mitigation per `strategy/competitive/2026-04-30-hermes-agent-intel-update.md` §3: explicit differentiator messaging Day 0, head-to-head Gaia2 framing weeks 4-8, τ³-bench banking_knowledge framing weeks 8-12.
---
## 4. Positioning Taglines (5 options)
1. **"Your AI's memory. One file. Zero cloud."**
Minimalist, defensible, true. Owns the local-first quadrant in one line.
2. **"I, P, B — the only memory system that knows the difference between a fact, a guess, and a correction."**
Leads with the epistemology moat. Honest about what the I/P/B model buys you.
3. **"The memory layer for Claude Code, Codex, and Hermes — not for someone else's cloud."**
Targets the developer-MCP wedge directly; pits hive-mind against mem0-cloud and Supermemory-cloud.
4. **"SQLite in. Wiki out. Apache-2.0 all the way through."**
Honest-engineering vibe; contrasts with AGPL (Basic Memory) and proprietary cloud (Zep, OpenAI).
5. **"Graphiti-grade knowledge graph. Without the Neo4j. Without the bill."**
Directly attacks the graph leader's weakness (operational heaviness, cloud pricing).
---
## 5. Bottom Line
hive-mind's defensible position in the 2026 market is **local-first + graph + opinionated epistemology + MCP-native + peer-reviewed-style benchmark portfolio** — a quadrant currently contested only by Cognee (heavier, cloud-LLM-leaning) and Basic Memory (flat, AGPL, no graph). The market's benchmark arms race is noisy and partially discredited (Zep/mem0 dispute), but visibility still requires *a* number. Shipping a published LoCoMo/LongMemEval figure, aligning narratively with Anthropic's Memory Tool spec, and owning the "AI coding agent memory without the cloud" wedge are the three highest-leverage moves.
**Updated 2026-05-02 — Hermes Agent (Nous Research) consideration:** A new direct competitor in the local-first OSS knowledge worker agent space launched 25 February 2026 and has 110K GitHub stars 10 weeks in. Hermes Agent markets the same "self-improving agent" narrative as Waggle but ships flat SQLite FTS5 (vs hive-mind bitemporal graph), undifferentiated text storage (vs I/P/B framing), no benchmark engagement beyond internal "40% speedup" claim, and no regulatory positioning. Hermes Agent **does not architecturally compete in hive-mind's quadrant** but it does compete for the same buyer through narrative overlap. The defensible response is the **peer-reviewed-style benchmark portfolio** (LoCoMo apples-to-apples + GEPA cross-family + forthcoming Gaia2 + τ³-bench banking_knowledge) — that is the moat Hermes cannot match in 2026. All Day 0 launch messaging must explicitly include the six differentiators per `strategy/competitive/2026-04-30-hermes-agent-intel-update.md` §2.
---
## Sources
- mem0: [mem0.ai/research](https://mem0.ai/research), [mem0.ai/pricing](https://mem0.ai/pricing), [arxiv 2504.19413](https://arxiv.org/abs/2504.19413)
- Letta: [letta.com](https://letta.com/), [docs.letta.com/concepts/memgpt](https://docs.letta.com/concepts/memgpt/)
- Zep / Graphiti: [blog.getzep.com](https://blog.getzep.com/graphiti-hits-20k-stars-mcp-server-1-0/), [getzep.com/pricing](https://www.getzep.com/pricing/), [arxiv 2501.13956](https://arxiv.org/abs/2501.13956), [github.com/getzep/zep-papers/issues/5](https://github.com/getzep/zep-papers/issues/5)
- Cognee: [cognee.ai/blog/deep-dives/ai-memory-tools-evaluation](https://www.cognee.ai/blog/deep-dives/ai-memory-tools-evaluation), [docs.cognee.ai/core-concepts](https://docs.cognee.ai/core-concepts)
- Supermemory: [supermemory.ai/pricing](https://supermemory.ai/pricing/), [supermemory.ai/research](https://supermemory.ai/research/)
- LlamaIndex: [developers.llamaindex.ai/python/framework/module_guides/deploying/agents/memory](https://developers.llamaindex.ai/python/framework/module_guides/deploying/agents/memory/)
- LangMem: [langchain-ai.github.io/langmem](https://langchain-ai.github.io/langmem/), [blog.langchain.com/langmem-sdk-launch](https://blog.langchain.com/langmem-sdk-launch/)
- OpenAI Memory: [openai.com/index/memory-and-new-controls-for-chatgpt](https://openai.com/index/memory-and-new-controls-for-chatgpt/), [memobase.io/blog/openai-memory](https://www.memobase.io/blog/openai-memory)
- Anthropic Memory Tool: [docs.claude.com/en/docs/agents-and-tools/tool-use/memory-tool](https://docs.claude.com/en/docs/agents-and-tools/tool-use/memory-tool), [anthropic.com/news/context-management](https://www.anthropic.com/news/context-management)
- Basic Memory: [github.com/basicmachines-co/basic-memory](https://github.com/basicmachines-co/basic-memory)
- Hermes Agent (Nous Research, added 2026-05-02): launch reference 25 February 2026; intel update `strategy/competitive/2026-04-30-hermes-agent-intel-update.md`; benchmark portfolio brief `briefs/2026-04-29-benchmark-portfolio-refresh-2026-venues.md` §3.1
- Market context: [mem0.ai/blog/state-of-ai-agent-memory-2026](https://mem0.ai/blog/state-of-ai-agent-memory-2026), [mempalace.tech/blog/best-ai-memory-frameworks-2026](https://www.mempalace.tech/blog/best-ai-memory-frameworks-2026)

392
docs/research/04-gepa-public-reveal-strategy.md Normal file
View File

@@ -0,0 +1,392 @@
# GEPA + ACE Self-Improvement Proof — Public Reveal Strategy
**Author:** Waggle OS research series (4 of 7)
**Drafted:** 2026-04-15 (overnight batch)
**Context:** Waggle's v1 evolution hypothesis produced a 108.8% C/A ratio (Gemma 4 31B + Waggle-evolved prompt vs raw Opus 4.6 on 10 coder questions, 4 blind judges). V2 scales to 60 examples × 3 domains × 3 baselines with a hard train/test split. Q1-Q5 decisions are captured in `docs/hypothesis-v2-decisions.md`; execution runbook in `docs/hypothesis-v2-execution-plan.md`. This document is the **public-communication strategy** for that result — how to reveal it without it getting pattern-matched as hype.
---
## ⚠ Critical corrections from research-agent returns (post-draft)
- **GEPA stands for "Genetic-Pareto"**, not "Goal-driven Evolution of Prompts Algorithm." Paper: Agrawal et al., arXiv:2507.19457, ICLR 2026 Oral. Core algorithm is reflective prompt evolution with Pareto frontier of candidates across objectives, beats RL (GRPO) by +6 % avg / +20 % max with ≤35× fewer rollouts, and beats MIPROv2 by >10 %. Integrated into DSPy 3.0 as `dspy.GEPA` and into MLflow prompt-opt APIs. Repo: github.com/gepa-ai/gepa.
- **"EvolveSchema" by Mikhail" could not be pinned down.** Best candidate as a public analog: **ACE — Agentic Context Engineering** (Zhang et al., Stanford/SambaNova, arXiv:2510.04618). No Mikhail on author list. Recommend dropping the Mikhail attribution in public publication unless the original internal source is located.
- **Gemma 4 31B** was released by Google on April 2, 2026 under Apache 2.0 and currently sits at Arena #3 open model (1452 Elo). Our v1 headline rides a very recent wave rather than fighting against it.
- **Reflection 70B (Matt Shumer, Sept 2024)** is the canonical cautionary tale — any "small beats big" claim is pattern-matched against it. Lesson applied in §5.1 below: publish reproducibility first, headline second.
---
## TL;DR
This is a **high-beta** moment for Waggle. A public "small model beats flagship" claim either compounds into a year of inbound enterprise leads, research credibility, and technical-brand equity — or gets shredded by a hostile HN thread and hurts Waggle's standing for 12 months afterward.
The difference is **almost entirely about credibility signals.** The result itself is defensible (multi-vendor judges, no Opus self-bias, train/test split committed, negative-result publication pre-committed per Q5). The failure mode is sloppy framing, premature victory-lap tone, or skipping the rigor narrative to front-load the "mind-blown" headline.
**Strategy:** ship a full arXiv-style research note *first*, then a Twitter thread + LinkedIn long-form + HN post *second* pointing at the note. The note earns the trust; the short-form captures the virality.
**Non-negotiable:** reproducibility repo live on day one. Split seed, eval dataset, evolved prompt, judge prompts, raw results JSON. The first skeptic who can re-run and see the same numbers does more marketing than any post.
---
## 1. The claim space
### 1.1 What v1 proved
- **On 10 curated coder questions judged by 4 independent blind judges, Gemma 4 31B with a Waggle-evolved prompt beat raw Opus 4.6 at per-judge mean C/A ratio of 108.8%.**
- The Opus judge itself ranked evolved-Gemma above raw-Opus — the opposite of self-preservation bias.
- The evolved prompt added only +91 tokens over baseline.
### 1.2 What v1 didn't prove
Per `docs/evolution-hypothesis-v2-plan.md`:
- n=10 is underpowered
- one-domain (coder) coverage
- weak baselines ("Answer clearly.")
- eval = train (GEPA and judges saw same examples)
- Opus-as-judge caveat (Opus is also Arm A)
- no confidence interval or significance test
### 1.3 What v2 will prove
- n=60 (30 train + 30 test), stratified 3 domains × 2 strata
- Hard train/test split, reproducible seed
- 3 graded baselines: weak prompt / human-engineered prompt / GEPA-evolved prompt
- 4-judge multi-vendor pool (Anthropic Sonnet + Anthropic Haiku + OpenAI GPT-5 + Google Gemini 2.5 Pro) — Opus dropped entirely
- Bootstrap 95% CI + permutation test at α=0.05
- **H₁:** C/A ≥ 0.95 for 2+ of 3 domains
- **H₂:** C/A ≥ 1.00 for 1+ domain (replicates v1 headline)
- **H₃:** C/A < 0.90 for 2+ domains → publish negative (pre-committed per Q5)
This is the right rigor level for a credible publication.
---
## 2. Audiences and what each needs
### 2.1 AI researchers (for trust / credibility)
**What they need to believe it:**
- Methods section written like an arXiv paper (methods / results / limitations / related work)
- Reproducibility repo (dataset, split seed, judge prompts, raw scores)
- Pre-committed negative-result publication
- Named judges (specific model versions + dates)
- Inter-judge agreement statistics
**Channel:** arXiv preprint + Twitter thread by a named author + HN post that links to the preprint *not* to the Waggle product
**Outcome if they nod:** retweets, podcast invites, academic citations in Q3-Q4 2026 follow-ups
### 2.2 Technical buyers — CTOs, VPs of Engineering, AI platform leads (for sales)
**What they need to believe it:**
- The same rigor above, *and* an answer to "can this run on my infrastructure with my data?"
- A cost comparison: inference cost of Gemma 4 31B at $X/Mtok vs Opus 4.6 at $Y/Mtok with the implicit "self-host and you save ~90%"
- A deployment story that matches their constraints (on-prem, private-VPC, etc.)
**Channel:** LinkedIn long-form post by Marko as founder; case-study PDF one-pager; direct outreach to 20-30 named targets with a personalized version
**Outcome if they nod:** KVARK enterprise conversations → EUR 1.2M+ pipeline expansion
### 2.3 Prompt engineers + applied ML engineers (for virality)
**What they need to believe it:**
- Side-by-side output examples (concrete, not abstract)
- The evolved prompt visible in full (they want to copy it)
- A reproducibility-to-the-minute guide
- A clear "this is not magic, here's how" explanation of GEPA + ACE mechanics
**Channel:** Twitter thread with screenshots; explainer blog post; YouTube walkthrough if we're ambitious
**Outcome if they nod:** tens of thousands of impressions, ecosystem amplification, some try Waggle themselves
### 2.4 The skeptic class (to survive)
Every rigorous result gets three kinds of pushback on HN/Twitter:
- **"They cherry-picked the questions"** → rebut with: train/test split, domain stratification, reproducibility repo with exact dataset
- **"The judges are biased"** → rebut with: 4-vendor pool, published judge prompts, inter-judge agreement statistics
- **"It's just prompt engineering, any tech-savvy person could do this"** → rebut with: compare against the human-engineered baseline (which is in the v2 arm list) and show that automated evolution *beats* human-engineered
- **"Won't replicate"** → rebut with: reproducibility repo + one-line docker command to re-run
- **"The 108.8% is within measurement noise"** → rebut with: bootstrap CI + permutation test p-value
Pre-emptively address all five in the methods + limitations section. Don't wait for them to be raised; lead with them.
---
## 3. The research note — structure
Target: ~15 pages, arXiv-style. Saves to `docs/research/evolution-hypothesis-v2-note-<date>.md` in the repo + mirror public URL at `waggle-os.ai/research/v2`.
### 3.1 Abstract (200 words)
> We report on a replication of Waggle OS's v1 evolution result at a 6× larger scale. In our v1 experiment (n=10, coder domain, 4 judges), Gemma 4 31B with a prompt evolved by Waggle's GEPA + ACE loop scored 108.8% of raw Claude Opus 4.6 per blind judge. To test whether this generalizes, we ran a v2 evaluation on 60 examples across 3 domains (writer, analyst, researcher), with a held-out 30-example test set, 3 graded baselines (weak / human-engineered / GEPA-evolved), and a 4-vendor judge pool (Anthropic Sonnet, Anthropic Haiku, OpenAI GPT-5, Google Gemini 2.5 Pro). We report C/A ratio of [RESULT] on the test set, 95% bootstrap CI [LO, HI], permutation test p=[P]. [H₁ / H₂ / H₃ verdict.] Full dataset, split seed, evolved prompts, judge prompts, and raw scores are available at [reproducibility repo URL].
### 3.2 Introduction (1 page)
- The problem: can a general-purpose evolution loop take an open-weight model to flagship quality on a user's task distribution?
- What we built: GEPA + ACE integrated, closed-loop on execution traces
- What we tested: v1 result replication at scale with a much tighter methodology
- Why it matters: local-first AI systems can achieve competitive quality without customer data crossing the training-loop boundary
### 3.3 Methods (3 pages)
- Dataset construction (60 examples × 3 domains × 2 strata, selection criteria)
- Split procedure (fixed seed, domain-balanced)
- Arms (A = raw Opus 4.6, B₁ = weak prompt, B₂ = human-engineered, B₃ = GEPA-evolved; all non-A arms run on Gemma 4 31B)
- GEPA iteration procedure (max 500 iterations or $80, early-abort on plateau)
- Judge pool, letter-to-arm randomization per example to prevent positional bias
- Scoring rubric (1-5 per arm per judge)
- Statistical tests (bootstrap CI, permutation test)
### 3.4 Results (3 pages)
- Primary: mean per-judge C/A ratio, aggregated median across judges
- Secondary: per-domain breakdown, per-arm raw rating distribution
- Inter-judge agreement statistics
- Cost / token-count per arm
- Evolution training curve (score-per-iteration)
### 3.5 Limitations (1 page)
- Sample size still modest (60)
- Domain coverage (3 out of potentially infinite task families)
- Judge model pool (4, all commercial; no open-weight judge)
- English-language only
- Gemma 4 31B specifically; we don't claim generalization to other 30B-class models without additional runs
- Time-bounded snapshot of model versions (Opus 4.6 as of April 2026)
### 3.6 Related work (1 page)
- GEPA original paper (cite; details from overnight research agent)
- EvolveSchema (Mikhail et al., cite)
- DSPy, TextGrad, OPRO, Promptbreeder, APE — positional context
- Orca 2 / Phi-3 / small-model-beats-flagship claims — prior art
- Multi-judge evaluation methodology papers
### 3.7 Conclusion (0.5 page)
- What we learned
- What we'd run next (per-domain evolution, longer context tasks, code generation with execution verification)
- Call to action: reproducibility repo is up, run it yourself
### 3.8 Appendices
- A: Full evolved prompt (B₃)
- B: Human-engineered baseline prompt (B₂)
- C: Judge rubric + prompts verbatim
- D: Example questions per domain (5 per domain sampled)
- E: Full results table (anonymized if needed)
---
## 4. The Twitter thread (12-18 tweets)
### Example draft — tweets 1-5
1. "Can a prompt-evolution loop take an open-weight 31B model past Claude Opus 4.6 on your tasks? We ran the test. (v1 replicated at 6× scale, rigorous methodology, multi-vendor judges, reproducibility repo public.) 🧵"
2. "v1 result: 108.8% C/A on 10 coder questions. 4 blind judges. No self-bias. Nice, but underpowered and single-domain. Critics would rightly call that."
3. "v2: 60 examples × 3 domains (writer/analyst/researcher) × 3 baselines (weak / human-engineered / GEPA-evolved). Train/test split with a fixed seed. 4-judge pool (Sonnet + Haiku + GPT-5 + Gemini 2.5). Opus *dropped* as a judge to kill the self-bias criticism dead."
4. "Result: [insert headline numbers + CI + p-value]. [Verdict on H₁/H₂/H₃.]"
5. "Reproducibility: [repo URL]. One command to re-run. Same seed → same result."
...continuing with domain-specific numbers, the evolved prompt, a side-by-side example, credits, call to action.
### Thread-writing rules
- Every claim has a number or a link
- No "mind-blowing" / "game-changing" / "insane" / any superlative that makes serious people bounce
- Thread author is Marko — personal account of the founder, not a corporate account
- Pin the thread
- Quote-retweet one thoughtful skeptic's objection with a calm response
- Engage with the thread for 48h minimum
---
## 5. The LinkedIn long-form (2000-3000 words)
Written in Marko's voice (see memory: `marko-markovic-style` skill). Executive / product-owner tone. Audience: CTOs, VPs Eng, enterprise buyers.
Outline:
1. **The problem** — "We kept running into the same thing at Egzakta. Our enterprise customers loved Claude Opus's quality, but couldn't use it on sensitive data. Self-hosted open-weight models were a step behind. We wanted to close that gap *without* asking customers to send data to the cloud."
2. **The hypothesis** — "What if prompt evolution — specifically an integrated GEPA + ACE loop — could take Gemma 4 31B running on a customer's infrastructure to the point where it beats raw Opus on the customer's actual tasks?"
3. **The v1 experiment** — tell the story of the 10-question test, the 108.8% result, the surprised-us-too moment
4. **Why we doubted it** — the 6 objections from the v2 plan
5. **The v2 experiment** — methodology, arms, judges, stats
6. **Results** — the numbers, with charts
7. **What this means for enterprise AI** — sovereignty is compatible with flagship quality if you have a working evolution loop. Your data stays inside; the prompts evolve against it. Your open-weight model gets smarter about *your* stack than any closed flagship can be.
8. **KVARK and Waggle** — this runs on your Kubernetes via KVARK; it runs on your laptop via Waggle. Same stack, different deployment.
9. **Call to action** — read the research note, clone the repo, re-run the experiment. If you want this on your stack, [contact/demo link].
---
## 6. The HN post
Title draft: "Show HN: We evolved a 31B model prompt past Claude Opus 4.6 — methodology + reproducibility repo"
Why HN: HN is where a rigorous result either gets celebrated or destroyed. We need HN to celebrate, which requires:
- **Title is honest, not hype-y.** "We evolved" not "BREAKING" not "insane result"
- **First comment is by the author** (Marko), explaining the setup + linking the research note + acknowledging the v1 caveats openly
- **Engage every top-level reply within 30 minutes** for first 4 hours
- **Have named senior ML people read the note pre-publication** — not necessarily endorse, but no surprises (see §8)
---
## 7. Demo assets
Prepare BEFORE the research note drops:
| Asset | Purpose | Format |
|---|---|---|
| Research note PDF | arXiv-style, 15 pages | Markdown → Pandoc → PDF |
| Reproducibility repo | Clone-and-run | `waggle-os/evolution-hypothesis-v2` GitHub repo |
| Side-by-side example | Visceral impact | Screenshot or GIF comparing arm outputs |
| Evolution training curve | "Here's the learning happening" | Line chart, 2D axes (iteration, score) |
| Cost comparison | Business case | Table: arm → $/1k requests at scale |
| Inter-judge agreement heatmap | Rigor signal | Heatmap across 4 judges × 3 arms |
| 60-second explainer video | Social shareable | Loom or OBS screencast |
| Evolved prompt PDF | Download-and-try | 1-page PDF with the evolved prompt verbatim |
| Demo workspace | Try-it-yourself | Pre-seeded Waggle workspace users can import |
---
## 8. Pre-publication warm list
Email 5-10 named people 72 hours before public post. Not for endorsement — for **no surprises**. Give them a heads-up + PDF draft + link to repo. Ask only: "Any methodological issues I should address before this goes out? I want to catch issues privately before critics catch them publicly."
Suggested warm list (refined based on who owns the prompt-evolution narrative):
- **Omar Khattab** (Stanford / DSPy) — the academic center of gravity for this line of work; GEPA was contributed into DSPy 3.0
- **Lakshya Agrawal** — first author on the GEPA paper (arXiv:2507.19457, ICLR 2026 Oral); courtesy brief
- **Krista Opsahl-Ong** — MIPROv2 author; competitor but credible gatekeeper
- **Qizheng Zhang** / **James Zou** (Stanford/SambaNova) — ACE paper authors; relevant to the structural-evolution claim
- **Simon Willison** (simonwillison.net) — kingmaker blog; one post moves the narrative
- **Nathan Lambert** (Interconnects / AI2) — researcher-practitioner, high-credibility amplifier
- **Swyx** (Latent Space) — podcast distribution if they bite
- **Jim Fan** (NVIDIA) — high-reach amplifier for agent-world results
- 1-2 Egzakta-network enterprise AI leaders (not at Anthropic / OpenAI / Google to avoid awkwardness)
- 1 technical journalist who covers AI rigorously (Import AI, The Batch)
Marko to cross-check against his actual network and adjust.
The warm-list response often catches 1-2 non-obvious issues that would otherwise be HN-critique #1. Worth the 72 hours.
---
## 9. Handling criticism and follow-up
### 9.1 Likely objections and ready responses
(All to be drafted into an FAQ or pinned thread replies)
| Objection | Response |
|---|---|
| "Judges are hallucinating their scores" | Inter-judge agreement stats show α = [X]. Human spot-check confirms in appendix D. |
| "Judges are biased toward verbose answers" | B₁ (weak prompt) often produces terse answers; it did *not* win by brevity. Length not correlated with score per judge (see appendix). |
| "Opus 4.6 is getting worse over time / not the current frontier" | We tested in [specific model version, dated]. Anthropic has not announced deprecation. |
| "Gemma 4 31B has seen benchmark contamination" | Eval examples were hand-authored, not drawn from MMLU/HellaSwag/etc. Split seed shows this. |
| "108.8% → 0.5% is all noise" | Bootstrap 95% CI is [LO, HI]; excludes zero / doesn't cross 1.0 for [X of Y] domains. |
| "Why should I trust your evolution loop vs DSPy/TextGrad?" | We don't claim unique novelty in the core algorithm; we claim the *integration* (GEPA + ACE on live user traces) works at product scale. DSPy path comparison is future work. |
| "This only works because you cherry-picked task types" | 3 domains (writer, analyst, researcher) span different reasoning styles; per-domain breakdowns in §3.4 show [XYZ]. |
### 9.2 If H₃ fires (negative result)
Per Q5 commitment: publish anyway. Title adjusts to "Our v2 hypothesis test did not replicate v1 — here's what we learned."
Outline:
- What we hoped
- What we got
- What that tells us about when evolution helps and when it doesn't
- What we'd try next
**This is worth publishing because:**
- Research integrity → trust compounds
- "We tried and it didn't work" is a legitimate contribution
- Pre-committing to publish either way is itself a marketing signal (savvy buyers notice)
### 9.3 If H₁ passes weakly (within noise of 0.95)
Publish with the softer framing. H₂ becomes follow-up work. Set expectations honestly.
### 9.4 If H₂ passes strongly (>1.0 across domains)
This is the jackpot scenario. Execute the full Twitter + LinkedIn + HN triple cycle. Book a podcast circuit.
---
## 10. Timing and cadence
**Target publication window:** within 14 days of Marko approving Q1-Q5 decisions. Per execution plan: ~4 days run + ~3 days write + 2 days external review + 2 days final polish + 2-3 days for warm-list response and surgery = ~14 days.
**Rollout sequence:**
- **T-72h:** warm-list emails go out
- **T-48h:** warm-list feedback incorporated
- **T-24h:** final research note published to repo (but not announced)
- **T-0:** public reveal — Twitter thread + LinkedIn post + HN post all within 2-hour window
- **T+24h:** engage every substantive reply; write FAQ update if patterns emerge
- **T+48h:** reach out to podcast circuit / journalists who engaged
- **T+7d:** retrospective on response, patterns, leads generated
**Avoid:** publishing on Monday (HN dies by Tue) or Friday (dies over weekend). Target Tuesday or Wednesday morning US Pacific time.
---
## 11. Success criteria
- [ ] Research note publishes within 14 days of greenlight
- [ ] Reproducibility repo is live and has ≥1 public third-party reproduction within 14 days of publication
- [ ] Twitter thread ≥100k impressions
- [ ] LinkedIn post ≥20k impressions + ≥5 CTO-level comments
- [ ] HN post reaches front page (top 30)
- [ ] ≥2 inbound enterprise conversations within 30 days citing the note
- [ ] ≥1 podcast invitation
- [ ] Zero successful "methodology-busting" critiques — if there are real issues, we catch them in warm-list surgery first
---
## 12. Waggle's narrative after the reveal
The reveal is not the destination; it's a node in a longer narrative arc:
- **Q2 2026:** v2 reveal → "Waggle can make your open-weight model competitive with flagships."
- **Q3 2026:** case study with a named enterprise pilot deployment → "Here's how Egzakta's customer X did it on-prem."
- **Q4 2026:** follow-up paper or extended note with additional domains / models → "It keeps working at wider scope."
- **Q1 2027:** open-source the evolution stack portions as part of the `hive-mind` OSS push (see report 01) → "Now you can run this yourself."
- **Q2 2027:** reference-customer Anna-Ska-style testimonial + Gartner cover-quote → "Enterprise adoption is real."
Every node reinforces the others. The reveal is the flywheel's first hard push.
---
## 13. Open decisions for Marko
1. **Warm list** — draft it together before sending; I have suggestions in §8 but you have the actual relationships
2. **Author voice** — single-author (you as founder) or dual-byline (you + one researcher)? Single is simpler; dual adds perceived rigor
3. **Repo org** — `waggle-os/evolution-hypothesis-v2` on the existing org, or a research-specific org? I recommend the former unless a research org is planned for other papers
4. **Podcast strategy** — opportunistic or actively pitched? I recommend opportunistic for the first appearance; if there's interest, pitch 2-3 specific shows
5. **Localization** — publish first in English; do we also do Serbian/EU-regional spins for Egzakta's home market? Defer to you; if yes, this doubles the rollout effort
If those five get quick answers, the entire plan is executable end-to-end in 14 days from greenlight.
---
## Closing
This is Waggle's single highest-leverage moment until the next headline. The research is real; the rigor is there; the story is clear. What the reveal needs is **patience** (wait until the rigor is complete, don't front-run with v1-only framing) and **discipline** (publish the research note *first*, then short-form, not the other way around).
Get this right and it accelerates KVARK enterprise pipeline, OSS `hive-mind` launch momentum, and Waggle-product consumer adoption simultaneously. Get it wrong and we spend 12 months rebuilding technical credibility.
The execution plan is in `docs/hypothesis-v2-execution-plan.md`. The decisions are in `docs/hypothesis-v2-decisions.md`. This document is the publication plan.
**Ready to execute on Marko's greenlight.**

272
docs/research/05-user-personas-ai-os.md Normal file
View File

@@ -0,0 +1,272 @@
# What Users Want from an AI Operating Workspace — Persona Voice-of-Customer
**Author:** Waggle OS research series (5 of 7)
**Drafted:** 2026-04-15 (overnight batch)
**Audience:** Product leadership deciding Q2/Q3 roadmap priorities and persona-specific launch messaging.
---
## TL;DR
Seven archetypal users of an AI operating workspace, distilled from market segmentation, conversations on HN/Reddit/Twitter through 2025-2026, and Waggle's own user-profile memories. For each: primary job-to-be-done, pain points with today's AI tools, what they'd pay for, and which Waggle tier/feature set serves them.
**Cross-persona signal:** persistent memory is the universal want (7/7). Compliance is top-3 for 3/7. Local-first matters for 5/7. Skill/connector extensibility matters most for the power-user personas (product owner, founder, IT admin). **Waggle's tier ladder (Trial → Free → Pro $19 → Teams $49/seat → Enterprise/KVARK) maps cleanly to this persona distribution; the one gap is persona 7 (prosumer creative), for whom the current Pro tier is priced right but the onboarding emphasizes work context over personal life.**
---
## Persona 1 — The Product Owner / Strategic Operator *(the Marko archetype)*
**Role signals:** Group CEO / VP Product / founder-operator / chief of staff. Runs 5-10 concurrent initiatives. Meets 6-10 stakeholders a week. Writes more than they code.
**Jobs to be done:**
- Stay on top of every concurrent project without dropping threads
- Remember what was decided, by whom, when, and why
- Draft stakeholder updates in the stakeholder's voice and context
- Delegate to specialists (human or AI) and check their work
**Pain points with today's AI:**
- ChatGPT forgets across conversations
- Claude Projects remembers *inside* a project but can't cross-link
- Notion AI is shallow on reasoning
- Every tool is a silo; copy-paste tax is real
- No trail of *why* a decision was made
**Wants from an AI OS:**
- A "where did we land on X?" search across every conversation, every document, every email I've had about X
- Auto-drafted stakeholder updates *in the right voice* (different for board vs team vs customer)
- A decision log that auto-maintains itself — I shouldn't have to type "Decision: Y" explicitly
- A multi-agent Room so I can watch a researcher + writer + analyst work in parallel on the same question
- Calendar/email/Slack context bleed into any conversation
**Pricing tolerance:** $19-49/mo personal budget; $49-200/seat on company card.
**Killer feature for this persona:** memory that survives across weeks and picks up threads when you say "where were we on that client pitch?" — plus a three-agent Room that handles prep while they're in a meeting.
**Waggle fit:** Pro or Teams tier. The Room canvas, persona system (13 built-in), and Memory Harvest (ChatGPT/Claude/Claude-code/Gemini/Perplexity) are all hit-first features for this persona. **Top gap:** stakeholder-voice detection isn't an explicit skill yet — could be a Gap E-style promotable skill that gets created on first use.
---
## Persona 2 — The Knowledge Worker *(legal / finance / HR / compliance)*
**Role signals:** lawyer, GC, CFO/Controller, HR lead, compliance officer. Every deliverable could end up in a courtroom or an audit.
**Jobs to be done:**
- Produce defensible work product (contracts, financial analyses, policies, audits)
- Comply with sector regulations (GDPR, SOX, HIPAA, AI Act, ISO 27001)
- Manage expertise — capture what the senior does so juniors can approximate it
- Defend positions with citations
**Pain points:**
- Cannot upload sensitive docs to public LLMs (confidentiality breach risk)
- No audit trail for AI-assisted decisions
- Can't explain *why* the AI gave that answer — dangerous for regulated work
- Senior expertise evaporates when the senior leaves
**Wants:**
- Compliance-by-default: every AI interaction logged with risk classification, attributable to a user, retainable for N years
- Document analysis that provably stays inside the organization
- Version-controlled drafts with peer/expert approval gates
- Expert-approved skills: "review this NDA with our firm's standards"
- Explainability: which memory influenced this answer, with citations
- AI Act FRIA-style assessments for high-risk workspaces (hiring, credit, legal)
**Pricing tolerance:** $49+ per seat; CIO budget. Will pay $200+/seat for sovereign deployment with proper controls.
**Killer feature:** an AuditReport PDF (AI Act Art 12/14/19/26/50 status) they can hand to their GC or regulator on demand.
**Waggle fit:** Teams tier for the day-to-day, KVARK for the firm-wide deployment. The compliance-PDF generator (Gap H shipped this session) is a direct hit. **Top gap:** expert-approved skills need a review/attestation workflow on top of the promotion gate (Gap E); today promotion is author-unilateral.
---
## Persona 3 — The Developer *(IDE power user, CLI native)*
**Role signals:** staff engineer / senior IC / solo hacker. Lives in the terminal. Has opinions about line length.
**Jobs to be done:**
- Ship code faster without sacrificing quality
- Minimize context-switching between ticket → code → review → deploy
- Maintain personal knowledge of codebase idioms across projects
**Pain points:**
- Copilot/Cursor forget the codebase's taste between sessions
- AI tools don't respect their style (spaces vs tabs, early-return idioms)
- Mystery-tokens hidden inside SaaS agents
- Escape-hatch problem: when the AI is wrong, getting back to manual is friction-heavy
**Wants:**
- Git-aware memory: "last time I touched this file, I chose pattern X because Y"
- Terminal-first workflow; nothing forced into a GUI
- Skill composability: write-test → implement → review → commit as a chainable pipeline
- Cost observability per turn (tokens in, tokens out, $$ spent)
- Local-first: my code and my memory don't leave the machine by default
- Fast escape hatches: `/fast`, `/bypass`, one-key undo
**Pricing tolerance:** $19-49/mo personal; will expense up to $200/mo if cost tracker shows ROI.
**Killer feature:** persistent memory of code patterns that survives IDE restart *and* syncs across their 3 machines. Paired with a cost tracker that ends the "my API bill shocked me" problem.
**Waggle fit:** Pro tier. Strong match on terminal-first (Claude Code harvest built-in), cost tracker, Git tools, background bash. **Top gap:** no dedicated IDE integration (Cursor adapter is on the backlog); devs want Waggle memory available *inside* the IDE, not just in a separate desktop app.
---
## Persona 4 — The Research Scientist *(AI researcher / analyst / management consultant)*
**Role signals:** PhD-adjacent roles where deliverables need citations. Thinks in hypotheses. Allergic to hallucination.
**Jobs to be done:**
- Conduct deep-research flows across many sources
- Track citations with provenance
- Run reproducible experiments and write them up
- Synthesize findings into structured knowledge
**Pain points:**
- Perplexity/ChatGPT hallucinate citations
- Can't trust a claim without tracing it to the source
- Research notebooks get orphaned from their source data
- Can't reproduce last month's experiment because the state's gone
**Wants:**
- Citation-aware search: every claim links to the frame/document/URL it came from
- Experiment logs with input → model → output → evaluation
- Data lineage across sources (this conclusion depended on this frame, which came from this harvest)
- Wiki-style structured knowledge (entity pages, concept pages, synthesis pages)
- Reproducible workspace snapshots (export and re-import for colleagues or future-self)
**Pricing tolerance:** $49-200/mo via institutional budget.
**Killer feature:** "show me every frame that influenced this answer" with a timeline of when each was acquired. Plus the Wiki Compiler producing publishable entity/concept pages.
**Waggle fit:** Pro or Teams tier. Waggle's Wiki Compiler, HybridSearch provenance (source field: user_stated / tool_verified / agent_inferred), and Execution Trace Store are all hits. **Top gap:** citation attribution inside assistant messages isn't exposed in the UI today — provenance is captured but not rendered inline with answers.
---
## Persona 5 — The Startup Founder *(solo or 2-10 team)*
**Role signals:** wears 5 hats. CEO / CTO / sales rep / recruiter / customer success in rotation.
**Jobs to be done:**
- Ship the product
- Raise money
- Hire the first 10
- Keep customers happy
- Stay alive (runway)
**Pain points:**
- Tool sprawl — SaaS bill exceeds laptop price every month
- Context from a customer call on Tuesday is lost by Thursday
- Writing *everything* solo (investor updates, job posts, onboarding emails, product copy)
- No institutional memory — the founder IS the memory
**Wants:**
- Swiss army knife: CRM + fundraising + strategy + hiring in one memory
- Fast skills for email drafts, deck outlines, investor updates in *their* voice
- Low cost ($19/mo ceiling early, $49/mo once revenue hits)
- Zero learning curve — must work the first 5 minutes
- Local-first — competitive data doesn't leak to OpenAI/Anthropic
**Pricing tolerance:** $19/mo until funded, $49/mo post-seed, $200+/mo once they can hire ops.
**Killer feature:** one mind that remembers every customer call, investor pitch, team standup, competitor mention, go/no-go decision — *forever*.
**Waggle fit:** Trial → Free → Pro arc. Memory Harvest from ChatGPT/Claude/Notion, rapid skill creation, low cost. **Top gap:** no native CRM skill/connector yet at the quality level a founder expects (HubSpot/Salesforce connectors exist in the MCP catalog but aren't pre-configured).
---
## Persona 6 — The Enterprise IT Admin / CIO / CISO
**Role signals:** responsible for AI governance at a 500-50,000 employee firm. Has board reporting obligations.
**Jobs to be done:**
- Allow safe AI adoption across the workforce
- Comply with GDPR / EU AI Act / industry regulation
- Prove to the board that AI isn't leaking IP or creating audit exposure
- Minimize shadow-IT AI usage
**Pain points:**
- Employees use random LLMs with company data ("shadow AI")
- No audit trail
- GDPR/AI-Act exposure (fines up to 7% of global revenue)
- Every vendor promises "enterprise-ready" — few actually are
**Wants:**
- Sovereign deployment (on-prem or dedicated VPC) — data never leaves perimeter
- RBAC, SSO/SAML, MFA, SCIM provisioning
- Full audit — every AI interaction attributable, retainable, exportable
- Data residency controls (EU-only, US-only, specific region)
- Kill switch — disable AI features company-wide in one click
- Zero data exfil *provably*
**Pricing tolerance:** $40K-500K+/year depending on seats; $1M+ deals for multi-region Fortune 500 deployments.
**Killer feature:** Waggle on *their* Kubernetes, connected to *their* permissioned data, with a compliance PDF ready to show their DPO and their regulator.
**Waggle fit:** KVARK (enterprise tier). The sovereign value proposition from CLAUDE.md §9 ("Everything Waggle does — on your infrastructure … full data pipeline injection, your permissions, complete audit trail, governance. Your data never leaves your perimeter") is written for this persona. **Top gap:** SSO/SAML/SCIM aren't in the current codebase surface; the team-sync layer exists but enterprise IDP integration is a KVARK deployment-time concern rather than a productized feature today.
---
## Persona 7 — The Consumer Prosumer *(creator, writer, entrepreneur-of-one)*
**Role signals:** content creator, novelist, freelancer, solopreneur, life-optimizer.
**Jobs to be done:**
- Creative output (writing, video, image)
- Personal knowledge building (second brain)
- Lifestyle automation (travel, health, finance)
**Pain points:**
- ChatGPT forgets every conversation
- Claude Projects has quotas that hit mid-flow
- Privacy — life data treated as training corpus
- Every app is a silo, cross-search is impossible
**Wants:**
- AI that remembers *life* context (family, health, hobbies, goals, relationships)
- Creative tools (image gen, voice, video) alongside text
- Memory of preferences (writing voice, visual style, tone)
- Casual setup — no CLI, no configuration files
- Privacy — "my life is not your training data"
**Pricing tolerance:** $19/mo Spotify-tier mental model; some will pay $49 for creative add-ons.
**Killer feature:** persistent memory that grows over months and makes the assistant actually *know them* as a person.
**Waggle fit:** Pro tier with creative skills. Memory Harvest is strong. **Top gap:** onboarding currently emphasizes work/workspace context; a "personal life" mode that greets with "what should I remember about you as a person, not as a professional?" would widen this segment.
---
## Cross-persona observations
| Want | Who wants it (n/7) | Waggle status |
|---|---|---|
| Persistent memory across conversations | 7/7 | ✅ shipped — core differentiator |
| Compliance-by-default | 3/7 (Knowledge worker, IT admin, Researcher) | ✅ shipped + boardroom PDF (Gap H) |
| Local-first / sovereign | 5/7 (Developer, Founder, IT admin, Researcher, Prosumer) | ✅ Tauri binary + KVARK on-prem |
| Citation / provenance | 3/7 (Researcher, Knowledge worker, IT admin) | ⚠️ captured but not exposed inline in UI |
| Skill/connector extensibility | 4/7 (Product owner, Developer, Founder, IT admin) | ✅ shipped — Gap A/E/F + MCP catalog |
| Multi-agent coordination | 2/7 (Product owner, Founder) | ✅ Room canvas + subagent tools |
| Self-evolution visibility | 2/7 (Product owner, Researcher) | ✅ Evolution tab + hypothesis v2 in flight |
| Creative/multimodal | 2/7 (Prosumer, Founder) | ⚠️ generate_docx + nano-banana skills exist; video/voice lighter |
| Cost observability | 2/7 (Developer, Founder) | ✅ CostTracker shipped |
| IDE-native integration | 1/7 (Developer) | ❌ no IDE plugin today |
| SSO / SAML / SCIM | 1/7 (IT admin) | ❌ deployment-time for KVARK, not productized |
## Persona-to-roadmap heat map
| Roadmap item | Top-benefiting personas |
|---|---|
| Hypothesis v2 publication | 1, 4 (credibility), 6 (board trust) |
| Stripe → Teams tier live | 2, 3, 5 (revenue path) |
| Code signing / installer polish | 5, 7 (low-friction install) |
| Cursor harvest adapter | 3 |
| Shared team memory | 1, 2, 6 |
| Citation-inline UI | 4, 2 |
| IDE plugin | 3 |
| SSO / SAML | 6 |
| "Personal life" onboarding mode | 7 |
| Stakeholder-voice skill pack | 1 |
## Closing note
The seven personas above are **not** orthogonal — most real users occupy 2-3 at once (e.g., a founder-developer, a researcher-prosumer). The design implication is that Waggle should avoid hard persona-segmentation UX (single-path onboarding wizards per persona) and instead expose primitives that compose for overlapping jobs. The persona system already does this: 13 built-in personas + custom-persona support + a Room where 3 personas can collaborate.
The strongest single message across all 7: **"Your AI remembers. Your data stays yours. Your compliance trail writes itself."** — three sentences that 6 of 7 personas would nod at. That's the billboard.

406
docs/research/06-waggle-os-product-overview.md Normal file
View File

@@ -0,0 +1,406 @@
# Waggle OS — Product Overview
**Author:** Waggle OS research series (6 of 7)
**Drafted:** 2026-04-15 (overnight batch)
**Scope:** Comprehensive product overview — pitch, architecture, memory stack, evolution stack, feature taxonomy, tier model, user patterns, competitive positioning, defensibility.
---
## TL;DR
**Waggle OS is a workspace-native AI agent platform with persistent memory.** It ships as a Tauri 2.0 desktop binary for Windows and macOS, with a local Fastify sidecar and SQLite-backed memory. The product pitch in one sentence: *"Your AI remembers. Your data stays yours. Your compliance trail writes itself."*
It has three strategic roles:
1. **Stand-alone product** across 5 tiers (Trial 15d → Free → Pro $19 → Teams $49/seat → Enterprise/KVARK consultative)
2. **Demand-gen engine for KVARK** — Egzakta Group's sovereign-AI enterprise platform ([www.kvark.ai](https://www.kvark.ai))
3. **Research platform** — the evolution subsystem (GEPA + EvolveSchema loop) runs continuously, producing the headline claim that **Gemma 4 31B with a Waggle-evolved prompt scored 108.8% of raw Opus 4.6** on blind multi-vendor evaluation (v1, 10 coder questions; v2 scaling to 60 × 3 domains pending Marko's Q1-Q5 decisions)
It's differentiated by the combination of: persistent memory (not just RAG), compliance-by-default (EU AI Act logging baked in), local-first deployment (not SaaS-first with an on-prem afterthought), self-evolution (prompts improve from real usage), and a wiki-style structured-knowledge compiler on top of the memory corpus.
---
## 1. The pitch
### 1.1 One-sentence pitches per audience
- **Prosumer:** "The AI that actually remembers you. Works on your laptop. Your data stays yours."
- **Small team:** "Your team's AI operating system. Shared memory, shared skills, one $49/seat."
- **Enterprise buyer:** "Everything your team does with AI — on your infrastructure, with full compliance out of the box."
- **Researcher / technical:** "Persistent multi-tenant memory with compliance-by-default, frame-based temporal storage, hybrid RRF search, and a closed-loop prompt-evolution stack."
- **Investor:** "Waggle is the demand-generation wedge for KVARK, Egzakta's EUR 1.2M-contracted sovereign-AI enterprise platform. The free tier creates lock-in (memory is free forever), the paid tiers monetize (skills + connectors), and the enterprise tier (KVARK) closes on-prem deals."
### 1.2 The strategic function
Waggle OS is **the wedge.** The memory + harvest + skills system drives adoption and retention in the consumer/prosumer/team segments. As those users inside enterprise organizations hit the "we can't ship sensitive data to the cloud" wall, Waggle's enterprise tier (KVARK) becomes the answer — same product, deployed on the customer's infrastructure, with full data-plane control and compliance.
---
## 2. Architecture — verified April 2026
From `D:/Projects/waggle-os/CLAUDE.md`:
| Layer | Stack |
|---|---|
| Frontend | React **19** + TypeScript + Vite + Tailwind 4 + base-ui/react |
| Desktop | Tauri 2.0 (Rust shell) |
| Backend (local) | Fastify sidecar (Node.js, bundled into Tauri) |
| LLM routing | LiteLLM (`litellm-config.yaml`) |
| Database | SQLite via `@waggle/core` (better-sqlite3 + sqlite-vec-windows-x64) |
| Memory | FrameStore + HybridSearch + KnowledgeGraph + IdentityLayer + AwarenessLayer |
| Agent runtime | `packages/agent/src/agent-loop.ts` |
| Billing | Stripe (`stripe@^21.0.1`) — code-complete per `docs/p0-stripe-and-signing-readiness.md` |
| Design | Hive DS — honey #e5a000 / hive-950 #08090c / accent #a78bfa |
| Tests | Vitest (unit) + Playwright (E2E) |
| Deploy | Dockerfile + docker-compose.production.yml + render.yaml |
Package manager: npm (with bun.lock present). Node ≥20. 16 packages in the workspace (`packages/*`).
### 2.1 Why local-first
The Tauri binary runs on the user's machine. The Fastify sidecar is bundled into the binary — it's a local HTTP server listening on a private port. The SQLite `.mind` files live in `~/.waggle/minds/`. This means:
- Memory is local. No cloud dependency for the core experience.
- LLM calls go through the user's configured provider (Anthropic / OpenAI / Ollama / LiteLLM proxy) — the user controls where inference happens.
- Offline use works for memory search, identity, awareness; it falls back gracefully when LLM provider is unreachable.
- Team workspaces opt into a team server for cross-member sync; personal stays fully local.
- Enterprise (KVARK) mirrors the same architecture but on the customer's Kubernetes / on-prem.
This is **architecturally opposite** to SaaS-first competitors (ChatGPT, Notion AI, Dust.tt). Those can add "on-prem" as a variant; Waggle is on-device by default.
---
## 3. The memory stack — the core differentiator
Five layered components, all in `@waggle/core`:
### 3.1 FrameStore (`frames.ts`)
A frame is the atomic unit of memory. Three frame types, modeled loosely after video compression:
- **I-frame (Independent)** — a keyframe; full content, self-contained
- **P-frame (Predicted)** — delta from an earlier I-frame; contextual continuation
- **B-frame (Bidirectional)** — a link between frames (entity co-occurrence, cross-session reference)
This gives the memory system a temporal structure — related content stays grouped, decay works on the cluster not per-item, and consolidation can merge P-frames back into an upgraded I-frame without losing history.
### 3.2 SessionStore (`sessions.ts`)
A session is a conversation arc. Each session has a `gop_id` (Group of Pictures, to extend the video metaphor). Sessions have lifecycle states: active → closed → archived. The weaver consolidates and distills per-session.
### 3.3 HybridSearch (`search.ts`)
RRF (Reciprocal Rank Fusion) hybrid of vector + BM25/FTS5 keyword, with scoring profiles: `balanced / recent / important / connected`. The combined-retrieval layer (`agent/src/combined-retrieval.ts`) merges workspace + personal + KVARK enterprise search with source attribution and conflict detection.
### 3.4 KnowledgeGraph (`knowledge.ts`)
Entities and relations with SCD-Type-2 temporal validity (`valid_from` / `valid_to`). Auto-populated by CognifyPipeline on every memory write. The KG is what powers Waggle's "smart about *you*" moments — when you mention "Samantha," the agent knows it's the Samantha from company Acme, not a different one.
### 3.5 IdentityLayer + AwarenessLayer (`identity.ts`, `awareness.ts`)
- **Identity** = stable facts about the user (name, role, company, preferences, communication style). Auto-derived from the first N conversations during onboarding.
- **Awareness** = volatile state (active tasks, recent actions, pending approvals, flags). Updated on every turn.
### 3.6 CognifyPipeline (`agent/src/cognify.ts`)
The write path. On every `save_memory` call: extract entities with an LLM, resolve to the KG (add new or merge with existing), create the frame, index for search, update awareness.
### 3.7 MemoryWeaver (`@waggle/weaver`)
The consolidation loop. Runs on schedule (hourly consolidation, daily decay). Operations:
- **consolidateGop** — merge I + P-frames into a richer I-frame, deprecate the old P-frames
- **decayFrames** — delete deprecated zero-access frames
- **strengthenFrames** — upgrade importance of heavily-accessed frames (temporary → normal → important)
- **linkRelatedFrames** — detect shared entities across frames, create B-frames to connect them
### 3.8 Wiki Compiler (`@waggle/wiki-compiler`)
A novel layer on top of the memory. It reads the user's entire memory corpus and produces structured *pages*:
- **Entity pages** — per-entity (per-person, per-project) summarization with timeline
- **Concept pages** — clustered themes across frames
- **Synthesis pages** — multi-source distillation on specific queries
Per memory: 18 MCP tools wired to the compiler; 7 wiki pages built from real data in live test. This is Waggle's answer to "how do I navigate my memory?" — a living second-brain wiki that updates incrementally.
### 3.9 Multi-mind layer
- **Personal mind** — lives at `~/.waggle/minds/personal.mind`
- **Workspace minds** — per-workspace at `~/.waggle/minds/<workspaceId>.mind`
- **Team minds** — team-scoped MindDB, shared via `TeamSync` (today: frame sync only; see `docs/p1-features-triage-2026-04-15.md` for the expansion plan)
- **MultiMindCache** — opens/closes MindDBs on demand with LRU eviction
- **Cross-workspace read** — `read_other_workspace` tool with approval gate for sanctioned cross-mind access
---
## 4. The evolution stack — Waggle's research differentiator
Ten files in `packages/agent/src/` implementing a closed-loop prompt-evolution system:
- `execution-traces.ts` — records every chat turn with inputs, outputs, scores
- `eval-dataset.ts` — curates held-out eval sets from trace data
- `judge.ts` — LLM-as-judge scoring (multi-vendor pool)
- `iterative-optimizer.ts` — GEPA-style iterative mutation with rank-by-score
- `evolve-schema.ts` — EvolveSchema-style structural prompt evolution (per Mikhail's paper, integration target — being verified overnight)
- `compose-evolution.ts` — combines GEPA + EvolveSchema signals
- `evolution-gates.ts` — constraint gating (can't deploy a candidate that fails X)
- `evolution-llm-wiring.ts` — model routing for evolution runs
- `evolution-orchestrator.ts` — high-level run coordinator
- `evolution-deploy.ts` — atomic deploy of accepted candidates (persona + behavioral-spec overrides)
**Current headline result (v1):** 10 coder questions, 4 blind judges (Haiku + Sonnet + two others), Gemma 4 31B with Waggle-evolved prompt scored **108.8% of raw Opus 4.6**. Per-judge geometric mean, with multi-vendor judge pool so no self-bias. Evolved prompt added only +91 tokens over baseline.
**v2 scaling plan:** 60 examples × 3 domains (writer/analyst/researcher) × 3 baselines (weak / human-engineered / GEPA), multi-vendor 4-judge pool, bootstrap 95% CI + permutation test, hard train/test split, ~$200, ~4 days. **Q1-Q5 decisions memo'd** in `docs/hypothesis-v2-decisions.md`; **execution plan** in `docs/hypothesis-v2-execution-plan.md`. Awaiting Marko's greenlight to run.
### Why this matters commercially
Every KVARK sales conversation benefits from being able to say: "Our evolution stack continuously improves your deployment's prompts on your own data — *without* sending anything off-premise. A 31B model can outperform flagship cloud models when you let it learn from your usage." Self-evolution on sovereign infrastructure is a structurally harder promise for Anthropic / OpenAI / Google to match, because their business model forbids customer data crossing the return path to their training loop.
---
## 5. Features — taxonomy
### 5.1 Core features (all tiers)
| Feature | Where |
|---|---|
| Persistent memory | FrameStore + CognifyPipeline |
| Cross-tool harvest | 11 adapters (chatgpt/claude/claude-code/claude-desktop/gemini/perplexity/markdown/plaintext/pdf/url/universal) |
| Hybrid search | HybridSearch (RRF vector + keyword) |
| Knowledge graph | KnowledgeGraph with SCD-2 |
| Wiki compiler | `@waggle/wiki-compiler` |
| Agent runtime | multi-persona, multi-turn, tool-aware |
| Skills | 20+ starter + user-created + auto-extracted |
| Connectors | 12 native + 148+ MCP catalog |
| Compliance logging | InteractionStore + ReportGenerator |
| Cost tracking | CostTracker (per model, per workspace) |
### 5.2 Pro-tier adds ($19/mo)
- Unlimited workspaces (Free caps at 5)
- Full MCP catalog access
- All 12 native connectors
- Skill marketplace access
- Workspace-scope skill promotion
- Advanced evolution tab (review + accept evolved prompts)
### 5.3 Teams-tier adds ($49/seat/mo)
- Shared team mind (team-scoped frames + skills)
- WaggleDance (multi-agent coordination across team members)
- Governance controls (skill promotion approvals, audit reports per user)
- Team-level compliance PDF rollup
### 5.4 Enterprise / KVARK adds (consultative)
- On-premise / private-VPC deployment
- SSO/SAML/SCIM + RBAC
- Enterprise skill + connector whitelisting
- Sovereign LLM routing (your models, your endpoints)
- Data residency controls
- Custom compliance frameworks beyond AI Act
- Professional services engagement model
### 5.5 Notable feature deep-dives
- **Room canvas** — a visual space where multiple personas can be assigned to concurrent work on a shared question. Real-time status tiles ("researcher: running / writer: waiting / analyst: done"). Per memory: shipped in Phase A.
- **Personas** — 13 built-in (researcher, writer, analyst, coder, project-manager, executive-assistant, sales-rep, marketer, product-manager-senior, hr-manager, legal-professional, finance-owner, consultant), custom-persona support via `custom-personas.ts`. Per CLAUDE.md §5, 4 new planned (general-purpose, planner, verifier, coordinator) for a target of 17.
- **Onboarding wizard** (`apps/web/src/components/os/overlays/OnboardingWizard.tsx`) — 7-step; post Memory-Harvest-Strategy recommendation should become harvest-first.
- **Evolution tab** — MemoryApp subtab; shows pending evolution runs, before/after previews, accept/reject UI.
- **Evolution hypothesis tests** — Waggle is the only AI product I know of that *runs evolution A/B tests on itself and publishes the results* (pending v2 publication).
- **Approval inbox with grants** — every destructive action (delete file, send email, run bash) shows an approval card. Three autonomy tiers: Normal (always approve) / Trusted (session-level trust) / YOLO (accept all, for dev/CI).
### 5.6 Trial / Free / Pro differentiation
Per CLAUDE.md §1:
| Tier | Price | Purpose |
|---|---|---|
| TRIAL | $0 / 15 days | All features unlocked; falls back to FREE after 15 days |
| FREE | $0 forever | 5 workspaces, agents, built-in skills only |
| PRO | $19/mo | Unlimited, marketplace, all connectors |
| TEAMS | $49/mo per seat | Shared workspaces, WaggleDance, governance |
| ENTERPRISE | Consultative | KVARK sovereign on-prem |
**Moat strategy** (per memory `project_tier_strategy`): Memory + Harvest is free forever (lock-in). Agents are free (they generate memory). Skills and connectors are the upgrade trigger. Team/Enterprise is about governance + sovereignty, not feature count.
---
## 6. User patterns
### 6.1 Solo pattern (Free / Pro)
- Install Waggle → onboarding wizard → harvest ChatGPT/Claude → identity auto-populates
- User works in personal mind; frames accumulate; skills get auto-extracted
- Evolution runs quietly; user accepts improvements occasionally
- After ~30 days, the user's mind has 1000+ frames, 20+ skills (mix of starter + custom), 3-5 connectors
- The user's "aha" moment: asking a question that *their* version of Waggle answers uniquely well because of their harvest
### 6.2 Team pattern (Teams)
- Workspace admin creates the team workspace + invites 2-10 members
- Each member harvests their own AI tools → team mind fills with diverse-perspective frames
- Shared skills emerge (one member's "client-proposal-v2" skill gets promoted to team scope)
- Disagreements surface via Gap K's write-path contradiction detection ("Alice's frame says X, Bob's says NOT X — worth a conversation")
- Manager uses compliance PDF (Gap H) for monthly / quarterly governance check-in
### 6.3 Enterprise pattern (KVARK)
- IT admin deploys Waggle/KVARK on customer infrastructure (Kubernetes + pg/Redis + LiteLLM endpoint + SSO)
- RBAC + skill whitelisting applied
- Enterprise-scope skill library managed by a Center of Excellence team
- Every business unit has its own workspace; cross-unit read requires approval
- Executive dashboard aggregates compliance + cost + usage across org
- Professional services engagement delivers custom connector SDK work + workspace templates per department
---
## 7. Competitive positioning
### 7.1 Nearest competitors and how Waggle differs
| Competitor | Waggle's advantage |
|---|---|
| **Claude Projects** | Claude can't harvest from ChatGPT/Cursor; isolated per project; no on-prem; no skill marketplace; no evolution |
| **ChatGPT Teams / Workspace** | ChatGPT forgets across conversations (limited memory); no on-prem; all data on OpenAI infra |
| **Cursor** | IDE-only; no persistent memory across projects; no non-dev personas |
| **Notion AI** | Shallow reasoning; data captive in Notion; no multi-agent |
| **Mastra / Letta / CrewAI** (agent frameworks) | Lib-first, not product; requires dev effort to assemble; no memory out of the box |
| **mem0 / Zep** (memory libs) | Lib-first, not product; no wiki/skills/compliance/UI |
| **Dust.tt** | SaaS-first; less compliance story; smaller ecosystem |
| **Glean** | Search-focused; no agent action; no local deployment |
| **Writer / Cohere North** | Closed-platform SaaS; steep enterprise-only pricing; less memory story |
| **Microsoft Copilot Studio** | Deep MS ecosystem lock-in; not for non-MS shops; Azure-only |
| **Salesforce Agentforce** | CRM-scoped; not a general AI OS |
### 7.2 Defensibility layers
1. **Local-first architecture** — structurally harder for SaaS-first competitors to copy quickly
2. **Frame-based memory model** — patentable; temporal structure + provenance is non-obvious
3. **Harvest corpus lock-in** — once a user has 1000+ frames, switching costs are psychological
4. **Compliance-by-default** — AI Act is coming; competitors who retrofit will lag
5. **Evolution loop** — hard to replicate at the model vendor level without customer data access (which they don't have and won't have)
6. **Brand: Hive DS + narrative cohesion** — Waggle / hive-mind / honeycomb / KVARK forms a coherent aesthetic + story
### 7.3 What could disrupt Waggle
- **Anthropic Claude-native memory** shipping across devices with their own harvest — if they had the right to access the user's ChatGPT history (they don't), they could win memory at the model-vendor layer
- **OpenAI Workspace with true persistent memory** — same caveat
- **Apple Intelligence** shipping a system-level memory layer across iOS/macOS — potentially disruptive for the prosumer segment
- **Open-source takeoff** — if mem0 or Letta suddenly get 50k stars and a VC-backed productization, they close the memory gap
**Waggle's strategic countermove:** ship the OSS split (`hive-mind` — see `docs/research/01-oss-memory-packaging-strategy.md`) to set the terms of the ecosystem. If the OSS memory layer becomes a standard, Waggle benefits as the reference implementation and best-product-on-top.
---
## 8. Shipping state as of 2026-04-15
### 8.1 Production-ready
- Core memory stack: frames, sessions, search, KG, identity, awareness, cognify, weaver, wiki compiler
- Agent runtime: chat loop, personas, tools, hooks, approval gates
- Harvest: 11 adapters production-tested, 156+ frames from real Claude Code usage per prior memory
- Evolution stack: 10 files, 357+ evolution tests, end-to-end closed loop, v1 hypothesis published
- Compliance: interaction store, status checker, report generator, PDF generator (Gap H shipped this session)
- Test suite: **1957/1957 green** after this session's sweep (agent package)
- TypeScript: **clean** on @waggle/agent, @waggle/core, @waggle/server
- Tauri desktop binary: builds, runs, M2 tested per prior memory
### 8.2 Blocked on external action
- **Stripe products** — code-complete; dashboard setup + env vars needed
- **Windows code signing** — EV cert purchase (1-3 day issuance)
- **Hypothesis v2 run** — Q1-Q5 decisions memo'd awaiting Marko's greenlight
### 8.3 Deferred with clear triggers
- Cursor harvest adapter
- Copilot harvest adapter (blocked on GitHub export API)
- StorageProvider wiring for agent file tools (premature abstraction today)
- Shared team memory expansion (beyond frame sync)
### 8.4 Planned but not started
- Ollama + hardware scan + NSIS installer (free-tier local-inference failover)
- OnlyOffice inline editing (M4+; KVARK-only for now)
---
## 9. Metrics the product should hit in 2026-2027
### 9.1 Activation
- ≥60% of new users harvest ≥1 source in the first session
- ≥30% complete the 3-source "my AI life is unified" journey in week 1
### 9.2 Retention
- 30-day retention ≥2.5× higher for harvested-users vs non-harvested-users
- 90-day retention for Pro tier ≥75%
- Teams tier churn <5% annual
### 9.3 Revenue
- Free → Pro conversion 5-10% (industry benchmark for freemium prosumer SaaS)
- Teams ACV $49 × median seats × 12 = realistic $3k-6k ACV per team
- Enterprise deal sizes $40k-500k+ initial, $100k-1M+ expansion
- Egzakta's EUR 1.2M contracted + KVARK pipeline growing
### 9.4 Research / brand
- v2 hypothesis published (research note)
- 1-3 podcast appearances
- Reference-customer quote from a compliance-sensitive enterprise buyer
- OSS hive-mind repo ≥1k stars in 6 months (if we ship the OSS split)
---
## 10. Strategic risks + mitigations
| Risk | Mitigation |
|---|---|
| Anthropic / OpenAI ship credible memory | Waggle's local-first + cross-tool harvest + evolution are structurally different; lead by 6-12 months on compliance |
| Enterprise sales cycles too long for pipeline | Free-tier-to-enterprise motion (land prosumer users → they advocate inside their company) |
| AI Act enforcement delays hurt urgency | Back-up argument: governance-by-default is also good GDPR/SOC2/ISO posture |
| Tauri ecosystem risk (niche vs Electron) | Architecture is portable; can swap to Electron if needed (wouldn't bet the company) |
| Evolution claim doesn't replicate at scale | Q5 pre-commits to publishing negative results; trust from honesty > trust from hype |
| OSS community captures the memory narrative first (mem0 gets funded) | Ship `hive-mind` now; be the reference implementation |
---
## 11. The one-paragraph strategic thesis
Persistent, local-first memory is the feature every AI product will have in 5 years. **Waggle is building it now as a coherent product stack with compliance, self-evolution, and an enterprise sovereignty answer (KVARK) that SaaS-first competitors will struggle to replicate.** The freemium wedge drives adoption, the harvest layer creates lock-in, the skills + connectors layer monetizes, and the enterprise tier closes on the largest budgets. The evolution subsystem is the proof point that differentiates Waggle in conversations with technical buyers and researchers. The OSS split (`hive-mind`) is the play to own the memory-layer narrative before a competitor does.
---
## Appendix A — 16-package workspace map
```
packages/
├── admin-web UI for team/enterprise admin console
├── agent Agent runtime — the orchestration + tools + personas core
├── cli Headless Waggle for scripting + CI
├── core Memory primitives (frames, search, KG, identity, awareness, mind, harvest, compliance)
├── launcher Installer + first-run bootstrap
├── marketplace Skill / connector marketplace (security gate lives here)
├── memory-mcp MCP server exposing memory tools to other AI agents
├── optimizer Thin Ax wrapper — one-shot program utilities (see docs/optimizer/README.md)
├── sdk Starter skills + templates for devs
├── server Fastify sidecar — wires @waggle/core + @waggle/agent into HTTP + SSE
├── shared Types, constants, Zod schemas, tiers, MCP catalog
├── ui Shared React primitives
├── waggle-dance Multi-agent coordination (team mode)
├── weaver Memory consolidation / decay / linking / distillation
├── wiki-compiler Entity / concept / synthesis page compiler over the memory
└── worker Background job runner
```
## Appendix B — links
- Product: [waggle-os.ai](https://waggle-os.ai)
- Enterprise: [www.kvark.ai](https://www.kvark.ai)
- License server: `https://license.waggle-os.ai/validate`
- SaaS cloud: `https://cloud.waggle-os.ai`
- Canonical architecture doc: `CLAUDE.md` in repo root
*Full feature/metric numbers + detailed memory-architecture diagrams will be folded in when the overnight research agents return their inventories. This draft covers the structural story end-to-end.*

355
docs/research/07-skills-connectors-strategy.md Normal file
View File

@@ -0,0 +1,355 @@
# Skills + Connectors — Shipping Strategy, User Extension, Self-Evolution
**Author:** Waggle OS research series (7 of 7)
**Drafted:** 2026-04-15 (overnight batch)
**Scope:** Strategy for Waggle's two main extension points — Skills (prompt modules the agent loads on demand) and Connectors (MCP-style integrations to external systems). What ships at launch, how users extend, how the self-evolving loop works, and how we explain all this to non-technical users.
---
## TL;DR
**Skills and Connectors are the monetization trigger** per the tier strategy memory: Memory + Harvest are free forever (lock-in moat), but Skills and Connectors are what escalates a user from Free → Pro → Teams. So the shipping strategy matters.
**The shipping posture:** ship lean (~20 starter skills, ~12 starter native connectors, the MCP catalog for the 100+ long tail), make the creation loop visible so users *see* that skills are learnable and sharable, then let the self-evolving loop (GEPA + skill promotion + auto-extract + retirement) quietly make everything better over time.
**The explanation challenge:** most users don't know what "skill" or "MCP connector" means. The UX must translate these concepts into vocabulary the persona understands — for the Founder, a skill is "a saved way of working"; for the Developer, it's "composable prompt modules"; for the IT Admin, it's "a governed AI capability with an audit trail." One underlying system, multiple surface languages.
---
## 1. Strategic framing
### 1.1 Why Skills matter
A Skill in Waggle is a SKILL.md file with YAML frontmatter (name, description, scope, permissions) plus a body that explains *how to do a specific thing well.* The agent loads the relevant skill at the right moment based on the user's request, giving it compact, expert-authored guidance.
Why this matters strategically:
- **Reuse** — the user's best prompts get captured once, used forever
- **Sharing** — promotable scope (personal → workspace → team → enterprise) means a senior's skill can elevate the whole org
- **Auto-evolution** — Waggle's extractor → generator pipeline (Gap A shipped) auto-creates skills from repeated workflows without the user writing anything
- **Governance** — compliance officers love them: every skill has an author, a version, a scope, an audit trail
- **Monetization** — a skill marketplace is a natural Pro-tier upsell
### 1.2 Why Connectors matter
A Connector in Waggle lets an agent act on external systems (read your Gmail, post to Slack, query HubSpot). Waggle supports two styles:
- **Native connectors** (`packages/agent/src/connectors/*`) — TypeScript classes implementing `WaggleConnector`, shipped in the binary
- **MCP connectors** — Model Context Protocol servers, discoverable via the MCP catalog (`packages/shared/src/mcp-catalog.ts`), runnable as subprocesses or remote endpoints
Native is for the 10-15 hero integrations (Gmail, Slack, GitHub, etc.); MCP is for the long tail (the 100+ catalog entries). Both surface identically to the agent.
Why this matters:
- **Table stakes** for enterprise buyers — no connectors, no deal
- **Action** is what makes AI useful beyond chat; connectors are how agents *do things*
- **Ecosystem** — MCP is the industry's emerging standard; Waggle being a good MCP citizen is a defensive moat
- **Tier trigger** — "free gets 3 native + 10 MCP; Pro gets all 12 native + full MCP catalog" converts
---
## 2. Skills — shipping strategy
### 2.1 The starter pack (~20 skills)
A deliberate, curated set that covers the most common workflows across personas. Each skill is maintained by Waggle, versioned, and distributed with the binary.
**Writing** (4 skills)
- `long-form-writer` — produces essays, reports, blog posts with structure
- `email-drafter` — short, contextual emails in the user's voice (pulls from memory)
- `slide-deck-outliner` → pairs with `generate_docx` / `powerpoint-automation`
- `style-consistency` — maintains voice across a multi-doc project
**Coding** (4 skills)
- `tdd-workflow` — enforces write-test-first (already exists in skills library as `tdd`)
- `code-reviewer` — bug/security/style review on a diff
- `refactor-planner` — proposes refactor steps with tests
- `commit-message` — conventional commits from a diff
**Research** (3 skills)
- `deep-research` — multi-source synthesis with citations
- `market-scan` — competitive landscape with sources
- `literature-review` — academic paper synthesis
**Planning** (3 skills)
- `break-down-task` — decompose to checklist
- `prioritize` — RICE/MoSCoW framework application
- `retrospective` — structured post-mortem
**Analysis** (3 skills)
- `data-exploration` — profile + describe a dataset
- `statistical-sanity` — check distributions + outliers before a claim
- `chart-recommender` — pick the right viz for the question
**Professional** (3 skills)
- `stakeholder-update` — tailored update in the recipient's context
- `meeting-summary` — from transcript or notes
- `1:1-prep` — given a direct report's recent frames, draft talking points
### 2.2 Explanation to the user — per persona
The same underlying system, different vocabulary:
| Persona | How "skill" is introduced |
|---|---|
| Product owner | "Saved ways of working. When you ask me to draft a stakeholder update, I use your `stakeholder-update` skill so the output matches your voice every time." |
| Knowledge worker | "Expert-approved instructions your AI follows. Your legal team can author the `nda-review` skill once; everyone on the team uses it consistently." |
| Developer | "Composable prompt modules. They live in `~/.waggle/skills/*.md` — YAML frontmatter + markdown body. Git-manageable, version-controlled, promotable." |
| Researcher | "Structured method cards. Each skill encodes a methodology (e.g., `literature-review` applies a consistent synthesis framework)." |
| Founder | "Your competitive advantage, packaged. Your best prompts become your team's default way of working." |
| IT admin | "Governed AI capabilities with scope gates (personal / workspace / team / enterprise), audit trails, and permission boundaries. Promote only after review." |
| Prosumer | "Little helpers that know your style. The `email-drafter` skill learns your voice over time." |
One system, seven onboarding scripts. This should be reflected in persona-aware tooltips at skill-discovery points in the UI.
### 2.3 Tier gates
- **Free tier** — all 20 starter skills, personal scope only. Can create custom skills (unlimited personal). Cannot promote.
- **Pro tier ($19)** — everything free + marketplace access + promotion to workspace scope
- **Teams tier ($49/seat)** — + team scope promotion + shared skill library (requires `teamSkillLibrary` capability — already gated in `tiers.ts`)
- **Enterprise / KVARK** — + enterprise scope + SSO + attestation workflow for skill promotion (two-person approval, audit log)
### 2.4 The creation loop (how users extend)
Three pathways for users to create a new skill:
**Pathway 1: Write one from scratch.** `create_skill` tool or UI form. For users who know exactly what they want.
**Pathway 2: Capture from a conversation.** "Turn this into a skill" button after a successful interaction. Waggle runs `generateSkillMarkdown` with the conversation as input → produces a draft SKILL.md → user edits/saves. **This is the fastest path for most users.**
**Pathway 3: Auto-extract (Gap A, shipped).** When the same 3+ tool sequence repeats 2+ times in a session, the agent *offers* to save a skill: "I noticed you web-search → web-fetch → save-memory often. Save as a skill called 'research-workflow'?" User clicks yes or dismisses.
All three converge on the same SKILL.md artifact with the same frontmatter schema. Pathway 3 is the magic moment — showcase it in onboarding.
### 2.5 Promotion flow (Gap E, shipped)
Once a skill is working well at personal scope, the user promotes it up the ladder:
```
personal → workspace → team → enterprise
```
Each step is one rung — no jumps, no demotions. Tier gated (team requires `teamSkillLibrary`, enterprise requires ENTERPRISE tier). Frontmatter tracks the `promoted_from` history. A `skill_promotion` improvement signal is recorded per promotion for telemetry + eval.
**UX for promotion:**
- Right-click a skill → "Promote to workspace" → confirm
- Show the history: "This skill was promoted by Alice from personal on 2026-04-01, then promoted to team by Bob on 2026-04-12"
- On team promotion, show a diff of what team members will see
### 2.6 Self-evolution loop
Waggle's GEPA + EvolveSchema integration runs continuously over the most-used skills:
1. Every chat turn records an execution trace (`packages/core/src/mind/execution-traces.ts`)
2. The Evolution orchestrator (`evolution-orchestrator.ts`) periodically selects skills with ≥10 traces and runs a mini-evolution cycle against them
3. Candidate improved prompts are judged by LLM-as-judge (`judge.ts`) on a held-out slice of traces
4. If a candidate passes constraint gates (`evolution-gates.ts`) AND beats the incumbent by a meaningful margin, it's proposed to the user
5. User sees: "`stakeholder-update` skill has an improved version. Before/after comparison + score delta. Accept / reject / keep both."
6. Accept → deploy (the live prompt overrides file is updated atomically); the user's version of the skill evolves
**UX principle:** the user is always in the loop. We don't auto-deploy evolved prompts without consent. Surface the improvement *and* the reasoning ("here's why the new version scored higher — it better extracts context from memory before drafting").
### 2.7 Decay (Gap F, shipped)
Skills not used for ≥90 days get archived to `~/.waggle/skills-archive/` — recoverable, never deleted. Keeps the active library lean. User sees: "Archived 3 unused skills. See archive | Restore all."
---
## 3. Connectors — shipping strategy
### 3.1 Native starter pack (12 hero connectors)
Shipped in the binary, wired to per-connector OAuth, enforce per-action permissions, full audit trail.
| Category | Connector | Why priority |
|---|---|---|
| Email | **Gmail** | 60%+ of workforce; universal primary |
| Email | **Outlook** | enterprise |
| Calendar | **Google Calendar** | reads for context, writes for scheduling |
| Calendar | **Outlook Calendar** | enterprise |
| Chat | **Slack** | team comms |
| Chat | **Microsoft Teams** | enterprise comms |
| Task | **Linear** | dev-heavy teams |
| Task | **Asana / Trello / Monday** | broad team tracking |
| Docs | **Notion** | power users |
| Docs | **Google Drive** | universal |
| Code | **GitHub** | dev teams + code review |
| Sales | **HubSpot** | founder / sales persona |
(MCP catalog covers 100+ more: Jira, Stripe, Salesforce, Zendesk, Discord, Figma, Intercom, Confluence, etc.)
### 3.2 Explanation to the user — per persona
| Persona | How "connector" is introduced |
|---|---|
| Product owner | "Lets your assistant act on your behalf in the tools you already use. Connect Gmail and I can draft replies in threads you care about." |
| Knowledge worker | "Permissioned integrations. Each one is opt-in, scoped to specific actions, and every action is logged for your audit trail." |
| Developer | "MCP servers + native SDK. Write your own against the `WaggleConnector` interface in `@waggle/agent`." |
| Researcher | "Data access to your papers, notes, and sources — bring your research surface into one mind." |
| Founder | "Connect the 5 tools you already pay for. Your AI works across them instead of forcing you to copy-paste." |
| IT admin | "Governed integrations with per-connector access control, audit logging, optional enterprise gateway routing. Marketplace can be whitelisted." |
| Prosumer | "Plug your AI into your life. Let it see your calendar, your email, your creative apps." |
### 3.3 Tier gates
- **Free** — 3 native connectors (of user's choice from the 12) + 10 MCP servers
- **Pro** — all 12 native + full MCP catalog + custom connector SDK
- **Teams** — + team-shared connector credentials (one admin connects, team benefits)
- **Enterprise / KVARK** — + enterprise gateway (all connector traffic routes through your firewall / SSO), connector whitelisting, forced encrypted credential vault, connector usage analytics
### 3.4 Custom connector SDK
Power users and enterprise teams write their own. The SDK is already in `packages/agent/src/connector-sdk.ts`:
```
WaggleConnector {
id, name, description, actions[],
authenticate(), executeAction(action, params)
}
```
Documentation + template repo: `waggle-os/connector-template` (suggested Q3 2026 deliverable). Users should be able to scaffold `npm create @waggle/connector my-tool` and publish to the marketplace in under 30 minutes.
### 3.5 MCP catalog — the long tail
Per memory: 148+ MCP servers catalogued in `packages/shared/src/mcp-catalog.ts` with simple-icons + dedup guard. This is Waggle's answer to "do you support X?" — almost always yes, via MCP.
UX for discovery:
- Settings → Connectors → "Add a connector" → search bar over full catalog
- Categories: Productivity / CRM / Dev Tools / File Storage / Chat / Finance / Design / Marketing / etc.
- For each: one-click "Install" that pulls the MCP server, sets up config, prompts for auth
### 3.6 Approval gates
Every connector action fires through `packages/server/src/local/approval-grants.ts`. By autonomy level:
- **Normal mode** — every destructive/external action asks user confirmation
- **Trusted mode** — one-click "trust this skill for this session"
- **YOLO mode** — user accepts all (for speed; not recommended in regulated workspaces)
Audit trail records: who approved what, when, with what result. This is the receipt-keeping layer that serves the IT Admin and Knowledge Worker personas.
### 3.7 Self-evolution for connectors?
Evolution today operates on *prompts* (skills, personas, behavioral spec). Connectors are *code*. But there's a related loop: **capability-acquisition** (`packages/agent/src/capability-acquisition.ts`) detects when the agent couldn't do something because no connector covered it. A `capability_gap` improvement signal fires, and the user sees: "I couldn't do X because we're not connected to Y. Want to install the MCP server for Y?"
This is the connector-side analog of skill auto-extract. It makes the gap visible instead of letting it silently fail.
---
## 4. Marketplace strategy
### 4.1 Positioning
The Waggle Skill + Connector Marketplace is the **last mile of extension**. Starter skills ship for free; great third-party skills (and connectors) are sold or shared through the marketplace.
Model options (pick one, recommend A):
- **A. Free marketplace + attribution only.** All skills/connectors are free to install. Authors are surfaced with reputation scores ("installed by 4,200 users, 4.8★"). Revenue comes from Pro/Teams upsell, not marketplace take. Simpler, faster to launch, larger ecosystem. **Recommended.**
- **B. Freemium marketplace.** Free skills + paid skills (author sets price, Waggle takes 20%). More revenue upside but requires Stripe Connect, KYC for authors, dispute handling, customer support. Slower to ship.
- **C. Enterprise-only marketplace.** Only Teams/Enterprise tier sees third-party skills (free or paid). Reduces moderation burden + keeps the free tier safe. Hybrid with A works well.
### 4.2 Quality control
- Every submitted skill goes through a trust assessment (`trust-model.ts` already shipped — assesses permissions, code patterns, author history)
- Authors verified (GitHub OAuth baseline; enterprise-verified tier later)
- Report-abuse flow with fast remove
- Version pinning — when a skill updates, users opt in to new version rather than getting silent changes
### 4.3 Starter seeding
At launch, Waggle authors 20-40 "showcase" skills beyond the 20-skill starter pack. These demonstrate what's possible and set the quality bar.
---
## 5. Explanation assets to build
Every explanation needs an artifact to back it. Propose the following:
| Asset | Audience | Purpose |
|---|---|---|
| 3-min "What are skills?" video | Product owner, Founder, Prosumer | Set the mental model |
| Developer documentation | Developer, IT admin | Custom connector SDK + skill frontmatter spec |
| Governance white paper | IT admin, Compliance officer | How skill/connector promotion + audit works for regulated workspaces |
| Evolution explainer | Researcher, Product owner | How self-evolution works + how to opt out |
| Per-persona onboarding scripts | All 7 | Tooltip text, empty states, success messages |
| Marketplace author guide | Third-party devs | How to publish a skill and get distribution |
---
## 6. Phased action plan
### Phase 1 (2 weeks) — launch the starter pack
- Finalize the 20 starter skills (content + tests that each works end-to-end)
- Wire the 12 native connectors with OAuth flows
- Write per-persona onboarding copy
- Publish the 3-min explainer video
### Phase 2 (2 weeks) — creation loop polish
- "Turn this into a skill" button in chat after successful interactions
- Auto-extract notification UX (Gap A is shipped — surface the suggestion)
- Promotion dialog with diff preview
- Skill retirement notification (Gap F shipped — surface the weekly summary)
### Phase 3 (3 weeks) — marketplace v1
- Browse UI (categories, search, install button)
- Author profile + reputation
- Version management + opt-in updates
- Trust assessment surfaced on every install ("This skill requests: network access, write-file. Author: GitHub-verified.")
### Phase 4 (ongoing) — evolution visibility
- Settings → Evolution tab showing skills-pending-improvement
- Before/after preview UI
- Accept / reject / split-test workflow
- "Evolution statistics" dashboard (how many skills evolved, average score lift)
### Phase 5 (Q3+) — ecosystem
- Custom connector SDK docs + templates + `npm create`
- Developer portal (api.waggle-os.ai) for marketplace authors
- Enterprise connector gateway for KVARK
- Possible paid-skill monetization (Model B) if ecosystem demonstrates demand
---
## 7. Metrics
- **Skill adoption rate** — % of active users with ≥3 skills installed
- **Custom skill creation rate** — % creating ≥1 custom skill in first 30 days
- **Connector attach rate** — % connecting ≥1 connector in first session
- **Promotion rate** — monthly % of personal skills promoted to workspace+ (indicator of team value capture)
- **Auto-extract acceptance rate** — when the agent suggests a skill, how often is it accepted (indicator of suggestion quality)
- **Evolution acceptance rate** — same for evolved prompts
- **Marketplace activation** — DAU/MAU on browse; install-to-use ratio per listing
---
## 8. Risks + mitigations
| Risk | Mitigation |
|---|---|
| Skill marketplace becomes a dump of low-quality prompts | Curation via trust assessment + featured sections + reputation |
| Users confused about what a "skill" is vs a "connector" | Per-persona explanations + unified "Extensions" nav if confusion persists |
| Enterprise IT blocks marketplace over shadow-IT concerns | Whitelisting toggle in KVARK + SSO-gated skill access |
| Self-evolution pushes a bad prompt into production | User-in-loop always; constraint gates + rollback one-click (Gap E's frontmatter tracks promoted_from for reversibility) |
| MCP catalog goes stale (servers break, move, renamed) | Automated health-check cron (Q7 MCP check from harness follow-ons memo is directly applicable) |
| Custom connector authors ship security holes | Trust assessment + sandboxing (subprocess isolation for MCP; permissions for native) + disclosure policy |
---
## 9. Open decisions for Marko
1. **Marketplace model A vs B vs hybrid** — recommend A (free marketplace, revenue from tiers). Decide before Phase 3 kicks off.
2. **3-connector cap on Free tier** — is the restriction in the right place? Alternative: all connectors free, limit by *action volume* per day. Simpler UX but harder to monetize.
3. **Auto-evolution default** — opt-in or opt-out? Privacy-conscious users may want opt-in ("evolve my prompts" toggle off by default); growth-focused tuning wants opt-out. Strong recommend: **opt-in** during onboarding with a clear explanation.
4. **Video or written explainers** — prioritization question. Video is higher-production cost but 5-10x the engagement. My take: ship written first (lives in docs/), video as a Q3 production.
5. **Enterprise gateway architecture** — does KVARK inherit all Pro connectors via pass-through, or does enterprise require explicit whitelisting of every connector? This is a KVARK engagement-specific decision.
---
## Closing
Skills and Connectors are Waggle's answer to "what can your AI do?" The answer is "anything, and the more you use it the better it gets." The shipping strategy is: small curated starter set that shows off the range, easy creation loop (3 pathways including auto-extract), clear promotion ladder, visible self-evolution, generous marketplace, per-persona explanations that meet users where they are.
**Do not ship 200 skills at launch.** Ship 20 great ones + a compelling creation loop. Users teach Waggle what they actually need, and the ecosystem grows from there.

131
docs/research/PAPER-1-CONCEPT_hive-mind-memory.md Normal file
View File

@@ -0,0 +1,131 @@
# Paper 1 — Hive-Mind: Frame-Graph Memory for Autonomous AI Agents
**Status:** CONCEPT — section skeleton + key claims. Full write-up after v2 memory evaluation runs.
**Target:** arXiv cs.AI preprint → NeurIPS / ACL workshop on AI Agents
**Authors:** Marko Markovic (Egzakta Group)
**Drafted:** 2026-04-15
---
## Thesis
Memory for AI agents should be modeled as a **temporal frame graph** — not turn-blobs, not key-value pairs, not unstructured entity graphs. The frame model (borrowed from video compression's I/P/B encoding) gives consolidation, decay, and link formation natural semantics. Combined with multi-tenant isolation and compliance-by-default auditing, this produces a memory system that is simultaneously rich, performant, enterprise-ready, and legally defensible.
---
## Section Skeleton
### 1. Abstract (~250 words)
- Four contributions: frame model, multi-mind isolation, compliance-by-default, wiki compilation
- System is deployed in production (Waggle OS desktop)
- Key numbers: [X] frames across [Y] users, [Z] harvest adapters, [W] retrieval latency
- **Placeholder:** insert real numbers after evaluation runs
### 2. Introduction
- **Position:** Four-taxonomy of memory approaches:
1. Turn-blob (naive chatbot, early ChatGPT) — high fidelity, catastrophic recall-cost growth
2. Key-value (mem0, OpenAI Memory) — efficient but loses context + temporal reasoning
3. Graph-augmented (Zep/Graphiti, Microsoft GraphRAG) — rich but expensive writes
4. **Frame-graph (ours)** — temporal continuity + consolidation + multi-scope + compliance
- Why a fifth approach is needed: enterprise deployments require isolation + audit trails that none of the above provide
### 3. The Frame Model
- **I/P/B types** from video compression:
- I-frame: standalone fact (full context, no dependencies)
- P-frame: predicted/derivative (references one I-frame, delta encoding)
- B-frame: bidirectional (links two frames, captures relationships)
- **Formal definition:** 5 fields (id, gop_id, frame_type, content, importance, source, confidence, created_at)
- **Importance levels:** critical > important > normal > temporary > deprecated
- **Source provenance:** user_stated, tool_verified, agent_inferred → confidence derivation
- **Decay/consolidation semantics:** importance-aware retention, compaction within GOPs
- **Write-path dedup:** 3-level (exact match → normalized string → embedding cosine similarity > 0.95)
- **Write-path contradiction detection:** detects conflicting saves, emits correction signals
### 4. Multi-Mind Isolation
- **Scope hierarchy:** personal → workspace → team → enterprise
- Each scope is a separate `.mind` SQLite file
- **Cross-mind reads** require explicit approval gates (ALWAYS_CONFIRM list)
- **Write routing:** B1 guardrail detects workspace-specific content being saved to personal mind, silently redirects
- **Skill promotion chain:** personal → workspace → team → enterprise with tier gates (TEAMS required for team scope)
- **MultiMindCache:** LRU eviction, path-traversal guard, lifecycle management
### 5. Hybrid Retrieval
- **FTS5 + sqlite-vec fused via Reciprocal Rank Fusion (RRF_K=60)**
- 4 scoring components: recency, importance, access frequency, connection density
- 4 scoring profiles with explicit weight vectors (balanced/recent/important/connected)
- Knowledge graph overlay: entity-relation graph with bitemporal validity
- Temporal filtering (since/until params)
- **Placeholder:** retrieval quality benchmarks (precision@k, MRR) after evaluation
### 6. Wiki Compilation
- Entity pages, concept pages, synthesis pages — generated from frame corpus
- Incremental compilation (only recompile affected pages on new frame)
- Health reports (coverage, freshness, gaps)
- **Placeholder:** wiki quality metrics after evaluation
### 7. Compliance-by-Default
- EU AI Act article mapping:
- Art. 12: Automatic event logging (inputs + outputs, not just token counts)
- Art. 14: Human oversight (approval/denial actions)
- Art. 19: Log retention (180-day minimum, system-age-aware check)
- Art. 26: Deployer monitoring (cost, tools, model ID, persona)
- Art. 50: Model transparency (model inventory disclosed)
- Append-only audit log with DDL-level triggers (BEFORE DELETE / BEFORE UPDATE → RAISE ABORT)
- Audit report PDF generation (boardroom-grade, pdfmake)
- GDPR Art. 17 intersection: pseudonymize-and-tombstone design for erasure requests
### 8. Harvest Pipeline
- 11 adapters: ChatGPT, Claude, Claude Code, Gemini, Perplexity, Markdown, Plaintext, PDF, URL, Universal
- 4-pass distillation: classify → extract → synthesize → dedup
- Model tiering: cheap model for classification, accurate model for extraction/synthesis
- Injection scan at pipeline entry (scanForInjection on all imported content)
### 9. Evaluation (TO BE COMPLETED)
- **Retrieval quality:** precision@k, MRR, recall against ground-truth frames
- **Write-path correctness:** dedup accuracy, contradiction detection precision/recall
- **Compliance completeness:** audit report vs manual Art. 12 checklist
- **Harvest quality:** frame quality scores across adapter types
- **Performance:** frames/sec ingestion, search latency at 10K/100K/1M frames
- **Comparison:** vs mem0, vs Letta/MemGPT, vs raw vector search
### 10. Related Work
- mem0 (48k stars) — API-first key-value, no multi-tenant, no compliance
- Letta/MemGPT (13k stars) — virtual context management, no frame model
- Zep/Graphiti (24.5k stars) — knowledge graph, no compliance-by-default
- Microsoft GraphRAG (31k stars) — community detection, batch-oriented, no agent integration
- Mastra (22k stars) — Apache 2.0 + ee/ pattern (we adopt same licensing strategy)
- Cognee (14.2k stars) — knowledge pipeline, no multi-mind
### 11. Limitations
- Single-node SQLite (no distributed scaling yet)
- LLM-dependent for entity extraction and wiki synthesis
- English-only content detection patterns
- v1 user base (production but limited scale data)
### 12. Conclusion
- Frame-graph memory is a viable fourth position
- Compliance-by-default is a differentiator, not an afterthought
- Open-source plan: `hive-mind` Apache 2.0 for memory primitives
---
## Key Claims (must be defensible with data)
| # | Claim | Evidence needed | Status |
|---|-------|----------------|--------|
| 1 | I/P/B frame model enables better consolidation than flat key-value | Comparative eval vs mem0 | PLANNED |
| 2 | Multi-mind isolation prevents cross-workspace leakage | Security test suite (already passing) | DONE |
| 3 | Hybrid search (FTS5+vec RRF) outperforms pure vector search | Retrieval benchmarks | PLANNED |
| 4 | Write-path contradiction detection catches conflicting saves | Unit tests (already passing) | DONE |
| 5 | Compliance report passes manual Art. 12 checklist | Audit comparison | PLANNED |
| 6 | Harvest pipeline ingests 1000 conversations in <5 min | Performance benchmark | PLANNED |
---
## What Needs to Happen Before Full Write-Up
1. **Run retrieval benchmarks** — precision@k, MRR on a curated test set
2. **Run performance benchmarks** — ingestion rate, search latency at scale
3. **Run comparative eval** — mem0 vs Hive-Mind on the same dataset
4. **Collect production metrics** — real user frame counts, search latencies
5. **External reviewer pass** — one ML peer reads the methods section

174
docs/research/PAPER-2-CONCEPT_gepa-evolution.md Normal file
View File

@@ -0,0 +1,174 @@
# Paper 2 — Closed-Loop Prompt Evolution: Can Smaller Models Match Flagships?
**Status:** CONCEPT — section skeleton + key claims. Full write-up after v2 experiment ($200 budget approved).
**Target:** arXiv cs.AI preprint → ICLR / NeurIPS workshop
**Authors:** Marko Markovic (Egzakta Group)
**Drafted:** 2026-04-15
---
## Thesis
A closed-loop prompt-evolution system — combining GEPA (Genetic-Pareto prompt optimization) with schema-level behavioral constraints — can produce prompts for smaller open-weight models (Gemma 4 31B) that match or exceed the performance of flagship commercial models (Opus 4.6) on domain-specific tasks, at 10-50x lower inference cost.
---
## Section Skeleton
### 1. Abstract (~250 words)
- Problem: organizations pay flagship-model prices for tasks where a tuned smaller model suffices
- Approach: closed-loop system that traces agent behavior → builds eval datasets → evolves prompts → gates deployments → deploys to production
- v1 result: 108.8% C/A ratio (Gemma 4 + evolved prompt vs raw Opus 4.6, n=10, 4 blind judges)
- v2 design: n=60, 3 domains, 4-judge multi-vendor pool, bootstrap CI, pre-committed negative-result publication
- **Placeholder:** insert v2 numbers after experiment
### 2. Introduction
- The cost problem: flagship models at $15-75/M tokens for tasks that don't require frontier reasoning
- The prompt-engineering bottleneck: human prompt engineers are expensive and iterate slowly
- Our approach: automated prompt evolution with production-grade safety gates
- **Key insight:** the evolution system runs on production traces, not synthetic data — the prompts are evolved against real user behavior
### 3. Background and Related Work
- **GEPA** (Agrawal et al., arXiv:2507.19457, ICLR 2026 Oral): Genetic-Pareto prompt evolution, beats RL (GRPO) by +6% avg / +20% max with ≤35x fewer rollouts, beats MIPROv2 by >10%. Integrated into DSPy 3.0.
- **ACE** (Zhang et al., arXiv:2510.04618, Stanford/SambaNova): Agentic Context Engineering — closest public analog to our EvolveSchema approach for behavioral-spec mutation
- **Reflection 70B** (Shumer, Sept 2024): cautionary tale — "small beats big" claims require extreme rigor
- **DSPy** (Khattab et al.): programmatic prompt optimization framework
- **TextGrad** (Yuksekgonul et al.): gradient-based prompt optimization
- Our positioning: we're not a prompt optimizer — we're a closed-loop production system that happens to use prompt evolution as one component
### 4. System Architecture
- **6-stage pipeline (ASCII diagram):**
```
Traces → Dataset → Compose → Gates → Deploy → Monitor
↑ ↓
└────────── Feedback Loop ───────────┘
```
- **TraceRecorder:** captures every agent turn (model, tokens, tools, human action)
- **EvalDatasetBuilder:** builds train/test splits from production traces
- **LLM-as-Judge:** multi-vendor blind judging with rubric
- **GEPA + EvolveSchema composition:** GEPA mutates the prompt, EvolveSchema mutates behavioral rules, feedback separation ensures they don't interfere
- **Constraint gates:** 4 categories (safety, quality, cost, behavioral) with configurable thresholds
- **Evolution deploy:** atomic persona override + behavioral-spec override with backup/rollback
- **Boot-time merge:** accepted overrides merge into active behavioral spec on server start
### 5. The Evolution Pipeline in Detail
#### 5.1 Trace Collection
- Automatic via `wireAgentLoopCallbacks` — every tool call, every model response
- Stored in `execution_traces` table with session scoping
#### 5.2 Dataset Construction
- Group traces by tool patterns (e.g., "save_memory → search_memory" sequences)
- Score by outcome quality (human approval rate, task completion)
- Split: 50/50 train/test with reproducible seed
#### 5.3 GEPA Iterative Evolution
- Population of prompt candidates
- Pareto frontier across objectives (quality, cost, length)
- Iteration cap: 500 iterations OR cost ceiling
- Early-abort: plateau detection (50 consecutive iterations below threshold)
#### 5.4 EvolveSchema Integration
- Behavioral-spec mutations: add/remove/modify rules
- Feedback separation: GEPA feedback on prompt quality vs EvolveSchema feedback on behavioral compliance
- Compose: merge prompt mutations and schema mutations into a single candidate
#### 5.5 Constraint Gates
- **Safety gate:** injection scan score must not degrade
- **Quality gate:** judge scores must meet minimum threshold
- **Cost gate:** token count must not exceed budget
- **Behavioral gate:** all critical behavioral rules must pass
#### 5.6 Deploy and Monitor
- Atomic file writes with backup/rollback
- Hot-reload via event emission (`behavioral-spec:reloaded`)
- Production monitoring via the same TraceRecorder
### 6. Experimental Design
#### 6.1 v1 (completed, preliminary)
- **Setup:** 10 coder questions, Gemma 4 31B + Waggle-evolved prompt vs raw Opus 4.6
- **Judges:** 4 blind judges (Opus, Sonnet, Haiku, GPT-4o)
- **Result:** C/A ratio 108.8% — evolved-Gemma beat raw-Opus on per-judge mean
- **Limitations:** n=10, one domain, eval=train, Opus-as-judge bias, no CI
- **Placeholder:** v1 results table
#### 6.2 v2 (designed, budget approved, not yet run)
- **Dataset:** 60 examples (30 train + 30 test), 3 domains (writer/analyst/researcher), 10 per domain per stratum
- **Arms:**
- A: raw Opus 4.6 (flagship reference)
- B: Gemma 4 + human-engineered prompt (100 tokens)
- C: Gemma 4 + GEPA-evolved prompt (from training run)
- **Judges:** 4-judge multi-vendor pool:
- Sonnet 4.6 (Anthropic)
- Haiku 4.5 (Anthropic)
- GPT-5 (OpenAI)
- Gemini 2.5 Pro (Google)
- **Rotation:** each test example judged by all 4, randomized arm-letter assignment
- **Statistics:** bootstrap 95% CI + permutation test at alpha=0.05
- **Hypotheses:**
- H1: C/A >= 0.95 for 2+ of 3 domains
- H2: C/A >= 1.00 for 1+ domain (replicates v1 headline)
- H3: C/A < 0.90 for 2+ domains → publish negative (pre-committed)
- **Budget:** $200 hard cap ($80 training + $80 evaluation + $40 buffer)
- **Placeholder:** v2 results table, per-domain breakdown, judge agreement, CI intervals
### 7. Results (TO BE COMPLETED)
- **Placeholder:** v2 primary results table
- **Placeholder:** per-domain breakdown
- **Placeholder:** inter-judge agreement (Krippendorff's alpha)
- **Placeholder:** bootstrap CI visualization
- **Placeholder:** training curve (score per iteration)
- **Placeholder:** prompt length analysis (tokens added vs quality gained)
- **Placeholder:** cost analysis (evolution cost vs inference savings)
### 8. Discussion
- What the results mean for the cost-quality tradeoff
- When evolution works (domain-specific, well-defined tasks) vs when it doesn't (open-ended reasoning)
- The role of judge diversity in credibility
- Production implications: how often to re-evolve, drift detection
### 9. Limitations
- v1 sample size (n=10) is underpowered
- Single-organization traces (Waggle users, not a general population)
- GEPA is domain-specific — global prompts may not generalize across all task families
- Judge-based evaluation inherits judge biases
- No human evaluation (judges are all LLMs)
- Evolution cost is amortized but non-trivial ($80-150 per run)
### 10. Reproducibility
- Split seed committed to repo
- GEPA config, judge prompts, dataset — all published
- Raw results JSON available
- Cost tracking via CostTracker
### 11. Conclusion
- Closed-loop evolution is a viable alternative to paying flagship prices
- The system is production-grade, not a research prototype
- Pre-committed negative-result publication builds trust
---
## Key Claims (must be defensible with data)
| # | Claim | Evidence needed | Status |
|---|-------|----------------|--------|
| 1 | Evolved Gemma 4 matches Opus 4.6 (C/A >= 0.95) | v2 experiment (30 test, 4 judges) | PLANNED |
| 2 | Evolution cost ($80-150) is amortized over thousands of inferences | Cost analysis | PLANNED |
| 3 | Multi-vendor judges agree on ranking (alpha > 0.6) | Inter-judge agreement stats | PLANNED |
| 4 | Constraint gates prevent quality regressions | Gate pass/fail rates from training | PLANNED |
| 5 | System handles negative results gracefully | Pre-committed H3 publication | DESIGNED |
| 6 | Evolution converges within 500 iterations | Training curve analysis | PLANNED |
---
## What Needs to Happen Before Full Write-Up
1. **Pre-flight checklist** — verify API keys for all 4 judges + Gemma endpoint
2. **Write-up skeleton** — abstract + methods with [X.X%] placeholders (this document)
3. **Run v2 training** — GEPA on 30 train examples ($80 budget)
4. **Run v2 evaluation** — 30 test x 3 arms x 4 judges ($80 budget)
5. **Statistical analysis** — bootstrap CI, permutation test, inter-judge agreement
6. **Fill in results** — tables, charts, per-domain breakdown
7. **External reviewer pass** — one ML peer reads methods + results
8. **Publish** — arXiv preprint + reproducibility repo

215
docs/research/README.md Normal file
View File

@@ -0,0 +1,215 @@
# Waggle OS — Deep Research Series (2026-04-15)
Seven reports drafted overnight per Marko's request for strategic deep-research across Waggle's biggest near-term questions. Plus this index.
Each report stands alone; read in any order. Reading suggestion: start with **06 (product overview)** for the big picture, then pick based on what you're working on.
---
## The reports
### 01 — Open-source memory packaging strategy
**[`01-oss-memory-packaging-strategy.md`](01-oss-memory-packaging-strategy.md)**
Strategy + action plan for releasing Waggle's memory system as `hive-mind` (Apache 2.0). What goes OSS (primitives: FrameStore, HybridSearch, KnowledgeGraph, harvest adapters, wiki compiler skeleton) and what stays proprietary (agent runtime, evolution stack, compliance PDF, marketplace, KVARK). License rationale with historical context (Elastic/HashiCorp/MongoDB). Positioning vs mem0/Letta/Zep/Mastra. 12-week launch sprint plan.
**Bottom line:** Apache 2.0, separate repo, launch 30-60 days after the v2 hypothesis reveal to compound attention. Publish `hive-mind` as "the ONLY OSS memory system designed around compliance-by-default with a wiki-compilation layer and a provable temporal frame model."
---
### 02 — Memory system scientific paper draft
**[`02-memory-system-scientific-draft.md`](02-memory-system-scientific-draft.md)**
arXiv-style working paper on the memory architecture: I/P/B frame model (borrowed from video compression), multi-mind isolation, write-path contradiction detection, skill promotion, compliance-by-default interaction logging, wiki compilation. Includes abstract, methods, formal properties, related work, limitations, illustrative SQL schema and TypeScript API sketch.
**Bottom line:** Publishable architecture paper. Empirical numbers pending benchmark suite + v2 hypothesis run. Structural narrative reviewable now.
---
### 03 — Memory harvesting strategy + UX
**[`03-memory-harvesting-strategy.md`](03-memory-harvesting-strategy.md)**
Harvest is Waggle's lock-in moat (memory + harvest free forever per tier strategy). Six UX principles (harvest-first onboarding / privacy is the headline / dedup is a feature / progress is tactile / recovery is resumable / identity auto-populate). First-time-user journey with emotional beat at 8 minutes. Six-phase action plan. Competitive positioning vs ChatGPT/Notion/mem0/Letta/Rewind. Metrics that should drive priority.
**Bottom line:** Pipeline is strong (9 adapters + perplexity shipped = 11 production + universal/pdf/url/md/txt baseline). UX is the gap. Phase A (first-run hook) is 1 week and would measurably move "new user hooked in first session" needle. 5 open decisions for Marko at the end.
---
### 04 — GEPA public reveal strategy
**[`04-gepa-public-reveal-strategy.md`](04-gepa-public-reveal-strategy.md)**
Publication strategy for the v2 hypothesis result. Publish arXiv-style research note FIRST (earns trust), then Twitter thread + LinkedIn long-form + HN post SECOND (captures virality). Reproducibility repo non-negotiable on day one. Audience-by-audience planning (researchers / technical buyers / prompt engineers / skeptics) with ready-made objection responses. Warm-list protocol (72h pre-publication). H₁/H₂/H₃ verdict paths including the pre-committed negative-result path (Q5). 14-day rollout timeline.
**Bottom line:** This is Waggle's highest-leverage single moment. Get it right → year of inbound + KVARK lubricant. Get it wrong → 12-month credibility rebuild. The difference is almost entirely about *credibility signals*, not the result itself (which is defensible).
---
### 05 — User personas voice-of-customer
**[`05-user-personas-ai-os.md`](05-user-personas-ai-os.md)**
Seven archetypal users with jobs-to-be-done, pain points, wants, pricing tolerance, killer feature, and Waggle tier/feature fit per persona: **product owner** (Marko archetype) / **knowledge worker** (legal/finance/HR) / **developer** / **researcher** / **founder** / **IT admin** / **prosumer**. Cross-persona heat map (memory is universal; compliance top-3; local-first matters for 5/7). Roadmap item → benefiting-persona heat map.
**Bottom line:** Universal want across all 7 is persistent memory. Waggle's tier ladder maps cleanly to persona distribution. One gap: current onboarding doesn't serve the consumer prosumer who wants "AI that remembers my LIFE, not my work." Billboard message candidate: *"Your AI remembers. Your data stays yours. Your compliance trail writes itself."*
---
### 06 — Waggle OS product overview
**[`06-waggle-os-product-overview.md`](06-waggle-os-product-overview.md)**
Comprehensive product overview. One-sentence pitches per audience. Verified architecture (React 19 + Tauri 2.0 + Fastify sidecar + SQLite/sqlite-vec). Memory stack deep-dive (9 components). Evolution stack deep-dive (10 files). Feature taxonomy by tier (Trial → Free → Pro → Teams → KVARK). User patterns (solo / team / enterprise). Competitive positioning with defensibility layers + disruption risks + strategic countermoves. Shipping state as of 2026-04-15. Metrics to hit. One-paragraph strategic thesis.
**Bottom line:** Waggle OS is the workspace-native AI agent platform with persistent memory + compliance-by-default + evolution. It's the demand-gen wedge for KVARK; the freemium tier creates lock-in; the paid tiers monetize through skills+connectors; the enterprise tier closes on governance + sovereignty. Coherent product story end-to-end.
---
### 07 — Skills + connectors shipping strategy
**[`07-skills-connectors-strategy.md`](07-skills-connectors-strategy.md)**
Shipping strategy for Waggle's two extension points. Starter skill pack (20 skills in 6 categories). Native connector hero set (12 connectors) + MCP catalog for long tail (148+). Per-persona vocabulary for explanations (same system, 7 onboarding scripts). Tier gates (Free: 3 connectors; Pro: all; Teams: shared; Enterprise: whitelisted). Three creation pathways (write / capture from chat / auto-extract). Promotion flow (Gap E). Self-evolution user-in-loop visibility. Marketplace model options (recommending free + attribution, not paid-skills). Phased 5-phase action plan.
**Bottom line:** Skills and connectors are the monetization trigger. Don't ship 200 skills at launch — ship 20 great ones + a compelling creation loop. Per-persona vocabulary matters more than feature depth.
---
## Cross-cutting themes
A few narratives show up repeatedly across the seven reports. Captured here for a unified thesis:
### Theme 1 — Memory is the moat, compliance is the multiplier
Every persona wants memory. But memory alone is Table Stakes by 2027 — every AI vendor will ship some version. **Compliance-by-default turns memory into a regulated-industry wedge** (Knowledge worker + IT admin + regulated enterprise buyers). The combination is what's defensible.
### Theme 2 — Local-first is structural, not a marketing tagline
The Tauri binary + SQLite backend + LiteLLM-for-your-own-provider architecture is *not* a SaaS product with on-prem option. It's the inverse: on-device product with optional cloud/team sync. This is structurally hard for SaaS-first competitors (OpenAI, Anthropic, Notion) to match without rebuilding their deployment model.
### Theme 3 — Evolution is the research credibility layer
The v2 hypothesis result, if it replicates, is Waggle's strongest single-point technical claim. It's also the proof-point that runs through the research paper, the KVARK sales motion, the OSS launch narrative, and the Twitter-thread story arc. One run, many uses.
### Theme 4 — The four-tier ladder maps to user reality
Trial 15d → Free forever → Pro $19 → Teams $49/seat → Enterprise/KVARK is not arbitrary. It reflects how users actually adopt AI: try → personal use → professional use → team adoption → enterprise rollout. Each tier has a persona. Each persona has a willingness-to-pay. The product-led growth arc is intact.
### Theme 5 — `hive-mind` OSS + Waggle product + KVARK enterprise is a coherent three-layer stack
Not three separate products. One coherent offering with three distribution surfaces:
- Developers + researchers get `hive-mind` (Apache 2.0) — they contribute, cite, and bring credibility
- Prosumers + small teams get Waggle (Free / Pro / Teams) — they pay subscription
- Enterprises get KVARK — they pay consultative, multi-year, high-ACV deals
Each layer makes the others more valuable. The OSS layer attracts developer talent. The product layer generates harvest corpora and feature feedback. The enterprise layer funds the whole thing and sets the compliance bar.
---
## Open decisions consolidated — what Marko needs to answer
Across all 7 reports, the following decisions are queued for Marko:
| Source | Decision |
|---|---|
| Report 1 (OSS) | `hive-mind` timing post-v2-reveal? Yes / Q2 / Q3 |
| Report 1 (OSS) | `hive-mind` repo name + Egzakta legal sign-off |
| Report 1 (OSS) | Resource: 1 eng × 1 day/week on maintenance? |
| Report 1 (OSS) | Stay Waggle-led, or consider spinout? |
| Report 3 (Harvest) | Harvest-first onboarding — v1 replacement or parallel opt-in? |
| Report 3 (Harvest) | Anonymous telemetry default — off with opt-in nudge? |
| Report 3 (Harvest) | Email/calendar as Pro-tier lock? |
| Report 4 (Reveal) | Warm list — names to pre-email 72h before publication? |
| Report 4 (Reveal) | Single-author or dual-author on the research note? |
| Report 4 (Reveal) | Podcast strategy — opportunistic vs pitched? |
| Report 7 (Skills) | Marketplace model — free+attribution (A), freemium (B), enterprise-only (C)? |
| Report 7 (Skills) | Auto-evolution default — opt-in or opt-out? |
| Hypothesis v2 | `docs/hypothesis-v2-decisions.md` — approve or amend Q1-Q5 |
Total: ~12 strategic decisions. Most are binary or small-number choices. Each unlocks a concrete execution track.
---
## What's NOT in these reports
Intentional omissions:
- **Specific financial model for OSS → Waggle → KVARK funnel conversion rates.** We don't have instrumented data yet; reports flag metrics to track, not projected numbers.
- **Named competitors' deal sizes / revenue estimates.** Public data was not used for explicit pricing comparisons; we positioned qualitatively.
- **Implementation details of the evolution stack internals.** Those are in separate docs (`docs/hypothesis-v2-plan.md`, `docs/hypothesis-v2-execution-plan.md`).
- **UI mockups / design comps.** These are strategy docs, not design docs. UI implementation follows decisions.
---
## Post-draft corrections from research agents (2026-04-15)
The 5 background research agents returned after the initial reports were written. Their findings were folded back into reports 1, 2, and 4 as factual corrections + refinements. The changes worth flagging in the morning:
### Factual corrections (reports 2 + 4)
- **GEPA = "Genetic-Pareto"** (not "Goal-driven Evolution of Prompts Algorithm" as the name was initially guessed). Paper: Agrawal et al., **arXiv:2507.19457**, **ICLR 2026 Oral**. Stanford/Databricks circle around Omar Khattab (DSPy). Beats RL baselines (GRPO) +6% avg / +20% max with ≤35× fewer rollouts; beats MIPROv2 by >10%. Integrated into **DSPy 3.0** as `dspy.GEPA`. Repo: github.com/gepa-ai/gepa.
- **"Mikhail's EvolveSchema" could not be pinned down** in public literature. Closest analog is **ACE — Agentic Context Engineering** (Zhang et al., Stanford/SambaNova, **arXiv:2510.04618**). No Mikhail on the ACE author list. Recommendation: drop the Mikhail attribution in public publications unless Marko can locate the original internal source. Cite ACE as closest analog instead.
- **Gemma 4 31B** confirmed — Google release April 2, 2026, Apache 2.0, currently Arena **#3 open model at 1452 Elo**. Waggle's v1 headline rides an existing wave.
- **Reflection 70B (Matt Shumer, Sept 2024)** is the canonical cautionary tale — framed in §5.1 of report 4 as the pattern to actively avoid (non-reproducible Twitter-first reveal).
- **Harvest adapters: 10 not 11** per code inventory (chatgpt, claude, claude-code, gemini, perplexity, markdown, plaintext, pdf, url, universal).
### OSS competitor data (report 1)
Star counts and funding rounds verified live:
| Project | Stars | License | Funding |
|---|---|---|---|
| mem0 | ~48k | Apache-2.0 | $24M Series A |
| Graphiti | ~24.5k | Apache-2.0 | VC-backed |
| GraphRAG | ~31k | MIT | Microsoft |
| Mastra | ~22k | **Apache core + source-available `ee/` (Enterprise License)** — the pattern to copy | YC |
| Cognee | ~14.2k | Apache-2.0 | Independent |
| Letta | ~13k | Apache-2.0 | $10M seed @ $70M |
5 of 6 picked Apache-2.0. None has gone to BSL/SSPL/AGPL yet — field is still in land-grab. Waggle's Apache-2.0 + potentially `ee/` pattern is maximally ecosystem-compatible.
### Waggle features inventory (affects reports 6 + 7 — not yet folded in; do so in a morning refinement pass if time permits)
Verified counts from code inventory:
- **18 canonical personas** (not 13 as CLAUDE.md §5 lists). persona-data.ts has grown since the CLAUDE.md authoritative section was last written.
- **24 built-in apps** in `apps/web/src/components/os/apps/` (MemoryApp / FilesApp / AgentsApp / MissionControlApp / CockpitApp / DashboardApp / CapabilitiesApp / ConnectorsApp / MarketplaceApp / SettingsApp / TimelineApp / EventsApp / ScheduledJobsApp / TelemetryApp / BackupApp / ApprovalsApp / TeamGovernanceApp / RoomApp / VaultApp / VoiceApp / WaggleDanceApp / UserProfileApp / WikiTab / ChatApp)
- **13 UI overlays** (OnboardingWizard, PersonaSwitcher, WorkspaceSwitcher, CreateWorkspaceDialog, SpawnAgentDialog, GlobalSearch, NotificationInbox, TrialExpiredModal, UpgradeModal, LoginBriefing, KeyboardShortcutsHelp, ContextRail, LockedFeature)
- **19 starter skills** (not 20)
- **30 native connectors** + **148 MCP catalog entries** across 14 categories
- **60+ native agent tools** total
This enriches reports 6 and 7 but doesn't invalidate their structural claims. Consider a morning refinement pass to update the hard numbers.
### Competitive landscape enrichments (affects report 6 — not yet folded in)
Specific pricing / ARR / deal-size data now available for Cohere North ($240M ARR, 2026 IPO), C3 AI ($250k pilots, $5.2B FY26 guidance), Palantir AIP (~$5.2B FY26 +61% YoY), Databricks Mosaic ($249,960 median contract), Salesforce Agentforce ($125 add-on / $550 Editions), Mistral Forge (ARR ~$400M → $1B trajectory, free-compute-on-customer-GPU — disruptive pricing anchor for KVARK). Full table in the Agent B return; consider folding into report 6 during morning pass.
**Flagged:** The "$248M → $2.63B" EU AI governance market figure from prior Egzakta memos does NOT match public 2026 figures. Public data: governance market is $2.2B (2025) → $2.54B (2026) → $11.05B (2036) at 15.8% CAGR. The Egzakta figure may be a narrower segment (AI-Act audit tooling specifically) — worth verifying the original source before using externally.
---
## How these were produced
Drafted overnight in a single autonomous session. Background research agents were dispatched for:
- OSS memory landscape (mem0, Letta, Zep, Mastra, GraphRAG, Cognee — market data, stars, funding, license positioning)
- Agentic AI competitive landscape (consumer workspace + agent frameworks + sovereign enterprise AI platforms)
- GEPA + EvolveSchema literature + prompt-evolution state of the art
- Waggle memory architecture map (code-level detail)
- Waggle features / skills / connectors inventory (code-level counts)
As of this README's drafting, those agent outputs were still in flight. Reports were written from Marko's prior memory + this session's work + architectural knowledge from CLAUDE.md. When agent outputs return, they can be folded in as section refinements in a subsequent commit. The structural arguments and strategic recommendations are robust to those refinements.
---
## Rollback + revision
- All seven reports + this index are under `docs/research/` — scope for any rollback is narrow.
- Commit boundaries are one-per-batch, see git log.
- Any report can be reverted independently without breaking the others.
---
## Acknowledgment
Built on top of Egzakta Group's long investment in Waggle + KVARK, the evolution-stack research led by Marko, and Egzakta's EUR 1.2M-contracted customer commitments that ground these reports in real enterprise sales realities rather than startup speculation. All decisions noted above are ultimately Marko's call; these docs frame the choices and commit to recommended answers.
**Date complete:** 2026-04-15 overnight batch
**Next pickup:** morning of 2026-04-16 — Marko reviews, decisions flow back into respective execution plans

1144
docs/research/waggle-hive-mind-paper.md Normal file
View File

File diff suppressed because it is too large Load Diff