waggle-os/docs/WAGGLE-MEMORY-PLUGIN-BRIEF.md

# Waggle Memory Plugin — Claude Code Agent Onboarding Brief

**Date:** 2026-04-13
**Author:** Marko Markovic + Claude (Waggle OS session)
**Purpose:** Everything a new Claude Code agent needs to build the Waggle Memory plugin for Claude Code, Claude.ai, Cowork, OpenClaw, and compatible systems.

---

## 1. What You're Building

A **persistent memory MCP server** that gives Claude Code (and any MCP-compatible AI system) the ability to:

1. **Remember** across conversations — save decisions, preferences, facts, project context
2. **Recall** with semantic search — keyword + vector hybrid search over all memories
3. **Harvest** — import conversation history from ChatGPT, Gemini, Perplexity, Cursor, Copilot exports
4. **Knowledge Graph** — automatically extract and track entities (people, projects, concepts) and their relationships
5. **Cross-workspace** — organize memories into workspaces and search across them

This is NOT a toy memory system. It's battle-tested production code from Waggle OS (a Tauri desktop app with 150+ API routes, 60+ agent tools, 305 test files) being extracted into a standalone MCP server.

---

## 2. Source Code Available

The full Waggle OS codebase has been copied to your working directory. The relevant packages:

```
packages/core/src/               ← THE MEMORY ENGINE (this is what you're wrapping)
├── mind/
│   ├── db.ts                    MindDB — SQLite wrapper (WAL mode, sqlite-vec loaded)
│   ├── schema.ts                Full schema: memory_frames, FTS5, vec table, entities, relations
│   ├── frames.ts                FrameStore — I/P/B frame CRUD, dedup, compact
│   ├── search.ts                HybridSearch — FTS5 keyword + sqlite-vec k-NN + RRF fusion
│   ├── knowledge.ts             KnowledgeGraph — entity/relation CRUD, traversal, validation
│   ├── identity.ts              IdentityLayer — user identity persistence
│   ├── awareness.ts             AwarenessLayer — active task/context tracking
│   ├── sessions.ts              SessionStore — conversation session tracking
│   ├── scoring.ts               Relevance scoring (temporal, popularity, importance, contextual)
│   ├── embeddings.ts            Embedder interface
│   ├── inprocess-embedder.ts    Zero-config embedder (Xenova/all-MiniLM-L6-v2, 384→1024 dims)
│   ├── ollama-embedder.ts       Ollama embedder (nomic-embed-text)
│   ├── api-embedder.ts          OpenAI/Voyage embedder
│   ├── embedding-provider.ts    Provider chain: InProcess → Ollama → API → Mock
│   ├── entity-normalizer.ts     Dedup entity names
│   ├── ontology.ts              Entity type validation
│   └── reconcile.ts             FTS/vec index repair
├── harvest/
│   ├── pipeline.ts              4-pass harvest: Classify → Extract → Synthesize → Dedup
│   ├── chatgpt-adapter.ts       ChatGPT export parser
│   ├── claude-adapter.ts        Claude export parser
│   ├── claude-code-adapter.ts   Claude Code session scanner
│   ├── gemini-adapter.ts        Gemini export parser
│   ├── universal-adapter.ts     Generic JSON/JSONL parser
│   ├── source-store.ts          Track registered sources + auto-sync
│   ├── dedup.ts                 SHA-256 dedup within 500-frame window
│   └── types.ts                 ImportSourceType, UniversalImportItem, etc.
├── multi-mind.ts                MultiMind (personal + workspace search)
├── multi-mind-cache.ts          LRU cache of open MindDB handles (max 20)
├── workspace-config.ts          Workspace CRUD (id, name, directory, config)
├── config.ts                    WaggleConfig (settings persistence)
├── team-sync.ts                 Push/pull frames to team server
└── file-store.ts                FileStore (Local, Linked, S3)
```

---

## 3. Architecture of the Memory System

### Frame Types (the fundamental unit)
- **I-Frame (Identity):** Base fact — "User prefers TypeScript over JavaScript"
- **P-Frame (Procedural):** Update to an I-Frame — "Also prefers functional style"
- **B-Frame (Bridging):** Links between frames

### Importance Levels
`critical | important | normal | temporary | deprecated`

### Source Provenance
`user_stated | tool_verified | agent_inferred | import | system | team_sync`

### Search Pipeline
1. **FTS5 keyword search** — stop-word filtered, OR-based
2. **sqlite-vec k-NN search** — 1024-dim embeddings
3. **RRF Fusion** (K=60) — combines keyword + vector results
4. **Relevance scoring** — 4 profiles: balanced, recent, important, connected

### Embedding Chain (probed in order)
1. **InProcess** — Xenova/all-MiniLM-L6-v2, 384→1024 dims, zero config, ~23MB download
2. **Ollama** — nomic-embed-text (needs local Ollama)
3. **Voyage/OpenAI** — API-based (needs keys)
4. **Mock** — deterministic fallback (always works)

### Knowledge Graph
- Entities: `{ type, name, properties, validFrom, validTo }`
- Relations: `{ sourceId, targetId, type, confidence, properties }`
- Traversal: BFS with depth limit
- Types: person, project, concept, organization, technology, tool, location, event

---

## 4. What the MCP Server Should Expose

### Tools (MCP tool definitions)

```
save_memory(content, importance?, workspace?)
  → Creates an I-Frame in the target workspace mind
  → Auto-indexes in FTS5 + vector
  → Returns { id, content, importance, timestamp }

recall_memory(query, limit?, workspace?, scope?)
  → HybridSearch across personal + workspace minds
  → scope: 'current' | 'personal' | 'all' | 'global'
  → Returns ranked results with scores

search_entities(query, type?, limit?)
  → Search the knowledge graph for entities
  → Returns matching entities with relations

save_entity(type, name, properties?)
  → Upsert an entity in the knowledge graph
  → Auto-dedup by normalized name

get_identity()
  → Returns the user's identity profile from IdentityLayer

set_identity(updates)
  → Update identity (name, role, preferences, etc.)

get_awareness()
  → Returns current awareness items (active tasks, context)

set_awareness(key, value, priority?, ttl?)
  → Set an awareness item (active context)

list_workspaces()
  → Returns all workspaces with stats

harvest_import(source, data_or_path)
  → Import conversations from external sources
  → source: 'chatgpt' | 'claude' | 'claude-code' | 'gemini' | 'universal'
```

### Resources (MCP resource definitions)

```
memory://personal/stats     → frame count, entity count, last session
memory://workspace/{id}     → workspace info + memory stats
memory://identity           → current identity profile
memory://awareness          → current awareness items
```

---

## 5. How to Build It

### Step 1: Create the MCP Server

The MCP server uses `@modelcontextprotocol/sdk` (the official MCP SDK). It communicates over stdio (for Claude Code) or HTTP/SSE (for web clients).

```typescript
// Structure
waggle-memory-mcp/
├── package.json
├── tsconfig.json
├── src/
│   ├── index.ts          ← Entry point, MCP server setup
│   ├── tools/
│   │   ├── memory.ts     ← save_memory, recall_memory
│   │   ├── knowledge.ts  ← search_entities, save_entity
│   │   ├── identity.ts   ← get/set identity
│   │   ├── awareness.ts  ← get/set awareness
│   │   ├── workspace.ts  ← list_workspaces
│   │   └── harvest.ts    ← harvest_import
│   ├── resources/
│   │   └── memory.ts     ← MCP resource handlers
│   └── core/
│       └── setup.ts      ← MindDB init, embedding provider, workspace manager
```

### Step 2: Depend on @waggle/core

The MCP server should import directly from the Waggle core package:

```typescript
import {
  MindDB, FrameStore, HybridSearch, KnowledgeGraph,
  IdentityLayer, AwarenessLayer, SessionStore,
  MultiMind, MultiMindCache, WorkspaceManager,
  createEmbeddingProvider,
  HarvestSourceStore, ChatGPTAdapter, ClaudeAdapter,
  ClaudeCodeAdapter, GeminiAdapter, UniversalAdapter,
} from '@waggle/core';
```

### Step 3: Data Storage

Default data directory: `~/.waggle/` (same as Waggle OS — the plugin IS the same memory)

```
~/.waggle/
├── config.json              ← settings
├── personal.mind            ← personal SQLite mind
├── workspaces/
│   ├── {id}/
│   │   ├── workspace.json   ← workspace config
│   │   ├── workspace.mind   ← workspace SQLite mind
│   │   └── files/           ← workspace files
```

**CRITICAL:** If the user also runs Waggle OS desktop, the MCP server shares the SAME data directory. This is BY DESIGN — memories saved in Claude Code appear in Waggle OS and vice versa. The SQLite WAL mode supports concurrent readers.

### Step 4: Registration in Claude Code

The user adds to `~/.claude/claude_desktop_config.json`:

```json
{
  "mcpServers": {
    "waggle-memory": {
      "command": "npx",
      "args": ["waggle-memory-mcp"],
      "env": {
        "WAGGLE_DATA_DIR": "~/.waggle"
      }
    }
  }
}
```

Or for Claude Code CLI in `~/.claude/settings.json` under `mcpServers`.

---

## 6. Key Design Decisions

### Shared data directory
The MCP server and Waggle OS desktop share `~/.waggle/`. This means:
- Memory saved in Claude Code → visible in Waggle OS
- Memory saved in Waggle OS → available to Claude Code
- The user has ONE memory, not two

### Embedding provider
The InProcess embedder works offline with zero config. It downloads a 23MB ONNX model on first run. This should be the default — no API keys needed.

### Workspace management
- The MCP server can create/list/switch workspaces
- Each workspace gets its own `.mind` SQLite file
- A "current workspace" can be set via a tool or env var
- Default: personal mind (no workspace)

### Concurrent access safety
- SQLite WAL mode allows concurrent readers
- The MCP server should open MindDB handles and keep them open (not open/close per request)
- Use MultiMindCache for LRU management of workspace handles

---

## 7. Compatibility Targets

### Primary: Claude Code
- MCP server over stdio
- `~/.claude/settings.json` registration

### Secondary: Claude.ai / Claude Desktop
- MCP server over stdio
- `~/.claude/claude_desktop_config.json` registration

### Tertiary: Compatible systems
- Any MCP-compatible host: OpenClaw, Hermes Agent, Cowork, etc.
- Same stdio protocol
- The MCP SDK handles the transport

---

## 8. What NOT to Build

- **No UI** — this is a headless MCP server. Waggle OS desktop IS the UI.
- **No LLM integration** — the host (Claude Code) provides the LLM. The MCP server is tools-only.
- **No auth** — local-only server. Auth is handled by the host.
- **No HTTP server** — stdio transport only (Claude Code standard). HTTP/SSE can be added later.
- **No team sync** — the MCP server works on the local mind only. Team features need Waggle OS.

---

## 9. SQLite Schema (from packages/core/src/mind/schema.ts)

The schema auto-creates when MindDB is instantiated. Key tables:

```sql
-- Memory frames (the fundamental unit)
memory_frames (
  id INTEGER PRIMARY KEY,
  gop_id TEXT NOT NULL REFERENCES sessions(gop_id),
  frame_type TEXT NOT NULL CHECK (frame_type IN ('I','P','B')),
  base_frame_id INTEGER REFERENCES memory_frames(id),
  t INTEGER NOT NULL DEFAULT 0,
  content TEXT NOT NULL,
  importance TEXT NOT NULL DEFAULT 'normal',
  source TEXT DEFAULT 'user_stated',
  access_count INTEGER NOT NULL DEFAULT 0,
  last_accessed TEXT,
  created_at TEXT NOT NULL DEFAULT (datetime('now')),
  content_hash TEXT
)

-- Full-text search index
memory_frames_fts USING fts5(content, content='memory_frames', content_rowid='id')

-- Vector embeddings (sqlite-vec)
memory_embeddings USING vec0(embedding float[1024])

-- Knowledge graph
knowledge_entities (id, type, name, properties JSON, valid_from, valid_to, created_at)
knowledge_relations (id, source_id, target_id, type, confidence, properties JSON, valid_from, valid_to)

-- Sessions
sessions (gop_id TEXT PK, title TEXT, summary TEXT, status TEXT, created_at, closed_at)

-- Identity
identity (key TEXT PK, value TEXT, updated_at TEXT)

-- Awareness
awareness_items (key TEXT PK, value TEXT, priority INTEGER, expires_at TEXT, created_at)

-- Harvest tracking
harvest_sources (source TEXT UNIQUE, display_name, source_path, last_synced_at, items_imported, frames_created, auto_sync, sync_interval_hours)
```

---

## 10. Testing Checklist

Before shipping:
- [ ] `save_memory` creates an I-Frame retrievable by `recall_memory`
- [ ] `recall_memory` returns relevant results with semantic search (not just exact match)
- [ ] FTS5 index stays in sync with frames
- [ ] Vector index stays in sync with frames
- [ ] Knowledge graph entities extractable from saved memories
- [ ] Harvest import works for at least: ChatGPT JSON, Claude JSON, Claude Code local scan
- [ ] Multiple workspaces can be created and searched independently
- [ ] `scope: 'global'` searches all workspaces
- [ ] InProcess embedder works with zero config
- [ ] Concurrent access doesn't corrupt (two MCP instances)
- [ ] Shared data dir with Waggle OS desktop works (read memories saved by the other)

---

## 11. Package Dependencies

From Waggle OS (copy or depend directly):
```json
{
  "dependencies": {
    "@modelcontextprotocol/sdk": "latest",
    "better-sqlite3": "^11.0.0",
    "sqlite-vec": "^0.1.0",
    "@xenova/transformers": "^2.17.0",
    "glob": "^10.0.0"
  }
}
```

If depending on `@waggle/core` directly (recommended):
```json
{
  "dependencies": {
    "@modelcontextprotocol/sdk": "latest",
    "@waggle/core": "workspace:*"
  }
}
```

---

## 12. Quick Start for the Building Agent

```bash
# 1. Read the core memory engine
cat packages/core/src/mind/db.ts          # MindDB constructor + schema
cat packages/core/src/mind/frames.ts      # FrameStore API
cat packages/core/src/mind/search.ts      # HybridSearch API
cat packages/core/src/mind/knowledge.ts   # KnowledgeGraph API

# 2. Read the embedding provider
cat packages/core/src/mind/embedding-provider.ts  # Provider chain

# 3. Read the harvest pipeline
cat packages/core/src/harvest/pipeline.ts
cat packages/core/src/harvest/types.ts

# 4. Read the workspace manager
cat packages/core/src/workspace-config.ts

# 5. Check the MCP runtime pattern (how Waggle already runs MCP servers)
cat packages/agent/src/mcp/mcp-runtime.ts
```

The entire memory engine is in `packages/core/`. You're wrapping it in an MCP server. Don't reinvent — import and expose.

---

*Generated from Waggle OS codebase audit, session 2026-04-12/13. 26 commits shipped this session.*