Waggle kept its own tables in the memory database. Before the memory
engine can be replaced by mem0 they need a home of their own.
- add WaggleDB (<dataDir>/waggle.db). Automations, notifications, held
actions, skill hashes, thumbs feedback, cost tracking, the profile card
and awareness items live there. Their stores take a WaggleDB, so passing
a memory database is a type error.
- the profile card and awareness items are now per scope ('personal' or
'workspace:<id>') in one table each, instead of one table per .mind file
- on the first start, copy these tables from personal.mind and every
workspace.mind (importLegacyMindData). The legacy files are only read,
each scope is copied once and a table only while its target is empty.
- copy the logger, injection scanner, external-memory ingress guard,
WorkspaceManager and the embedders from @waggle/hive-mind-core into
@waggle/core, which now re-exports its own copies
- point the server, agent orchestrator and CLI at waggle.db; update the
tests that seeded these tables in a memory database
ADR-0007 records the layout and the licensing question about the copied
files.
104 lines
6.5 KiB
Markdown
104 lines
6.5 KiB
Markdown
# ADR-0007: The memory contract and how it maps onto mem0
|
|
|
|
- Status: Accepted
|
|
- Date: 2026-10-18
|
|
|
|
## Context
|
|
|
|
ADR-0002 decided that Waggle reaches memory through a contract backed by local mem0. The
|
|
inventory of today's callers ([research/memory-inventory.md](../research/memory-inventory.md))
|
|
shows what the contract must cover: 68 files use the frame store; content prefixes such as
|
|
`[mind-fact]` or `Decision:` act as types and are queried with `LIKE`; named "sessions" group
|
|
writes by producer; personal and workspace minds are separate databases whose ids collide; the
|
|
same databases also hold the user's profile card, a task list and a dozen non-memory tables.
|
|
What mem0 3.3.1 offers is recorded in [research/mem0-3.3.1.md](../research/mem0-3.3.1.md).
|
|
|
|
The owner decided on 2026-10-16 that the profile card keeps exact fields while facts learned
|
|
about the user live in mem0, and that facts are extracted in the background, in batches.
|
|
|
|
## Decision
|
|
|
|
1. **Two packages.** `@waggle/memory-contract` holds the `MemoryStore` interface, the record
|
|
model, the capability map, an in-memory store for development and tests, and a
|
|
compatibility suite every implementation must pass. `@waggle/memory-mem0` implements the
|
|
contract on mem0. Callers depend only on the contract.
|
|
2. **Record model.** A memory has text, a scope (personal or one workspace), a kind (fact,
|
|
event, preference, profile, decision, summary, note) that replaces the content prefixes, an
|
|
importance (critical, important, normal, temporary), a source, an origin (the producer:
|
|
'chat', 'agent-run', 'loop'…) that replaces the named sessions, an optional chat session id,
|
|
timestamps and free metadata. Ids are unique across scopes.
|
|
3. **Operations.** `remember` stores text (identical text in the same scope is not written
|
|
twice) or extracts facts from a conversation; `search` ranks; `list` returns newest first;
|
|
`count`, `get`, `update`, `forget`, `forgetMatching` and `history` complete the set. Filters
|
|
cover scopes, kinds, minimum importance, origin, session and a creation-time window.
|
|
4. **No direct access.** Code outside a `MemoryStore` implementation does not query a memory
|
|
database with SQL and does not import `@waggle/hive-mind-*`, `memory-mcp` or
|
|
`wiki-compiler`. New code follows this rule now; P1.5 removes the existing violations.
|
|
5. **Not memory.** The user's profile card (name, role, department, style and the other
|
|
identity fields), the task list (awareness) and the other Waggle tables move to `waggle.db`
|
|
in P1.5. What the model learns about the user is memory: kind 'profile', personal scope.
|
|
Done on 2026-10-23, see "waggle.db" below.
|
|
6. **Extraction policy.** Facts are extracted in the background, in batches (after several
|
|
turns or a pause), with the model the user chose, through the sidecar's built-in
|
|
OpenAI-compatible proxy. A setting turns it off. Wiring this is part of P1.5.
|
|
7. **mem0 configuration.**
|
|
- One store per data directory, in `<dataDir>/memory/`: `vectors.db`, `vectors_entities.db`,
|
|
`history.db` and mem0's `config.json`. Every memory belongs to user 'local'; the scope is
|
|
mem0's `agent_id` ('personal' or 'workspace:<id>'), so mem0 never compares facts across
|
|
scopes. Contract fields live in mem0 metadata.
|
|
- Embeddings come from Waggle's in-process embedder, passed through mem0's `langchain`
|
|
provider, so memory works offline. Verified on 2026-10-18 with network access blocked:
|
|
no request was made and paraphrased queries found the right memory.
|
|
- Fact extraction uses mem0's `openai` provider pointed at an OpenAI-compatible endpoint.
|
|
Without one, `capabilities.extraction` is false.
|
|
- `MEM0_TELEMETRY=false` and `MEM0_DIR` are set before mem0 is imported. With telemetry on,
|
|
mem0 sends every call to its PostHog project and downloads a notice configuration from
|
|
GitHub; a test asserts that no request is made.
|
|
- `mem0ai` is pinned to exactly 3.3.1. An upgrade re-runs the compatibility suite and
|
|
re-checks the telemetry behaviour.
|
|
8. **Knowledge graph.** `capabilities.knowledgeGraph` is false; graph features stay hidden
|
|
(ADR-0002).
|
|
|
|
## Consequences
|
|
|
|
- P1.5 maps the callers onto the contract: prefix lanes become `kinds`, the importance lane
|
|
becomes `minImportance`, named sessions become `origin`, `getLatestIFrame`/`createPFrame`
|
|
chains become plain `remember` calls.
|
|
- Compared with the current engine: a write cannot set its own creation time (mem0's OSS build
|
|
rejects timestamps; history import is hidden in v1 anyway); importance filters but does not
|
|
boost ranking; P/B-frame chains, access counts and full-text search are gone, replaced by
|
|
mem0's hybrid vector and BM25 search. P1.5 compares search quality on a reference set.
|
|
- `list`, `count` and `forgetMatching` read every memory of the scopes involved. That is fine
|
|
at personal scale and should be revisited if a store grows past tens of thousands.
|
|
- mem0 keeps its SQLite handles open for the life of the process; `close` only marks the store
|
|
closed.
|
|
|
|
## waggle.db (P1.5, day 6)
|
|
|
|
`WaggleDB` in `@waggle/core` opens `<dataDir>/waggle.db`. It holds:
|
|
|
|
- automations: `cron_schedules`, `cron_execution_history`, `cron_run_leases`;
|
|
- `notifications` and held actions (`pending_actions`);
|
|
- `skill_hashes` and thumbs feedback (`feedback_entries`);
|
|
- cost tracking: `execution_traces`, `execution_trace_spend`,
|
|
`execution_trace_spend_reservations`;
|
|
- the profile card (`profiles`) and awareness items (`awareness_items`), one set per scope:
|
|
`personal` or `workspace:<id>`.
|
|
|
|
The stores of these tables take a `WaggleDB`, not a generic SQLite handle, so handing them a
|
|
memory database is a type error. Telemetry (`telemetry.db`) and the audit log (`audit.db`)
|
|
already had their own files and stay there.
|
|
|
|
On the first start the server copies these tables from the legacy `personal.mind`, and the
|
|
profile and awareness items of every `workspace.mind`, into `waggle.db`
|
|
(`importLegacyMindData`). The legacy files are only read, because the data directory may be
|
|
shared with the commercial edition; each scope is copied once (recorded in `waggle_meta`) and a
|
|
table only while its target is empty.
|
|
|
|
The logger, injection scanner, external-memory ingress guard, `WorkspaceManager` and the
|
|
embedders were copied from `@waggle/hive-mind-core` into `@waggle/core`, which now re-exports
|
|
its own copies. The substrate keeps its copies until it is removed. Those files came from an
|
|
Apache-2.0 package into the MIT `core` package; both are Egzakta's code, and the licence of the
|
|
copies is part of the licensing review.
|
|
|