docs: add the install guide and the memory guide (P2.4)
docs/install.md covers both ways to install (the Tauri desktop packages and the server installer), the unsigned-installer warnings on each system, first launch, the data directory, update and uninstall. docs/memory.md describes the memory contract, its mem0 implementation, where mem0 keeps its files, recall and background fact extraction.
This commit is contained in:
176
docs/memory.md
Normal file
176
docs/memory.md
Normal file
@@ -0,0 +1,176 @@
|
||||
# Memory
|
||||
|
||||
Waggle remembers facts, decisions and preferences across chats and keeps them on your computer.
|
||||
This page describes how memory works in the community edition, for users who want to know what
|
||||
is kept and for contributors who change memory code.
|
||||
|
||||
The decisions behind it are in [ADR-0002](oss/adr/0002-memory-contract-local-mem0.md) and
|
||||
[ADR-0007](oss/adr/0007-memory-contract.md).
|
||||
|
||||
## The memory contract
|
||||
|
||||
Waggle reaches memory only through one interface, `MemoryStore`, in the package
|
||||
`@waggle/memory-contract` (`packages/memory-contract/src/types.ts`). No other code queries a
|
||||
memory database directly. The package also holds an in-memory store for development and tests,
|
||||
and a compatibility suite that every implementation must pass.
|
||||
|
||||
The implementation that ships is `@waggle/memory-mem0` (`packages/memory-mem0/src/index.ts`).
|
||||
It runs the open-source [mem0](https://github.com/mem0ai/mem0) library (`mem0ai`, Apache-2.0,
|
||||
pinned to 3.3.1) inside the Waggle server. The hosted Mem0 Platform is not used, and nothing is
|
||||
sent to a cloud memory service.
|
||||
|
||||
### Records
|
||||
|
||||
A memory record has:
|
||||
|
||||
| Field | Values |
|
||||
|---|---|
|
||||
| `text` | The memory itself |
|
||||
| `scope` | `personal`, or one workspace |
|
||||
| `kind` | `fact`, `event`, `preference`, `profile`, `decision`, `summary`, `note` |
|
||||
| `importance` | `critical`, `important`, `normal`, `temporary` |
|
||||
| `source` | `user_stated`, `tool_verified`, `agent_inferred`, `import`, `system` |
|
||||
| `origin` | Which part of Waggle wrote it, for example `chat`, `agent-run`, `loop` or `extraction` |
|
||||
| `sessionId` | The chat it came from, if any |
|
||||
| `createdAt`, `updatedAt` | ISO 8601 timestamps |
|
||||
| `metadata` | Free fields set by the caller (title, tags and so on) |
|
||||
|
||||
Ids are unique across all scopes.
|
||||
|
||||
### Scopes
|
||||
|
||||
- **Personal** memory follows you across all workspaces: who you are, how you like to work,
|
||||
facts that are not tied to one project.
|
||||
- **Workspace** memory belongs to one workspace: its decisions, findings and files.
|
||||
|
||||
In a workspace, the agent reads both its personal memory and that workspace's memory. It never
|
||||
reads another workspace's memory unless you ask it to search all workspaces (for example with
|
||||
`/search-all`).
|
||||
|
||||
### Operations
|
||||
|
||||
| Operation | What it does |
|
||||
|---|---|
|
||||
| `remember` | Stores a text as one memory, or extracts facts from a conversation with a model. Identical text in the same scope is not stored twice. |
|
||||
| `search` | Returns the memories most relevant to a query, best first (default 10) |
|
||||
| `list` | Returns memories newest first (default 50) |
|
||||
| `count` | Counts the memories that match a filter |
|
||||
| `get` | Reads one memory by id |
|
||||
| `update` | Changes the text, kind, importance or metadata of a memory |
|
||||
| `forget` | Deletes one memory |
|
||||
| `forgetMatching` | Deletes every memory that matches a filter |
|
||||
| `history` | Lists the changes to one memory, oldest first |
|
||||
|
||||
Filters select by scopes, kinds, minimum importance, origin, chat session and a creation-time
|
||||
window.
|
||||
|
||||
### Capabilities
|
||||
|
||||
A store reports what it can do, and the web UI hides what it cannot:
|
||||
|
||||
| Capability | mem0 store | Meaning |
|
||||
|---|---|---|
|
||||
| `extraction` | Yes, when a model is set up | `remember` can extract facts from a conversation |
|
||||
| `semanticSearch` | Yes | Search ranks by meaning, not only by shared words |
|
||||
| `history` | Yes | The change log of each memory is kept |
|
||||
| `knowledgeGraph` | No | Entities and relations between memories. Not part of this release. |
|
||||
|
||||
## Where mem0 keeps its data
|
||||
|
||||
Everything is in `memory/` inside the data directory (`~/.waggle/memory/` by default):
|
||||
|
||||
| File | Content |
|
||||
|---|---|
|
||||
| `vectors.db` | Memories and their embeddings (SQLite) |
|
||||
| `history.db` | The change log, and the last messages used for fact extraction (SQLite) |
|
||||
| `vectors_entities.db` | A second store mem0 creates next to `vectors.db` (SQLite) |
|
||||
| `waggle-memory.json` | Which embedding model built the store |
|
||||
| `config.json` | mem0's own settings |
|
||||
|
||||
All memories belong to one local user. The scope is stored as mem0's agent id (`personal` or
|
||||
`workspace:<id>`), so mem0 never compares facts across scopes.
|
||||
|
||||
mem0's own telemetry is turned off before the library loads, so it makes no network requests
|
||||
of its own. A test checks this.
|
||||
|
||||
When you forget a memory, Waggle also deletes its old text from `history.db`, so the text does
|
||||
not survive in the change log.
|
||||
|
||||
### Embeddings
|
||||
|
||||
Search compares embeddings: number vectors that capture the meaning of a text. Waggle computes
|
||||
them on your computer with a small model (`all-MiniLM-L6-v2`), downloaded on first use into
|
||||
`models/` in the data directory. After that, memory works without a network connection.
|
||||
|
||||
The store records which embedding model built it and refuses to open with a different one,
|
||||
because their vectors cannot be compared. To start over with another model, move the
|
||||
`memory/` directory aside.
|
||||
|
||||
## How the agent uses memory
|
||||
|
||||
### Recall
|
||||
|
||||
Before it answers, the agent recalls what is relevant to your message:
|
||||
|
||||
- it searches personal memory and the current workspace's memory (mem0 combines vector and
|
||||
keyword search);
|
||||
- it adds the most important memories of the workspace, even if they do not match the words;
|
||||
- it adds facts about you (kind `profile`) and recent events, each in its own section;
|
||||
- a question about a period ("what did we do last week") limits the search to that period;
|
||||
- a catch-up question ("where were we?", "what did we decide?") recalls the workspace's
|
||||
important memories and decisions instead of searching for the words;
|
||||
- `temporary` memories are left out.
|
||||
|
||||
The recalled text passes the prompt-injection scanner first. If it looks like an attempt to
|
||||
give the agent instructions, the whole recall is dropped for that turn.
|
||||
|
||||
An optional reranker can reorder search results; it is off by default
|
||||
(`WAGGLE_RERANKER=1` turns it on).
|
||||
|
||||
### Writing
|
||||
|
||||
Memories are written:
|
||||
|
||||
- when you ask ("Remember that we chose PostgreSQL") or the agent decides something is worth
|
||||
keeping, through its `save_memory` tool;
|
||||
- from finished agent runs, scheduled loops, connector fetches and indexed workspace files;
|
||||
- from summaries when a long chat is compacted;
|
||||
- by background fact extraction (next section).
|
||||
|
||||
### Background fact extraction
|
||||
|
||||
While you chat, Waggle can pick out facts about you, such as your role or how you like answers,
|
||||
and save them to personal memory as kind `profile`.
|
||||
|
||||
- Finished chat turns are collected per chat. After six messages, or after a two-minute pause,
|
||||
they are handed to mem0, which asks a model to extract facts.
|
||||
- The model is the one you chat with. mem0 calls it through the Waggle server's built-in
|
||||
provider proxy, so a local model keeps this on your computer, and a provider key sends these
|
||||
messages to that provider, as chat does.
|
||||
- Long messages are cut to their first 4,000 characters.
|
||||
- A failed extraction is logged and never blocks a chat.
|
||||
- Turn it off in **Settings** → **General** → **Memory** ("Learn facts about me from chats").
|
||||
It is on by default, and unavailable when no model is set up.
|
||||
|
||||
## Managing memory
|
||||
|
||||
Open **Memory** in Waggle to browse, search, correct, confirm or forget what is kept.
|
||||
`/memory <query>` searches from a chat.
|
||||
|
||||
Not in this release: importing chat history from other AI tools, the knowledge graph, the wiki,
|
||||
and the memory server for external tools (MCP).
|
||||
|
||||
## For contributors
|
||||
|
||||
- Reach memory only through `MemoryStore`. Do not query a memory database with SQL and do not
|
||||
import mem0 outside `@waggle/memory-mem0`.
|
||||
- Waggle's own data (automations, cost tracking, notifications, your profile card, the task
|
||||
list) is not memory. It lives in `waggle.db`, opened by `WaggleDB` in `@waggle/core`.
|
||||
- A new implementation of the contract must pass the compatibility suite in
|
||||
`@waggle/memory-contract`.
|
||||
- `list`, `count` and `forgetMatching` read every memory of the scopes involved. That is fine at
|
||||
personal scale.
|
||||
- Upgrading `mem0ai` means re-running the compatibility suite and re-checking that mem0 sends
|
||||
no telemetry (ADR-0007).
|
||||
- Tests use a word-based embedder that needs no model download:
|
||||
`WAGGLE_MEMORY_EMBEDDER=words`.
|
||||
Reference in New Issue
Block a user