diff --git a/docs/install.md b/docs/install.md new file mode 100644 index 0000000..32fc522 --- /dev/null +++ b/docs/install.md @@ -0,0 +1,257 @@ +# Installing Waggle + +There are two ways to install Waggle. Both run the same server and the same web UI, and both +keep your data in the same place. + +| | Desktop app | Server installer | +|---|---|---| +| What you get | A Waggle window, started from the Start menu, Applications or your launcher | A `waggle-server` command; you use Waggle in your browser | +| Needs | Nothing else | Node.js 20 or newer for the first `npx` call only | +| Formats | Windows `.exe` and `.msi`, macOS `.dmg`, Linux `.deb` and `.AppImage` | One archive per system, downloaded by the installer | + +Supported systems in this release: Windows x64, macOS on Apple silicon (arm64) and Intel +(x64), and Linux x64 with glibc (for example Debian, Ubuntu, Fedora). Other systems, such as +Windows on ARM or Linux on ARM, have no build. + +Neither way needs Python, Docker or a compiler. Each build carries its own Node.js runtime +(22.23.2) and the native modules that match it. + +> **Unsigned first release.** The installers of the first community release are not signed. +> Windows and macOS warn about them. The sections below show how to get past the warning. +> Download Waggle only from the project's release page. + +## Desktop app + +### Windows + +1. Download `Waggle__x64-setup.exe`. A `.msi` package is also available; use the + `.exe` unless you deploy with MSI tools. The `.exe` installs for the current user and does + not need administrator rights. +2. If the browser warns that the file is not commonly downloaded, choose to keep it (in Edge: + **...** → **Keep**). +3. Run the installer. SmartScreen shows "Windows protected your PC". Choose **More info**, then + **Run anyway**. +4. Start Waggle from the Start menu. + +Waggle uses the WebView2 runtime, which ships with current Windows 10 and 11. + +### macOS + +1. Download the `.dmg` for your Mac: `Waggle__aarch64.dmg` for Apple silicon, + `Waggle__x64.dmg` for Intel. (Apple menu → **About This Mac** shows the chip.) +2. Open the `.dmg` and drag **Waggle** to **Applications**. +3. Open Waggle. Because the app is not signed, macOS blocks the first launch: + - macOS 15 and later: after the warning, open **System Settings** → **Privacy & Security**, + scroll to the message about Waggle and choose **Open Anyway**. + - Earlier versions: Control-click (or right-click) Waggle in Applications, choose **Open**, + then **Open** again in the dialog. + - If macOS says the app "is damaged" or still refuses, remove the download quarantine flag + in Terminal and open it again: + + ```bash + xattr -dr com.apple.quarantine /Applications/Waggle.app + ``` + +You only need to do this once. + +### Linux + +Debian, Ubuntu and other distributions with apt: + +```bash +sudo apt install ./Waggle__amd64.deb +``` + +apt installs the libraries the app needs (WebKitGTK and GTK). Start Waggle from your +application launcher. + +AppImage, for other distributions: + +```bash +chmod +x Waggle__amd64.AppImage +./Waggle__amd64.AppImage +``` + +An AppImage needs FUSE 2. If it fails with a FUSE error, install it (`libfuse2` on Ubuntu 22.04, +`libfuse2t64` on Ubuntu 24.04 and later, `fuse-libs` on Fedora). + +Linux packages are not signed. If the release page lists SHA-256 checksums, compare them before +you install: + +```bash +sha256sum Waggle__amd64.deb +``` + +### What the desktop app does on start + +The window shows "Starting Waggle…" while the app starts its own server in the background. The +server listens on `127.0.0.1` port 3333, or on another free port if 3333 is taken. Its log is in +`~/.waggle/logs/service.log` (on Windows `%USERPROFILE%\.waggle\logs\service.log`). + +## Server installer + +The server installer puts the Waggle server on your computer and starts it. You then use Waggle +in your browser at http://localhost:3333. + +> **Temporary names.** The package name `@waggle-ai/waggle` and the command `waggle-server` are +> working names until the product name is decided. The installer also has no default download +> location yet, because the location of the public repository is not decided. Until then, pass +> the archive with `--from`, as shown below. + +### Install + +You need Node.js 20 or newer for this one command. After that, Waggle runs on its own Node.js +runtime and does not use yours. + +```bash +npx @waggle-ai/waggle install --from +``` + +The archive for your system is named `waggle-server--.tar.gz`, where +`` is `windows-x64`, `macos-arm64`, `macos-x64` or `linux-x64`. A `.sha256` file is +published next to it. + +The installer: + +1. checks the archive against its SHA-256 (for a URL it reads `.sha256`; for a local file it + reads `.sha256` if one is next to it). A mismatch stops the install. Pass + `--sha256 ` to give the checksum yourself; +2. unpacks it into a program directory; +3. writes the `waggle-server` command. + +| System | Program directory | `waggle-server` command | +|---|---|---| +| Windows | `%LOCALAPPDATA%\Waggle\server` | `%LOCALAPPDATA%\Waggle\server\bin\waggle-server.cmd` | +| macOS | `~/Library/Application Support/Waggle/server` | `~/.local/bin/waggle-server` | +| Linux | `~/.local/share/waggle/server` (or `$XDG_DATA_HOME/waggle/server`) | `~/.local/bin/waggle-server` | + +The installer never edits your `PATH`. If the command's directory is not on your `PATH`, it +prints the full path to run and the directory to add. + +Once a download location exists, `npx @waggle-ai/waggle install` without `--from` downloads +the archive for your system. `WAGGLE_RELEASE_BASE_URL` points it at a mirror. + +### Run + +```bash +waggle-server +``` + +This is the same as `waggle-server run`. It starts the server, waits until it answers and opens +http://localhost:3333 in your browser. The server runs in that terminal; press Ctrl+C to stop +it. It listens on this computer only (`127.0.0.1`). + +| Option | Meaning | +|---|---| +| `--port `, `-p` | Use another port (default 3333) | +| `--no-open` | Do not open the browser | +| `--data-dir ` | Keep data in another directory (default `$WAGGLE_DATA_DIR` or `~/.waggle`) | +| `--dir ` | Use a program directory other than the default | + +If Waggle already runs on the port, `waggle-server` only opens the browser. + +### Check the installation + +```bash +waggle-server doctor +``` + +`doctor` checks that this system has a build, the installed files, the bundled Node.js runtime, +that SQLite loads, that the data directory is writable, the port, and whether +`waggle-server` is on your `PATH`. It exits with an error code if a check fails. + +### Update + +`update` installs the version of the installer you run it with: + +```bash +npx @waggle-ai/waggle@latest update # once a download location exists +waggle-server update --from # a specific archive +``` + +Your data is not touched. On Windows a running program cannot be replaced: stop the server +first, and run `update` through `npx`, not through the installed `waggle-server`. + +### Uninstall + +```bash +waggle-server uninstall +``` + +This removes the program directory and the `waggle-server` command. Your data stays in the data +directory. On Windows, stop the server and run `npx @waggle-ai/waggle uninstall`. + +All commands also take `--help` and `--version`. + +## First launch + +Waggle opens a short setup: + +1. **Welcome** and **Who are you**: your name, role and the kind of work you do, kept on this + computer in your profile. +2. **Model**: choose how Waggle thinks (see below). You can skip this step, but chat stays + unavailable until a model is set up. +3. **Pick a starting point**: a template for your first workspace (for example Research Hub, + Sales Pipeline or Engineering), or **Blank Workspace**. +4. **What's first?**: give the agent its first task, or pick a suggestion. + +### Choose a model + +- **API key.** Paste a key from a provider you already use: Anthropic, OpenAI, Google, + DeepSeek, xAI, Mistral, Alibaba (Qwen), MiniMax, Zhipu (GLM), Moonshot (Kimi), Perplexity or + OpenRouter. Waggle checks the key with the provider and stores it encrypted on this computer. + Requests go from your computer straight to that provider. +- **Local model.** Choose **Install private runtime**. Waggle downloads the official Ollama + runtime into its data directory, checks the download's checksum and runs it for you. Then + download a model (Waggle suggests one that fits your hardware). Nothing else needs to be + installed, and chats stay on your computer. The downloads are large: the Linux runtime alone + is about 1.4 GB, and a model adds several GB. If you already run Ollama yourself, Waggle uses + it. + +You can change both later in **Settings** → **Models**. Keys are also listed in the **Vault**. + +### Analytics + +Release builds may carry a key for product analytics. If yours does, Waggle asks once whether it +may send usage analytics. Nothing is sent until you choose **Allow**. If you allow it, Waggle +sends one event when you finish the setup, with the template, persona and model you chose. Your +chats, files and memories are never sent. You can turn analytics off again in **Settings** → +**General** → **Privacy & Telemetry**. Builds from source have no key: they never ask and never +send. + +## Where your data lives + +Both ways of installing use one data directory: `~/.waggle` (on Windows +`%USERPROFILE%\.waggle`). Set `WAGGLE_DATA_DIR` to use another directory; the server installer +also takes `--data-dir`. + +| Path in the data directory | What it holds | +|---|---| +| `config.json` | Settings | +| `vault.json`, `.vault-key` | Provider keys and connector credentials, encrypted with AES-256-GCM | +| `waggle.db` | Automations, notifications, cost tracking, your profile card and task list | +| `memory/` | Your memories (see [memory.md](memory.md)) | +| `workspaces//` | Each workspace: settings, chat sessions (`sessions/*.jsonl`) and files | +| `skills/`, `plugins/` | Installed skills and plugins | +| `marketplace.db` | The marketplace catalog | +| `models/` | The embedding model used by memory, downloaded on first use; local model weights in `models/ollama/` | +| `runtimes/ollama/` | The Waggle-managed Ollama runtime | +| `channels/`, `browsers/` | Messaging channel sessions, and a browser downloaded for WhatsApp if none is installed | +| `logs/` | The desktop app's server log | + +Uninstalling Waggle, either way, keeps this directory. To remove your data, quit Waggle and +delete the directory, or use **Settings** → **General** → **Erase all my data**, which wipes it +the next time Waggle starts. + +To move Waggle to another computer, quit it and copy the whole directory. + +The commercial edition of Waggle uses the same default directory. If you use both on one +computer, give the community edition its own directory with `WAGGLE_DATA_DIR`. + +## Run from source + +See the [README](../README.md#run-from-source) and [CONTRIBUTING.md](../CONTRIBUTING.md). + +## Problems + +See [troubleshooting](guides/troubleshooting.md). diff --git a/docs/memory.md b/docs/memory.md new file mode 100644 index 0000000..f48d141 --- /dev/null +++ b/docs/memory.md @@ -0,0 +1,176 @@ +# Memory + +Waggle remembers facts, decisions and preferences across chats and keeps them on your computer. +This page describes how memory works in the community edition, for users who want to know what +is kept and for contributors who change memory code. + +The decisions behind it are in [ADR-0002](oss/adr/0002-memory-contract-local-mem0.md) and +[ADR-0007](oss/adr/0007-memory-contract.md). + +## The memory contract + +Waggle reaches memory only through one interface, `MemoryStore`, in the package +`@waggle/memory-contract` (`packages/memory-contract/src/types.ts`). No other code queries a +memory database directly. The package also holds an in-memory store for development and tests, +and a compatibility suite that every implementation must pass. + +The implementation that ships is `@waggle/memory-mem0` (`packages/memory-mem0/src/index.ts`). +It runs the open-source [mem0](https://github.com/mem0ai/mem0) library (`mem0ai`, Apache-2.0, +pinned to 3.3.1) inside the Waggle server. The hosted Mem0 Platform is not used, and nothing is +sent to a cloud memory service. + +### Records + +A memory record has: + +| Field | Values | +|---|---| +| `text` | The memory itself | +| `scope` | `personal`, or one workspace | +| `kind` | `fact`, `event`, `preference`, `profile`, `decision`, `summary`, `note` | +| `importance` | `critical`, `important`, `normal`, `temporary` | +| `source` | `user_stated`, `tool_verified`, `agent_inferred`, `import`, `system` | +| `origin` | Which part of Waggle wrote it, for example `chat`, `agent-run`, `loop` or `extraction` | +| `sessionId` | The chat it came from, if any | +| `createdAt`, `updatedAt` | ISO 8601 timestamps | +| `metadata` | Free fields set by the caller (title, tags and so on) | + +Ids are unique across all scopes. + +### Scopes + +- **Personal** memory follows you across all workspaces: who you are, how you like to work, + facts that are not tied to one project. +- **Workspace** memory belongs to one workspace: its decisions, findings and files. + +In a workspace, the agent reads both its personal memory and that workspace's memory. It never +reads another workspace's memory unless you ask it to search all workspaces (for example with +`/search-all`). + +### Operations + +| Operation | What it does | +|---|---| +| `remember` | Stores a text as one memory, or extracts facts from a conversation with a model. Identical text in the same scope is not stored twice. | +| `search` | Returns the memories most relevant to a query, best first (default 10) | +| `list` | Returns memories newest first (default 50) | +| `count` | Counts the memories that match a filter | +| `get` | Reads one memory by id | +| `update` | Changes the text, kind, importance or metadata of a memory | +| `forget` | Deletes one memory | +| `forgetMatching` | Deletes every memory that matches a filter | +| `history` | Lists the changes to one memory, oldest first | + +Filters select by scopes, kinds, minimum importance, origin, chat session and a creation-time +window. + +### Capabilities + +A store reports what it can do, and the web UI hides what it cannot: + +| Capability | mem0 store | Meaning | +|---|---|---| +| `extraction` | Yes, when a model is set up | `remember` can extract facts from a conversation | +| `semanticSearch` | Yes | Search ranks by meaning, not only by shared words | +| `history` | Yes | The change log of each memory is kept | +| `knowledgeGraph` | No | Entities and relations between memories. Not part of this release. | + +## Where mem0 keeps its data + +Everything is in `memory/` inside the data directory (`~/.waggle/memory/` by default): + +| File | Content | +|---|---| +| `vectors.db` | Memories and their embeddings (SQLite) | +| `history.db` | The change log, and the last messages used for fact extraction (SQLite) | +| `vectors_entities.db` | A second store mem0 creates next to `vectors.db` (SQLite) | +| `waggle-memory.json` | Which embedding model built the store | +| `config.json` | mem0's own settings | + +All memories belong to one local user. The scope is stored as mem0's agent id (`personal` or +`workspace:`), so mem0 never compares facts across scopes. + +mem0's own telemetry is turned off before the library loads, so it makes no network requests +of its own. A test checks this. + +When you forget a memory, Waggle also deletes its old text from `history.db`, so the text does +not survive in the change log. + +### Embeddings + +Search compares embeddings: number vectors that capture the meaning of a text. Waggle computes +them on your computer with a small model (`all-MiniLM-L6-v2`), downloaded on first use into +`models/` in the data directory. After that, memory works without a network connection. + +The store records which embedding model built it and refuses to open with a different one, +because their vectors cannot be compared. To start over with another model, move the +`memory/` directory aside. + +## How the agent uses memory + +### Recall + +Before it answers, the agent recalls what is relevant to your message: + +- it searches personal memory and the current workspace's memory (mem0 combines vector and + keyword search); +- it adds the most important memories of the workspace, even if they do not match the words; +- it adds facts about you (kind `profile`) and recent events, each in its own section; +- a question about a period ("what did we do last week") limits the search to that period; +- a catch-up question ("where were we?", "what did we decide?") recalls the workspace's + important memories and decisions instead of searching for the words; +- `temporary` memories are left out. + +The recalled text passes the prompt-injection scanner first. If it looks like an attempt to +give the agent instructions, the whole recall is dropped for that turn. + +An optional reranker can reorder search results; it is off by default +(`WAGGLE_RERANKER=1` turns it on). + +### Writing + +Memories are written: + +- when you ask ("Remember that we chose PostgreSQL") or the agent decides something is worth + keeping, through its `save_memory` tool; +- from finished agent runs, scheduled loops, connector fetches and indexed workspace files; +- from summaries when a long chat is compacted; +- by background fact extraction (next section). + +### Background fact extraction + +While you chat, Waggle can pick out facts about you, such as your role or how you like answers, +and save them to personal memory as kind `profile`. + +- Finished chat turns are collected per chat. After six messages, or after a two-minute pause, + they are handed to mem0, which asks a model to extract facts. +- The model is the one you chat with. mem0 calls it through the Waggle server's built-in + provider proxy, so a local model keeps this on your computer, and a provider key sends these + messages to that provider, as chat does. +- Long messages are cut to their first 4,000 characters. +- A failed extraction is logged and never blocks a chat. +- Turn it off in **Settings** → **General** → **Memory** ("Learn facts about me from chats"). + It is on by default, and unavailable when no model is set up. + +## Managing memory + +Open **Memory** in Waggle to browse, search, correct, confirm or forget what is kept. +`/memory ` searches from a chat. + +Not in this release: importing chat history from other AI tools, the knowledge graph, the wiki, +and the memory server for external tools (MCP). + +## For contributors + +- Reach memory only through `MemoryStore`. Do not query a memory database with SQL and do not + import mem0 outside `@waggle/memory-mem0`. +- Waggle's own data (automations, cost tracking, notifications, your profile card, the task + list) is not memory. It lives in `waggle.db`, opened by `WaggleDB` in `@waggle/core`. +- A new implementation of the contract must pass the compatibility suite in + `@waggle/memory-contract`. +- `list`, `count` and `forgetMatching` read every memory of the scopes involved. That is fine at + personal scale. +- Upgrading `mem0ai` means re-running the compatibility suite and re-checking that mem0 sends + no telemetry (ADR-0007). +- Tests use a word-based embedder that needs no model download: + `WAGGLE_MEMORY_EMBEDDER=words`.