diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..e890c4c --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,92 @@ +# Changelog + +All notable changes to the Waggle community edition are listed here. The format follows +[Keep a Changelog](https://keepachangelog.com/en/1.1.0/). + +## Unreleased: first community edition + +The first open-source release of Waggle, made from the commercial Waggle OS product. This entry +lists what changed for users compared with that product. The reasons are recorded in the +[decision records](docs/oss/adr/README.md). + +### Removed + +- **Billing and plans.** Trial, Solo, Teams and Enterprise plans, Stripe checkout, license + checks, trial banners and upgrade prompts. Everything a plan used to unlock is available + (ADR-0001). +- **KVARK** enterprise retrieval, its agent tools, and the enterprise capability packs. +- **Compliance reports** and the compliance subsystem. +- **Self-learning** (ADR-0004): behavior evolution and prompt optimization (the `optimizer` + package), learning from corrections, the Evolution screen (Memory → Improvements), the + nightly prompt-optimization and monthly-assessment jobs, and the idle-session reviewer. + Thumbs up and down are still stored, and corrections you make are saved as ordinary memories. +- **Team mode** (ADR-0003): connecting to a team server, team workspaces and shared memory, + WaggleDance, the team screens, and S3 file storage for workspaces. The code stays in the + repository but is not built or shipped. +- **Features of the old memory engine** (ADR-0002): the knowledge graph, the wiki, importing chat + history from ChatGPT and Claude exports, memory consolidation, the memory server for + external tools (MCP), and the memory hooks for Claude Code, Codex and other coding agents. +- **`install.sh`** and `scripts/waggle-server.sh`, replaced by the server installer. +- The automatic start of a LiteLLM proxy, which needed Python. It is now opt-in (see Changed). + +### Changed + +- **Memory runs on local mem0** behind a memory contract (ADR-0002, ADR-0007). Memories are kept + in `~/.waggle/memory/`, embedded by a model that runs on your computer, and searched with + mem0's combined vector and keyword search. Nothing goes to a cloud memory service. Memories + from the old `.mind` files are not carried over. See [docs/memory.md](docs/memory.md). +- **Waggle's own data** (automations, notifications, cost tracking, your profile card and task + list) moved to a separate `waggle.db`, apart from memory. +- **Analytics are opt-in** (ADR-0006). Waggle asks once; nothing is sent until you choose + **Allow**, and Settings → General → Privacy & Telemetry turns it off again. When allowed, + Waggle sends one event at the end of setup with the template, persona and model you chose. +- **WhatsApp uses `whatsapp-web.js`** instead of Baileys (ADR-0005). Pair your phone again with + the QR code. The session is kept in the data directory as a browser profile, protected by file + permissions rather than the vault. If no Chrome, Edge or Chromium is installed, Waggle + downloads Chrome for Testing (about 170 MB) the first time you start WhatsApp. The channel + handles text messages only. +- **`npx @waggle-ai/waggle` installs a prebuilt server** (ADR-0009) instead of running the server + from npm. It adds the `waggle-server` command with `run`, `doctor`, `update` and `uninstall`, + and the installed server serves the web UI. The package and command names are temporary. +- **Desktop app and server archives carry their own Node.js** (22.23.2) with matching native + modules, so no Python or compiler is needed to install (ADR-0008). +- **Model calls go through the built-in provider proxy** by default. LiteLLM runs only when + `config.json` contains `"litellm": { "enabled": true }`, on your own Python. +- **The Waggle-managed Ollama runtime works on Linux**, and installs no longer download CUDA + libraries. +- **The recall reranker is off by default**; `WAGGLE_RERANKER=1` turns it on. +- **The license is MIT** for the whole repository. Notices for shipped third-party components are + in [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md). +- UI text no longer uses decorative emoji or em dashes, and generic AI icons were replaced. + +### Added + +- **Background fact extraction.** While you chat, Waggle can save facts about you to personal + memory, every six messages or after a two-minute pause, with the model you chat with. Turn it + off in Settings → General → Memory. +- **Memory capabilities.** The UI hides memory features the store does not support. +- **The server installer** (`packages/launcher`): `install`, `run`, `doctor`, `update` and + `uninstall`, with SHA-256 checks of the downloaded archive. +- **A license check** for shipped npm packages and Rust crates. +- **A new design**, off by default, switched on with "New design" at the bottom of the sidebar. +- **A bottom navigation bar on phones**, instead of the sidebar rail. + +### Fixed + +Fixes made while preparing this edition. Each is recorded in +[docs/oss/upstream-fixes.md](docs/oss/upstream-fixes.md). + +- The packaged macOS app could not find its bundled server and stayed on "Starting Waggle…" + (UF-004). +- On macOS and Linux the desktop app stopped its server without letting it shut down (UF-046). +- Local model downloads were cancelled after five minutes (UF-008). +- A working API key showed as not responding, and other internal model calls failed (UF-044). +- A memory saved to an unknown workspace landed in personal memory (UF-039). +- Memory searches from Timeline and the chat recall toast used the wrong scope (UF-035), and the + command palette searched the wrong workspace (UF-033). +- The web client showed "important" and "temporary" memories as normal (UF-036). +- Generating a persona from a description failed (UF-045). +- The onboarding Industry dropdown opened under the dialog (UF-007), and startup text was + invisible in the light theme (UF-017). +- Brand fonts did not load under the content security policy. +- Smaller interface fixes: UF-042, UF-043, UF-047 to UF-057. diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md index 683a20d..9df5ce6 100644 --- a/CODE_OF_CONDUCT.md +++ b/CODE_OF_CONDUCT.md @@ -66,6 +66,10 @@ and fairly. All community leaders are obligated to respect the privacy and security of the reporter of any incident. +Security vulnerabilities are not conduct issues. Report them privately as +described in [SECURITY.md](SECURITY.md): through GitHub private vulnerability +reporting, or to **hello@egzakta.com** if you cannot use GitHub. + ## Enforcement Guidelines Community leaders will follow these Community Impact Guidelines in determining diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..190f27e --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,131 @@ +# Contributing to Waggle + +Thank you for helping. This page covers the setup, the checks and how to propose a change. + +**Read [AGENTS.md](AGENTS.md) before you change code.** It is the operating contract for this +repository, for people and coding agents alike: what is in scope, the rules that must not be +broken, and the commands that verify a change. The decisions behind it are in +[docs/oss/adr/](docs/oss/adr/README.md). [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) explains +how the parts fit together. + +## Prerequisites + +- Node.js `^20.19.0` or `>=22.12.0`, and npm. Node 22 is recommended; the release builds bundle + 22.23.2. +- Git. +- For the desktop app only: a current Rust toolchain, plus the platform's Tauri prerequisites + (on Windows, Visual Studio Build Tools with the C++ workload; see + [docs/oss/build-windows.md](docs/oss/build-windows.md)). + +Python, Docker and a database server are not needed. + +## Setup + +Clone the repository, then in its root: + +```bash +npm ci +npm run build # type-checks the web UI and builds it into dist/ +npm run dev:server # the sidecar on http://localhost:3333, serving dist/ +``` + +Open http://localhost:3333 and add a model in Settings → Models. + +- Workspace packages export their TypeScript source under the `source` condition, which tsx, + Vitest, Vite and the sidecar build resolve. You do not need `npm run build:packages` before + running or testing; it type-checks the packages and builds their `dist/` for publishing. +- The sidecar registers the files in `dist/` when it starts. After `npm run build`, restart it, + or new assets return 404. +- `npm run dev:web` (Vite) cannot authenticate against the sidecar yet. Use the UI the sidecar + serves. +- Run one sidecar at a time. To keep test data apart from your own, start it with another data + directory: `WAGGLE_DATA_DIR=/tmp/waggle-dev npm run dev:server`. + +Other entry points: + +```bash +npm run dev -w @waggle/cli # terminal REPL on the same memory and waggle.db +(cd app && npm ci && npm run tauri:build:mac:arm64) # desktop app; see AGENTS.md section 4 +``` + +## Checks + +Run the checks that cover your change before you open a pull request. CI runs them too. + +```bash +npm run build:packages # tsc -b over every in-scope package +npm run typecheck:web # the web UI +npx tsc --noEmit --project app/tsconfig.json +npm run typecheck:tests # test suites; advisory until existing errors are fixed +npm run lint # root config; ignores apps/web +(cd apps/web && npx eslint .) # web config +npm run check:deps # every imported package is declared in its package.json +npm run check:licenses # shipped dependencies allow MIT distribution +npm test # root Vitest: packages, tests/, scripts/ +(cd apps/web && npx vitest run) # web UI tests +``` + +Some checks fail before your change: [docs/oss/baseline.md](docs/oss/baseline.md) records the +known failures and counts. A check that passes there must not start failing. + +End-to-end tests use Playwright (`npm run test:e2e`). They reuse a server already listening on +:3333 and its data directory. For an isolated run set `WAGGLE_E2E_REUSE_EXISTING_SERVER=0` and a +free `WAGGLE_E2E_PORT`. The e2e web server step runs `npm run build:all`, which rewrites `dist/`. + +Tests live in each package's `tests/` directory, in `apps/web/src` next to the code, and in the +root `tests/`. New behavior needs a test; a bug fix needs a test that fails without the fix. + +## Rules that reviews check first + +The full list is in [AGENTS.md](AGENTS.md), section 3. In short: + +- No monetization: no tiers, trials, payments, license checks or upsell. +- Memory goes through `MemoryStore` from `@waggle/memory-contract`. No SQL against a memory + database, and no knowledge graph in this release. +- Team mode (`packages/team-server`, `waggle-dance`, `worker`) is frozen, and self-learning is + out of scope. +- Every new dependency must allow MIT distribution: no GPL or AGPL code in shipped artifacts. + Name its license in the commit message. +- Analytics stay opt-in. Never commit a PostHog key or any other secret. +- Platform-specific code needs Windows, macOS and Linux paths. End users must not need Python, + Docker, a compiler or a developer Node.js install. +- Security constraints (secrets in the vault, the injection scanner on external input, no + `eval`, an explicit Tauri IPC allowlist, parameterized SQL) are in AGENTS.md, section 8. + +## Code style + +- TypeScript in strict mode, ES modules, `.js` extensions in relative imports, and + `import type` for types. +- File names in `kebab-case.ts`; React components in `PascalCase.tsx`. +- Match the style of the file you change. Do not reformat code you do not otherwise touch. +- Search before you add a file: [AGENTS.md](AGENTS.md), section 9, lists modules that already + exist. +- Code, comments, commit messages and documentation are in English. Write plainly. + +## Commits and pull requests + +- One focused change per pull request. Describe what changed, why, and how you tested it. +- Commit messages: `type(scope): summary` in the imperative, for example + `fix(server): close the memory recall step when nothing is found`. Types: `feat`, `fix`, + `refactor`, `test`, `docs`, `build`, `chore`, `perf`. +- Keep commits small enough to review. Do not mix a refactor with a behavior change. +- Never commit secrets, `.env` files or personal data. + +## Reporting bugs and security issues + +Open an issue for bugs and feature proposals. Report security vulnerabilities privately as +described in [SECURITY.md](SECURITY.md), not in a public issue. + +## Troubleshooting + +**Windows: the sidecar fails to start with an esbuild error** such as +`Cannot find module @esbuild/win32-x64`. npm skipped esbuild's platform binary. Install the +binary that matches the esbuild version named in the error, without changing `package.json`: + +```bash +npm i @esbuild/win32-x64@ --no-save +``` + +**Assets return 404 after a build.** Restart the sidecar; see Setup. + +More user-facing problems are in [docs/guides/troubleshooting.md](docs/guides/troubleshooting.md). diff --git a/README.md b/README.md index 36d8408..618b8e0 100644 --- a/README.md +++ b/README.md @@ -1,89 +1,113 @@ -# Waggle OS Community Edition +# Waggle Community Edition -Waggle is a local-first AI workspace: an agent with persistent memory, personas, skills, -connectors and scheduled automations. It runs as a Node.js sidecar with a React web UI, and -can be wrapped in a Tauri desktop shell. You bring your own model: a local model through -Ollama or a provider API key. +Waggle is an AI agent workspace that runs on your own computer. You organize work into +workspaces; in each one an agent chats with you, uses tools (files, web search, documents, +connected services), runs scheduled automations, and remembers facts and decisions between +chats. Memory, settings and keys stay on your computer. -> **Status: work in progress.** This branch is being converted from the commercial Waggle -> OS codebase into an open-source community edition. Billing, KVARK, compliance reporting -> and self-learning are already removed; memory is moving to a local mem0 store, and -> installers for Windows, macOS and Linux are not ready yet. The rules for this conversion -> are in [`AGENTS.md`](AGENTS.md) and the decision records in -> [`docs/oss/adr/`](docs/oss/adr/README.md). +The agent needs a model: a local model that Waggle downloads and runs for you, or your own key +for a provider such as Anthropic, OpenAI or Google. -## Repository layout +This is the open-source community edition (MIT). It has no paid tiers, no license checks and no +team features. Analytics are off unless you allow them. -``` -waggle-os/ -├── apps/web/ # Web UI (React 19, Vite, Tailwind 4) -├── app/ # Tauri 2 desktop shell (Rust); loads the apps/web build -├── packages/ # npm workspaces: agent, server, core, shared, marketplace, -│ # memory-contract, memory-mem0, cli, launcher, sdk, weaver, ... -└── docs/ # Guides, reference, and docs/oss for the community edition -``` +## Install -`AGENTS.md` lists the status of every package in this edition. +Two ways, on Windows x64, macOS (Apple silicon and Intel) and Linux x64: + +- **Desktop app.** Download the installer for your system (`.exe` or `.msi`, `.dmg`, `.deb` or + `.AppImage`) and run it. The first release is unsigned, so Windows and macOS show a warning + the first time; [the install guide](docs/install.md) shows how to get past it. +- **Server installer.** Install the server and use Waggle in your browser: + + ```bash + npx @waggle-ai/waggle install --from + waggle-server # starts Waggle at http://localhost:3333 + ``` + + Node.js 20 or newer is needed only for the `npx` call. The package and command names are + temporary, and there is no default download location yet, which is why `--from` is needed + for now. + +Neither way needs Python, Docker or a compiler. Details, updates and uninstalling: +[docs/install.md](docs/install.md). + +## What is in this edition + +- Workspaces with chat sessions, files and a choice of personas +- Memory on your computer: facts, search and a profile of what Waggle learned about you, through + a memory contract backed by a local [mem0](https://github.com/mem0ai/mem0) store + ([docs/memory.md](docs/memory.md)) +- Models: the Waggle-managed Ollama runtime, an Ollama you run, or your own provider keys +- Skills, plugins, MCP servers and a marketplace with a security scan before install +- Connectors (GitHub, Slack, Jira, Notion, Google Workspace and Microsoft apps among others) and + messaging channels (Telegram, Discord, Slack, WhatsApp) +- Automations, approvals for sensitive actions, cost tracking and budgets +- An encrypted vault for keys and credentials + +Not in this edition: billing and tiers, KVARK, compliance reports, self-learning, team mode, the +knowledge graph, chat-history import, the wiki, memory hooks for external coding agents (Claude +Code, Codex) and the memory server for external tools (MCP). +[CHANGELOG.md](CHANGELOG.md) lists what changed from the commercial product. ## Run from source -Prerequisites: Node.js `^20.19.0` or `>=22.12.0`, and npm. +Prerequisites: Node.js `^20.19.0` or `>=22.12.0` (22 recommended) and npm. ```bash -npm install -npm run build:packages # workspace packages export dist/; the sidecar needs it -npm run build # builds the web UI into dist/ -npm run dev:server # the sidecar on http://localhost:3333, serving the UI +npm ci +npm run build # type-checks the web UI and builds it into dist/ +npm run dev:server # the sidecar on http://localhost:3333, serving dist/ ``` -Open http://localhost:3333. Rebuild the UI and restart the sidecar to see UI changes: -the Vite dev server (`npm run dev:web`) cannot authenticate against the sidecar yet -(UF-003 in [`docs/oss/upstream-fixes.md`](docs/oss/upstream-fixes.md)). +Open http://localhost:3333. After changing the UI, run `npm run build` again and restart the +sidecar: it registers the files in `dist/` when it starts. The Vite dev server +(`npm run dev:web`) cannot sign in to the sidecar yet. -Provider keys are entered in the app under **Settings → API Keys** and stored in the -encrypted vault, so a basic local run needs no `.env` file. `.env.example` lists the -optional variables. +Workspace packages are resolved from their TypeScript source, so no package build is needed +before running or testing. Data goes to `~/.waggle` unless you set `WAGGLE_DATA_DIR`. -**Embeddings.** By default `EMBEDDING_PROVIDER=auto` downloads a small in-process model -(about 23 MB, cached under `~/.waggle/models/`) and works offline. To use Ollama instead, -install [Ollama](https://ollama.com), run `ollama pull nomic-embed-text`, and set -`EMBEDDING_PROVIDER=ollama`. +Add a model in the app (Settings → Models). `.env.example` lists optional environment +variables; a basic local run needs none of them. -| Variable | Description | -|---|---| -| `ANTHROPIC_API_KEY` | Claude API key; can be set in the app instead. | -| `OPENAI_API_KEY` | Enables OpenAI models and optional OpenAI embeddings. | -| `EMBEDDING_PROVIDER` | `auto` (default), `inprocess`, `ollama`, `voyage`, `openai` or `mock`. | -| `LITELLM_BASE_URL` | Optional external LiteLLM-compatible proxy. | +The desktop app and release archives are built per system: +see [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md#how-a-release-is-built). ## Checks ```bash -npm run build:packages # build all workspace packages -npm run typecheck:web # type-check the web UI -npm run lint # lint the packages -npm run check:deps # every import is a declared dependency -npm test # unit tests +npm run build:packages # type-check every package +npm run typecheck:web # type-check the web UI +npm run lint # lint the packages +(cd apps/web && npx eslint .) # lint the web UI +npm run check:deps # every import is a declared dependency +npm run check:licenses # shipped dependencies allow MIT distribution +npm test # unit tests (packages, tests/, scripts/) +(cd apps/web && npx vitest run) # web UI unit tests ``` -Known failures and the numbers to compare against are in -[`docs/oss/baseline.md`](docs/oss/baseline.md). +[CONTRIBUTING.md](CONTRIBUTING.md) has the full list, and +[docs/oss/baseline.md](docs/oss/baseline.md) records the known failures to compare against. ## Documentation -- [Getting Started](docs/GETTING-STARTED.md) +- [Install](docs/install.md) +- [Getting started](docs/guides/getting-started.md) +- [Workspaces](docs/guides/workspaces.md) +- [Skills, plugins and the marketplace](docs/guides/capabilities.md) +- [Connectors and channels](docs/guides/connectors.md) +- [Troubleshooting](docs/guides/troubleshooting.md) +- [Memory](docs/memory.md) - [Architecture](docs/ARCHITECTURE.md) -- [Contributing](docs/CONTRIBUTING.md) -- [Threat Model](THREAT_MODEL.md) -- [Security Policy](SECURITY.md) -- [Code of Conduct](CODE_OF_CONDUCT.md) - -The guides predate the community edition and still describe some removed features; they -are rewritten before the first release. +- Reference: [slash commands](docs/reference/commands.md), [local API](docs/reference/api.md) +- [Decision records](docs/oss/adr/README.md) +- [Contributing](CONTRIBUTING.md), [Security policy](SECURITY.md), + [Threat model](THREAT_MODEL.md), [Code of conduct](CODE_OF_CONDUCT.md) +- [Changelog](CHANGELOG.md) ## License -This repository is licensed under the [MIT License](./LICENSE), **except** the -`hive-mind-*` packages under `packages/`, which are licensed under **Apache-2.0**. -Each `hive-mind-*` package carries its own `LICENSE` file, which governs that -package. +The code in this repository is licensed under the [MIT License](LICENSE). + +Waggle uses third-party packages under their own licenses. The notices for the components that +ship with Waggle are in [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md). diff --git a/SECURITY.md b/SECURITY.md index 814f1fa..5f213cf 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -1,63 +1,72 @@ # Security Policy -Waggle OS runs on a user's own machine and touches sensitive material — API -keys, connector credentials, the local filesystem, and imported conversation -history. We take reports seriously and appreciate responsible disclosure. +Waggle runs on the user's own computer and handles sensitive material: provider API keys, +connector credentials, the local file system, chats and memories. We take reports seriously +and appreciate responsible disclosure. -For the trust boundary, the controls that enforce it, and the currently known -gaps, read [`THREAT_MODEL.md`](./THREAT_MODEL.md). It is the authoritative -description of what Waggle defends against and what it does not. +[`THREAT_MODEL.md`](./THREAT_MODEL.md) describes the trust boundary, the controls that enforce +it and the known gaps. -## Reporting a Vulnerability +## Supported versions -**Please do not open a public GitHub issue for security vulnerabilities.** +| Version | Supported | +|---|---| +| The latest community edition release | Yes | +| Earlier community edition releases | No. Update to the latest release. | +| Unreleased code on the default branch | Best effort | -Report privately through either channel: +The commercial Waggle OS product is not covered by this policy. -1. **GitHub Security Advisories (preferred).** On the repository, go to the - **Security** tab → **Report a vulnerability**. This opens a private advisory - visible only to the maintainers. -2. **Email.** Send details to **marko@egzakta.com** with `SECURITY` in the +## Reporting a vulnerability + +**Do not open a public issue for a security vulnerability.** + +1. **GitHub private vulnerability reporting (preferred).** On the repository, open the + **Security** tab and choose **Report a vulnerability**. The report is visible only to the + maintainers. +2. **Email, if you cannot use GitHub.** Write to **hello@egzakta.com** with `SECURITY` in the subject line. Please include: -- A description of the issue and the impact you believe it has. -- Step-by-step reproduction instructions (or a proof of concept). -- The affected component/package and version or commit SHA. -- Your environment (OS, Node version, desktop build vs. web). +- what the issue is and the impact you expect; +- steps to reproduce it, or a proof of concept; +- the affected component and the version or commit; +- your environment: operating system, desktop app or server installer, and the model provider + if it matters. -## What to Expect +## What to expect -- **Acknowledgement** within 5 business days. -- An initial assessment and severity classification shortly after. -- Coordinated disclosure: we will agree on a timeline with you and credit you in - the release notes unless you prefer to remain anonymous. +- An acknowledgement within 5 business days. +- An assessment and a severity rating after that. +- Coordinated disclosure: we agree on a timeline with you and credit you in the release notes + unless you prefer to stay anonymous. -Please give us a reasonable window to release a fix before any public -disclosure. +Please give us reasonable time to release a fix before you disclose the issue publicly. ## Scope In scope: -- The desktop app (Tauri shell + bundled sidecar), the web app, and the - workspace packages under `packages/` and `apps/`. -- Memory (`packages/memory-contract`, `packages/memory-mem0`). -- Prompt-injection paths, secret handling (the vault), the filesystem boundary, - and the capability/approval gates — see `THREAT_MODEL.md` for the details. +- the desktop app (the Tauri shell and its bundled server); +- the server installer (`packages/launcher`) and the server archives it installs; +- the web UI (`apps/web`) and the shipped packages under `packages/`; +- memory (`packages/memory-contract`, `packages/memory-mem0`); +- prompt-injection paths, secret handling in the vault, the file-system boundary, the local + API's authentication, and the approval gates. -Out of scope (documented, not vulnerabilities): +Out of scope: -- The known gaps enumerated in `THREAT_MODEL.md` (e.g. the pattern-based - injection scanner and the absence of an OS-level shell sandbox). If you can - demonstrate impact meaningfully beyond what is already documented there, we - still want to hear about it. -- Findings that require a compromised operator account or physical access to the - user's machine — the operator is trusted by design. +- the known gaps listed in `THREAT_MODEL.md`, such as the pattern-based injection scanner and + the absence of an operating-system sandbox for shell commands. If you can show impact beyond + what is documented there, we still want to hear about it; +- findings that need control of the user's account or physical access to the computer: the user + is trusted by design; +- team mode packages (`packages/team-server`, `packages/waggle-dance`, `packages/worker`), which + are not built or shipped in this edition; +- third-party services that Waggle connects to. ## Secrets -If a report involves an exposed secret (an API key, token, or credential), note -it explicitly so we can rotate it immediately. Never include live secrets in a -public issue or PR. +If a report involves an exposed secret (an API key, a token or a credential), say so clearly so +it can be rotated at once. Never put live secrets in a public issue or pull request. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 8c30efb..5a9c83e 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -1,263 +1,218 @@ # Architecture -Waggle is a monorepo with **28 packages** under `packages/` organized around a layered architecture: the memory substrate, agent intelligence, server API, and UI presentation. This document covers the package structure, data flow, and extension points. +How the community edition of Waggle is put together: the parts, how a chat turn flows through +them, where data lives and how a release is built. The decisions behind it are in +[`docs/oss/adr/`](oss/adr/README.md); [`AGENTS.md`](../AGENTS.md) lists the status of every +package. -> **Note:** memory goes through `@waggle/memory-contract`, implemented on local -> mem0 by `@waggle/memory-mem0` (ADR-0007). The React UI is **not** a package — it -> lives in `apps/web/src`. There is no `@waggle/ui` package. This document predates -> the community edition and is rewritten in P2.4. - -## Package Overview +## Overview ``` -waggle-os/ - apps/ - web/ # Main web app UI (React 19 + Vite + Tailwind 4 + base-ui/react) - browser-ext/ # Browser extension (unpacked; not an npm workspace) - packages/ - # Product packages (MIT) - agent/ # Agent loop, tools, sub-agents, workflows, trust, hooks, personas, evolution - core/ # Config, vault (secrets), cron, file store, telemetry, compliance/audit - server/ # Fastify API server, local + team routes, daemons, KVARK client, scheduler - shared/ # Shared types, Zod schemas, tiers, MCP catalog - marketplace/ # Marketplace catalog, installer, security gate, sync - optimizer/ # Prompt optimization (GEPA engine) - weaver/ # Memory consolidation daemon - waggle-dance/ # Swarm orchestration protocol - worker/ # Background task processing (BullMQ) - sdk/ # Plugin/skill SDK, capability packs, starter skills - cli/ # Command-line REPL - launcher/ # AI-tool launcher / dock backend - memory-contract/ # MemoryStore interface, record model, capability map - memory-mem0/ # MemoryStore on local mem0 - sidecar/ # Node.js sidecar for Tauri desktop app - app/ # Tauri 2.0 desktop shell (Rust + WebView2) — loads the apps/web build + Browser tab or Tauri window + ┌──────────────────────────┐ + │ Web UI (apps/web, React) │ + └────────────┬─────────────┘ + │ HTTP + Server-Sent Events, bearer session token + ▼ + ┌─────────────────────────────────────────────────────────────┐ + │ Sidecar: Fastify on Node.js, 127.0.0.1:3333 │ + │ (packages/server/src/local) │ + │ │ + │ agent runtime ─────── built-in provider proxy ──► provider │ + │ (packages/agent) (/v1/chat/completions) APIs or │ + │ Ollama │ + │ MemoryStore ──► @waggle/memory-mem0 ──► /memory/ │ + │ WaggleDB ─────► /waggle.db │ + │ vault ────────► /vault.json │ + │ connectors, channels, scheduler, marketplace, skills │ + └─────────────────────────────────────────────────────────────┘ ``` -## Package Details +Everything runs on the user's computer. The only outside calls are the ones the user sets up: +model providers, connectors, messaging channels, marketplace sources, downloads of local +models and the embedding model, and analytics if the user allows them. -### @waggle/memory-contract and @waggle/memory-mem0 +## Parts -`MemoryStore` (remember, search, list, get, update, forget, history) with its record -model and capability map, and its implementation on the open-source mem0 library, -running locally in the sidecar (ADR-0007). +### Web UI (`apps/web`) -### @waggle/core +React 19, TypeScript, Vite 6, Tailwind CSS 3.4 and Radix UI primitives. It is an application, +not a package. `npm run build` builds it into the root `dist/`, which the sidecar serves; the +desktop build uses `apps/web/dist`. -The foundation layer for the desktop/server runtime. Zero network dependencies. +The UI talks to the sidecar over HTTP, with Server-Sent Events for chat streaming and +notifications. Every `/api/*` request carries a bearer session token. A page served by the +sidecar gets the token from `GET /api/auth/session-token`, which answers same-origin requests +only; the desktop window gets it through a per-launch credential from the Tauri shell. -- **WaggleConfig**: Configuration management (`~/.waggle/config.json`). Provider keys, default model, team server config. -- **Vault**: AES-256-GCM encrypted secret storage. Stores API keys, connector credentials, and sensitive metadata. -- **MultiMind**: Manages personal + workspace minds simultaneously — routes searches to both and merges results. -- **FileStore**: Workspace filesystem access with a segment-boundary + symlink-aware containment guard and a secret deny-list (see the [threat model](../THREAT_MODEL.md)). -- **CronStore**: Schedule management for the cron service. CRUD on cron expressions with last/next run tracking. -- **ImportParser** (`memory-import.ts`): parses ChatGPT and Claude export files into importable knowledge items. -- **Telemetry / Compliance**: telemetry pipeline and compliance reporting (`compliance/`). +Product analytics (PostHog) start only after the user allows them, and only in builds that carry +a key in `VITE_POSTHOG_KEY` (ADR-0006). -### @waggle/agent +### Sidecar (`packages/server/src/local`) -The intelligence layer. Orchestrates tool execution, sub-agents, and workflows. +A Fastify server on Node.js. In development it runs from TypeScript source with tsx; for +release it is bundled into one `service.js` with esbuild. It listens on `127.0.0.1` port 3333 +by default (`WAGGLE_PORT`, `WAGGLE_HOST`), rejects requests whose `Host` header is not local, +and serves the web UI from `WAGGLE_FRONTEND_DIR` or the root `dist/`. -- **AgentLoop**: Core loop that sends messages to the LLM, parses tool calls, executes tools, and streams results. Supports up to 200 turns per conversation. -- **Tools (97+)**: Organized across 12 categories: - - System tools: `bash`, `read_file`, `write_file`, `edit_file`, `search_files`, `search_content`, `list_directory` - - Memory tools: `search_memory`, `save_memory`, `forget_memory` - - Web tools: `web_search`, `web_fetch` - - Git tools: `git_status`, `git_diff`, `git_log`, `git_commit` - - Plan tools: `create_plan`, `add_plan_step`, `execute_step`, `show_plan` - - Document tools: `generate_docx` - - Sub-agent tools: `spawn_agent`, `coordinate_agents` - - Skill tools: dynamically generated from installed skills - - KVARK tools: `kvark_search`, `kvark_ask_document`, `kvark_feedback`, `kvark_action` - - Team tools: `request_team_capability`, `assign_task`, `update_task` - - Audit tools: `audit_trail`, `trust_assessment` - - Connector tools: dynamically generated from connected services -- **CapabilityRouter**: Routes user intents to the appropriate tool or workflow based on context. -- **WorkflowComposer**: Dynamically composes multi-step workflows from templates. -- **Workflow Templates**: `research-team` (parallel research), `review-pair` (draft + review), `plan-execute` (plan + execute steps). -- **SubagentOrchestrator**: Manages sub-agent lifecycle -- spawning, monitoring, result collection. -- **CommandRegistry**: Slash command registration and execution (14 commands). -- **HookRegistry**: Event-driven hooks (before/after tool calls, session start/end, etc.). -- **Personas**: 8 predefined agent configurations with system prompts and tool presets. -- **TrustModel**: Assesses capability risk level, trust source, and approval class. -- **SkillRecommender**: Context-aware skill suggestions based on conversation content. +It holds: -### @waggle/server +- the HTTP routes (`routes/`) for chat, workspaces, sessions, memory, settings, vault, skills, + plugins, connectors, marketplace, automations and the rest; +- the **built-in provider proxy** (`routes/anthropic-proxy.ts`): an OpenAI-compatible + `/v1/chat/completions` endpoint that translates Claude requests to Anthropic's Messages API + and forwards requests for other providers to their OpenAI-compatible APIs. The agent loop and + memory extraction call cloud models through it (or through LiteLLM when the user runs it); +- the **managed Ollama runtime** (`managed-ollama-runtime.ts`): downloads the official Ollama + build, checks its SHA-256 and size, and runs it on `127.0.0.1:11434` with models in + `/models/ollama`; +- the scheduler for automations (`cron.ts`), notifications, held actions and approvals; +- messaging channels (`channels/`): Telegram, Discord, Slack, and WhatsApp through + `whatsapp-web.js` (ADR-0005); +- the memory store and background fact extraction (`memory-runtime.ts`, + `memory-extraction.ts`). -The API layer. Fastify server exposing 29 route modules. +An optional LiteLLM proxy runs only when `config.json` contains `"litellm": { "enabled": true }`, +on the user's own Python. Nothing else needs Python. -- **Local Server** (`src/local/`): Solo mode on localhost:3333. Routes for chat, workspaces, sessions, memory, settings, vault, skills, plugins, connectors, marketplace, cron, fleet, etc. -- **InstallAuditStore** (`src/local/install-audit.ts`): append-only capability-install trail (proposed / approved / installed / rejected / uninstalled) in `waggle.db`, shown in the Audit and Activity tabs. -- **Team Server** (`src/routes/`): Multi-user mode with PostgreSQL (Drizzle ORM), Redis, Clerk auth, and WebSocket presence. -- **SSE Streaming**: Chat responses stream via Server-Sent Events. -- **Anthropic Proxy**: Built-in `/v1/chat/completions` endpoint that translates OpenAI format to Anthropic API. -- **KVARK Client** (`src/kvark/`): HTTP facade for enterprise retrieval. User-level Bearer tokens. -- **Daemons**: Background processes (memory consolidation, proactive checks). -- **Scheduler** (`src/scheduler/`): LocalScheduler that ticks cron schedules and dispatches jobs. -- **ConnectorRegistry**: Registers and manages 29 native connectors. Generates agent tools from connected services. -- **Session Manager**: Manages parallel workspace sessions for Mission Control. -- **Notification System**: Event bus + SSE stream for real-time notifications (cron, approval, task, agent events). +### Agent runtime (`packages/agent`) -### apps/web (UI) +The agent loop (`agent-loop.ts`), the orchestrator that builds the system prompt and recalls +memory (`orchestrator.ts`), personas (`persona-data.ts`, `personas.ts`), tools, slash +commands (`commands/`), sub-agents and workflow templates, connectors (`connectors/`), MCP +client support (`mcp/`), the injection scanner and cost tracking (`cost-tracker.ts`). -The React UI is an application, not a package — it lives in `apps/web/src` -(React 19 + Vite + Tailwind 4 + base-ui/react). There is no `@waggle/ui` -package. The desktop binary (`app/`) loads the `apps/web` build. Representative -surfaces: +### Other packages -- **ChatArea**: Main conversation interface with streaming, tool cards, approval gates, and file upload. -- **MemoryBrowser**: Frame list with search, importance filters, and knowledge graph visualization. -- **WorkspaceHome**: Context-rich home screen with summary, decisions, threads, and suggestions. -- **Settings**: Tabbed settings (Models, Permissions, Vault, Appearance, Advanced). -- **Cockpit**: System dashboard showing health, schedules, runtime stats, connectors, trust audit. -- **Capabilities**: Skill/pack browser with install state and family grouping. -- **Events**: Tool event log with grouping and completion animations. -- **Onboarding**: First-run setup flow (API key, workspace creation, starter skills, import). +| Package | Role | +|---|---| +| `@waggle/core` | Config (`config.json`), the vault, `WaggleDB` (`waggle.db`), workspaces, file storage, embeddings, local telemetry, logger | +| `@waggle/shared` | Shared types, Zod schemas, the MCP server catalog | +| `@waggle/memory-contract` | The `MemoryStore` interface, record model, capability map, an in-memory store and the compatibility suite | +| `@waggle/memory-mem0` | `MemoryStore` on the open-source mem0 library | +| `@waggle/marketplace` | Marketplace catalog (SQLite with full-text search), installer, security scan, sync | +| `@waggle/sdk` | Skill and plugin SDK, starter skills, capability packs | +| `@waggle/cli` | A terminal REPL on the same memory store and `waggle.db` | +| `@waggle-ai/waggle` (`packages/launcher`) | The server installer (`install`, `run`, `doctor`, `update`, `uninstall`) | -### @waggle/marketplace +`packages/team-server`, `packages/waggle-dance` and `packages/worker` hold team mode. They stay +in the repository but are not built, tested in the root run, or shipped (ADR-0003). +`apps/browser-ext` is not part of the release. -Marketplace infrastructure. SQLite-based catalog with FTS5 search. +### Desktop shell (`app/`) -- **MarketplaceDB**: 120+ packages across skills, plugins, and MCP servers. Auto-seeded from bundled data. -- **MarketplaceInstaller**: Install/uninstall with dependency tracking. -- **SecurityGate**: Heuristic-based security scanner. Scans for dangerous patterns before install. -- **MarketplaceSync**: Syncs catalog from configured sources. -- **Enterprise Packs**: KVARK-dependent packs (only available with enterprise connection). +A Tauri 2 shell in Rust (`app/src-tauri`). There is no separate frontend in `app/`: the window +loads the `apps/web` build. The shell: -### @waggle/waggle-dance +- starts the bundled sidecar with the bundled Node.js, on port 3333 or a free port if 3333 is + taken, waits until it reports ready, and stops it (SIGTERM first) when the app quits; +- writes the sidecar's output to `/logs/service.log`; +- keeps one instance running, adds a tray icon, and toggles the window with Ctrl+Shift+W; +- exposes a short list of IPC commands, allowed explicitly in `app/src-tauri/capabilities/`. -Swarm orchestration protocol for multi-agent coordination. +## A chat turn -- **Protocol**: Message types for task assignment, status updates, and result collection. -- **Dispatcher**: Routes work to available agents based on capability and load. -- **HiveQuery**: Broadcast queries across multiple agents for parallel investigation. +1. The UI sends `POST /api/chat` with the message, workspace and session, and reads the reply as + a Server-Sent Events stream. +2. The orchestrator recalls memory for the message (see [memory.md](memory.md)) and builds the + system prompt from the core rules, the persona, active skills and workspace context. +3. The agent loop calls the model through the built-in proxy (local Ollama models are called + directly). When the model asks for a tool, + the loop checks the approval rules (some actions wait for the user to approve), runs the + tool, and scans the tool output for prompt injection before the model sees it. External + output is passed to the model as data, never as instructions. +4. Text, tool calls and approval requests stream to the UI as they happen. +5. The session is appended to `/workspaces//sessions/.jsonl`. Costs go into + the execution traces in `waggle.db`, and the finished turn is queued for background fact + extraction. -### @waggle/worker +## Data -Background task processing for team mode. +One data directory per user: `WAGGLE_DATA_DIR`, or `~/.waggle`. [install.md](install.md#where-your-data-lives) +lists what it contains. -- **BullMQ Integration**: Redis-backed job queues. -- **Execution Strategies**: Parallel (fan-out), sequential (pipeline), and coordinator (master-worker). -- **Agent Worker**: Real `runAgentLoop` execution in background worker processes. +- **Memory** goes only through `MemoryStore` (`@waggle/memory-contract`). The shipped store keeps + its SQLite files in `/memory/`. Embeddings come from an in-process model, so memory + works offline. See [memory.md](memory.md). +- **`waggle.db`** holds Waggle's own data, which is not memory: automations, notifications, + held actions, skill hashes, thumbs feedback, cost tracking, the install audit trail, the file + index, and the profile card and task list per scope. +- **The vault** (`vault.json`, key in `.vault-key`) keeps provider keys and connector credentials, + each encrypted with AES-256-GCM. +- Local telemetry (`telemetry.db`, off by default) and the audit log (`audit.db`) have their own + files. -## Data Flow +The SQLite files are opened with better-sqlite3, a native module (ADR-0008). -### Chat Message Flow +## Models -``` -User Input - --> POST /api/chat (Fastify route) - --> Workspace context loaded (memory, state, persona) - --> System prompt composed (core + persona + skills + context) - --> runAgentLoop() invoked - --> LLM API call (Anthropic proxy or direct) - --> Tool calls parsed and executed - --> Approval gate check (if sensitive) - --> Tool result returned - --> Memory auto-save (decisions, facts, preferences) - --> SSE events streamed to client - --> Session persisted to .jsonl file - --> UI renders streaming response +- **Provider keys.** The user enters keys in Settings → Models or in the Vault. The proxy calls + the provider directly from the user's computer. +- **Local models.** The Waggle-managed Ollama runtime, or an Ollama the user already runs + (`OLLAMA_HOST`, default `http://localhost:11434`). +- **Cost tracking.** `CostTracker` prices each call and the execution traces are kept in + `waggle.db` (ADR-0004). Settings → Models sets an optional daily budget. + +## Installer and server archive + +The server installer (`packages/launcher`, ADR-0009) is one self-contained script that uses only +Node.js built-in modules. It downloads or reads a server archive, checks its SHA-256, unpacks it +into a per-user program directory and writes a `waggle-server` command. The command runs the +archive's own Node.js, so the user's Node.js is needed only for the first `npx` call. + +A server archive (`waggle-server--.tar.gz`) contains the same sidecar payload +the desktop app ships, the built web UI under `web/`, and a `manifest.json` with the version, +target and Node.js version. `run` starts `service.js` with the environment the desktop shell +uses, plus `WAGGLE_FRONTEND_DIR` pointing at `web/`. + +## How a release is built + +Native modules (`better-sqlite3`, `onnxruntime-node`) must match the bundled Node.js, so each +target is built on its own system (ADR-0008). The desktop build scripts in `app/package.json` +run one staging chain into `app/src-tauri/resources/`: + +1. `scripts/bundle-node.mjs` downloads and verifies the pinned Node.js (22.23.2); +2. `scripts/build-sidecar.mjs` bundles the sidecar into `service.js`; +3. `scripts/bundle-native-deps.mjs` copies the native modules for the target; +4. `scripts/stage-sidecar-deps.mjs` stages the packages the bundle keeps external; +5. `apps/web` is built with Vite into `apps/web/dist`; +6. `tauri build` produces the installers: `.exe` (NSIS) and `.msi` on Windows, `.app` and + `.dmg` on macOS, `.deb` and `.AppImage` on Linux. + +Then `npm run build:server-archive` (`scripts/build-server-archive.mjs`) packs the same staged +payload and `apps/web/dist` into the server archive for that target, with a `.sha256` next to it. + +```bash +npm ci +(cd app && npm ci) +(cd app && npm run tauri:build:mac:arm64) # or tauri:build:mac:x64, tauri:build:win, tauri:build +npm run build:server-archive # set TARGET_ARCH=x64 after an x64 macOS build ``` -### Memory Flow +The Windows build must run on Windows; see [docs/oss/build-windows.md](oss/build-windows.md). +The first community release is unsigned ([docs/oss/signing.md](oss/signing.md)). The release +workflow for the community edition is not set up yet, so releases are built by hand on each +system. -``` -Conversation - --> Agent detects important information - --> save_memory tool called - --> FrameStore.add() writes to workspace .mind - --> Embeddings generated (sqlite-vec) - --> Knowledge graph updated (entity extraction) +Shipped dependencies are checked for licenses that allow MIT distribution +(`npm run check:licenses`, and `cargo deny` for the Rust crates), and their notices are in +[`THIRD_PARTY_NOTICES.md`](../THIRD_PARTY_NOTICES.md). -Later search: - --> search_memory tool called - --> MultiMind.search() queries personal + workspace minds - --> FTS5 + vector similarity results merged - --> Top results injected into agent context -``` +## Extension points -### Workspace Startup Flow +- **Skills**: Markdown files with YAML frontmatter in `/skills/`, added to the system + prompt when active. See [guides/capabilities.md](guides/capabilities.md). +- **Plugins and MCP servers**: installed from the marketplace or a local directory. +- **Connectors**: classes in `packages/agent/src/connectors/`, registered in + `packages/server/src/local/setup-connectors.ts`. A connected service adds tools to the agent. +- **Slash commands**: `CommandDefinition`s in `packages/agent/src/commands/`, registered with the + `CommandRegistry`. +- **Memory backends**: an implementation of `MemoryStore` that passes the compatibility suite in + `@waggle/memory-contract`. -``` -Open workspace - --> GET /api/workspaces/:id/context - --> Load workspace mind (MindDB) - --> Read recent memory frames - --> Extract decisions from memories - --> Read session files (titles, summaries) - --> Extract progress items (tasks, completions, blockers) - --> Build workspace state summary - --> Generate contextual suggested prompts - --> UI renders Home screen -``` +## Security -## Extension Points - -### Adding a New Tool - -1. Define the tool in the appropriate category under `packages/agent/src/` -2. Follow the `ToolDefinition` interface: name, description, parameters (JSON Schema), handler function -3. Register the tool in the agent's tool list -4. If the tool is sensitive, add it to the approval gate check list - -### Adding a New Connector - -1. Implement the `ConnectorCapability` interface in `packages/server/src/services/connectors/` -2. Define `id`, `name`, `service`, `authType`, `connect()`, `healthCheck()`, and `generateTools()` -3. Register in `packages/server/src/local/index.ts` with `connectorRegistry.register()` -4. The connector's tools are automatically available when credentials are in the vault - -### Adding a Skill - -Create a markdown file in `~/.waggle/skills/`. The skill content is appended to the agent's system prompt. Use YAML frontmatter for metadata: - -```markdown ---- -name: my-skill -permissions: - - read_file - - web_search ---- -# My Skill - -Instructions for the agent... -``` - -### Adding a Slash Command - -1. Create a `CommandDefinition` in `packages/agent/src/commands/` -2. Implement `name`, `aliases`, `description`, `usage`, and `handler` -3. Register with `registry.register()` in the appropriate registration function - -### Adding a Workflow Template - -1. Define a factory function that returns a `WorkflowTemplate` with steps -2. Register in the `WORKFLOW_TEMPLATES` map in `packages/agent/src/` -3. Each step defines a role, instructions, and optional tool restrictions - -## Storage Locations - -| Data | Location | Format | -|------|----------|--------| -| Personal memory | `~/.waggle/default.mind` | SQLite | -| Workspace memory | `~/.waggle/workspaces/{id}/workspace.mind` | SQLite | -| Sessions | `~/.waggle/workspaces/{id}/sessions/*.jsonl` | JSON Lines | -| Tasks | `~/.waggle/workspaces/{id}/tasks.jsonl` | JSON Lines | -| File registry | `~/.waggle/workspaces/{id}/files.jsonl` | JSON Lines | -| Config | `~/.waggle/config.json` | JSON | -| Vault | `~/.waggle/vault.db` | SQLite (encrypted values) | -| Marketplace | `~/.waggle/marketplace.db` | SQLite | -| Skills | `~/.waggle/skills/*.md` | Markdown | -| Plugins | `~/.waggle/plugins/` | Package directories | -| Permissions | `~/.waggle/permissions.json` | JSON | - -## Security Model - -- **Vault**: AES-256-GCM encryption for all secrets. Keys never stored in plain text after vault migration. -- **Approval Gates**: Sensitive tool executions require explicit user approval. -- **SecurityGate**: Marketplace installs scanned for dangerous patterns. CRITICAL severity always blocked. -- **Content Hashing**: SHA-256 hashes detect unauthorized skill modifications. -- **Audit Trail**: Every capability install, uninstall, and security decision is recorded. -- **Input Validation**: All route parameters validated against path traversal and injection. -- **YOLO Mode**: Opt-in auto-approval, disabled by default. +The threat model, the controls and the known gaps are in [`THREAT_MODEL.md`](../THREAT_MODEL.md). +In short: the user is trusted, external content is not. Connector input, tool output and recalled +memory pass the injection scanner; secrets live in the vault; sensitive tools wait for approval; +the sidecar answers only local requests that carry the session token. diff --git a/docs/CONTRIBUTING.md b/docs/CONTRIBUTING.md index 3f2bebf..6e4bc75 100644 --- a/docs/CONTRIBUTING.md +++ b/docs/CONTRIBUTING.md @@ -1,172 +1,3 @@ -# Contributing to Waggle +# Contributing -## Prerequisites - -- **Node.js 20+** (required) -- **Rust toolchain** (only for the desktop app -- `app/` package) -- **Docker** (only for team mode tests -- PostgreSQL and Redis) - -## Setup - -```bash -git clone waggle-os # the public repository location is not decided yet -cd waggle-os -npm install -``` - -## Running the App - -```bash -# Local server / sidecar (http://localhost:3333) -cd packages/server && npx tsx src/local/start.ts -# — or, from the repo root: npm run dev:server - -# Web app (http://localhost:8080) -npm run dev # — or: npm run dev:web - -# Desktop app (requires Rust) -cd app && npm run tauri dev - -# CLI REPL -cd packages/cli && npx tsx src/index.ts -``` - -## Running Tests - -Tests are the product's safety net. All PRs must pass the full test suite. - -```bash -# Run all tests (3000+ across 190+ files) -npx vitest run - -# Watch mode during development -npx vitest - -# Run tests for a specific package -npx vitest run packages/core - -# Run a specific test file -npx vitest run packages/agent/src/__tests__/tools.test.ts - -# Coverage report -npx vitest run --coverage -``` - -### Test Requirements - -- All existing tests must pass before submitting a PR -- New features require accompanying tests -- Bug fixes require a regression test -- Team mode tests require Docker services running (`docker compose up -d`) - -### Test Organization - -Tests live alongside source files in `__tests__/` directories or as `.test.ts` siblings. The monorepo uses a single Vitest config at the root. - -## Pull Request Process - -1. **Fork** the repository and create a feature branch from `main` -2. **Read the CLAUDE.md** for execution rules and product truths -3. **Make your changes** following the slice-based approach (one focused change per PR) -4. **Write tests** for new functionality -5. **Run the full test suite**: `npx vitest run` -6. **Verify the build**: `npx tsc --noEmit` -7. **Submit a pull request** against `main` -8. **Describe your changes**: what was added, what was preserved, what was tested - -### PR Title Format - -Use descriptive titles that indicate the type of change: - -- `feat: add Notion connector` -- new feature -- `fix: memory search scope filtering` -- bug fix -- `refactor: extract FrameStore from MindDB` -- code restructuring -- `test: add coverage for approval gates` -- test additions -- `docs: update API reference` -- documentation - -## Code Style - -### TypeScript - -- ESM modules (`"type": "module"` in package.json) -- Explicit imports (no barrel re-exports) -- Strict TypeScript (`strict: true`) -- Use `type` imports where possible: `import type { Foo } from './foo.js'` -- Include `.js` extensions in imports (ESM requirement) - -### File Organization - -- One concept per file where practical -- Co-locate tests: `foo.ts` and `__tests__/foo.test.ts` -- Route files export a Fastify plugin async function -- Types go in the `@waggle/shared` package if used across packages - -### Naming Conventions - -- Files: `kebab-case.ts` -- Types/Interfaces: `PascalCase` -- Functions/variables: `camelCase` -- Constants: `UPPER_SNAKE_CASE` -- Route handlers: descriptive comments with HTTP method and path - -### Error Handling - -- Route handlers catch errors and return appropriate HTTP status codes -- Non-critical operations use try/catch with empty catch (logging is acceptable) -- Critical operations throw with descriptive error messages -- Avoid swallowing errors silently in core logic - -## Package Structure - -### Adding to an Existing Package - -1. Add your source file in the appropriate directory -2. Export from the package's `index.ts` if it's a public API -3. Add tests in the `__tests__/` directory -4. Run `npx vitest run packages/` to verify - -### Key Directories - -``` -packages// - src/ - index.ts # Public exports - __tests__/ # Test files - package.json # Package metadata - tsconfig.json # TypeScript config -``` - -## Product Rules - -Read `CLAUDE.md` in the repo root for the full execution protocol. Key rules: - -- **Waggle is workspace-native** -- do not collapse into a global chat -- **Memory is a product primitive** -- do not treat it as decorative -- **Tool transparency matters** -- users see what the agent does -- **Approval gates are required** for sensitive operations -- **No scope reduction without approval** -- do not simplify features "for now" -- **Tests are part of the product** -- not an afterthought - -## Troubleshooting - -### Windows: sidecar fails to start with an esbuild platform error - -The Fastify sidecar runs through `tsx`, which uses esbuild. On a clean Windows -machine the platform-specific esbuild binary is sometimes not resolved, and -`npm run dev:server` (or `npx tsx src/local/start.ts`) fails with an error like -`Cannot find module @esbuild/win32-x64` or an esbuild version/host mismatch. - -Fix it by installing the matching Windows esbuild binary without adding it to -`package.json`: - -```bash -npm i @esbuild/win32-x64@0.28.0 --no-save -``` - -This affects **Windows only** — macOS and Linux resolve their esbuild binaries -normally. The version should match the esbuild your install resolved; `0.28.0` -is the known-good pin for the sidecar. - -## Questions? - -Open an issue for architecture questions, feature proposals, or bug reports. +Moved to [CONTRIBUTING.md](../CONTRIBUTING.md) in the repository root. diff --git a/docs/GETTING-STARTED.md b/docs/GETTING-STARTED.md deleted file mode 100644 index 09f9294..0000000 --- a/docs/GETTING-STARTED.md +++ /dev/null @@ -1,97 +0,0 @@ -# Getting Started with Waggle - -Welcome to Waggle — your AI operating system with persistent memory. - -## Quick Start - -### 1. Install Waggle for Windows - -Use only the signed Windows Solo `.exe` installer and SHA-256 identified by a **GO** [launch recommendation](production-readiness/09-LAUNCH_RECOMMENDATION.md). If that recommendation is not GO, no packaged desktop artifact is release-approved; use the source-development instructions in the repository instead. macOS desktop packaging and certification remain roadmap work. - -Run the installer and launch Waggle. The Windows Solo release contract does not require developer Node.js, Docker, Python, LiteLLM, administrator access, or a separate Ollama installation. - -### 2. Connect a model - -The first-launch wizard offers two supported paths: - -- **Private local model:** open **Local model**, choose **Install private runtime**, then download and verify the recommended model. Waggle keeps the managed runtime and model in its private data directory; no system Ollama or Docker installation is required. -- **Cloud provider:** open **API key**, choose a provider, and paste your own provider API key. Waggle validates it live when the provider is reachable, labels any unverified result, and stores it encrypted in Vault. - -At least one working local model or cloud provider is needed for chat. You can choose **I'll do this later**, but chat remains unavailable until a model is configured. - -External coding-agent authentication is separate from Waggle's model setup. Install each supported client yourself and use its own sign-in flow: Claude Code (`claude auth login --claudeai`), Codex (`codex login`), and Hermes' built-in provider OAuth (`hermes auth add openai-codex --type oauth`). Waggle uses the client's existing user session; do not paste or copy those credentials into Waggle. - -### 3. Create Your First Workspace -- Choose a template (Sales, Research, Legal, etc.) or start blank -- Pick a persona (Researcher, Writer, Analyst, etc.) -- Name your workspace - -### 4. Start Working -Type anything in the chat. Your agent can: -- Search the web (works immediately, no setup) -- Create Word documents with professional formatting -- Read and write files in your workspace -- Run shell commands (sandboxed) -- Run 15 workflow commands (type `/` to see them) -- Remember everything across sessions - -### 5. The Memory Magic -After your first conversation, try closing and reopening the workspace. -Type: "What do you remember about our discussion?" -Your agent remembers everything — decisions, context, preferences. - -## Key Features - -### Workflow Commands -Type `/` in chat to see all commands: -- `/research [topic]` — deep web research with synthesis -- `/draft [type]` — create documents with professional formatting -- `/plan [goal]` — structured planning with steps -- `/catchup` — get a summary of where things stand -- `/decide [question]` — structured decision analysis -- `/review [topic]` — review work or documents -- `/spawn [task]` — delegate work to a sub-agent -- `/skills` — browse and install new capabilities -- `/status` — workspace health and statistics -- `/memory` — explore what your agent remembers -- `/focus` — set the current work focus -- `/now` — quick workspace snapshot -- `/help` — see all available commands - -### Personas -Press `Ctrl+Shift+P` to switch personas. Each changes how your agent thinks: -- **Researcher**: deep investigation with citations -- **Writer**: document creation with formatting -- **Analyst**: data analysis with structured output -- **Planner**: project planning and task breakdown -- ...and more - -### Keyboard Shortcuts -- `Ctrl+K` — search everything -- `Ctrl+Shift+1` through `Ctrl+Shift+7` — switch views -- `Ctrl+Shift+P` — switch persona -- `/` — workflow commands - -### Memory View -Press `Ctrl+Shift+5` to browse everything your agent remembers. -Search, filter by type, and explore the knowledge graph. - -### Workspace Home -When you open a workspace, you see the "Workspace Now" overview: -- A summary of where things stand -- Recent decisions and open items -- Recent conversation threads -- Key memories - -Once you start chatting, click the **Workspace Overview** toggle above the messages to return to this view at any time. - -## Product Scopes - -Waggle uses four product tiers: -- **Trial** — 15 days of the Team experience, then Solo -- **Solo (Free)** — personal productivity with persistent memory -- **Team** — shared workspaces, WaggleDance, and governance -- **Enterprise** — sovereign KVARK deployment, compliance, and audit trails - -## Need Help? -Type `/help` in chat for a full command reference. diff --git a/docs/guides/capabilities.md b/docs/guides/capabilities.md index d1bcbdf..22b2823 100644 --- a/docs/guides/capabilities.md +++ b/docs/guides/capabilities.md @@ -1,211 +1,100 @@ -# Capabilities +# Skills, plugins and the marketplace -Capabilities are the extensions that make Waggle powerful: skills, plugins, MCP servers, workflow templates, hooks, and commands. This guide covers how to discover, install, and manage them. +Capabilities extend what the agent knows and can do. This guide covers the kinds of capability, +how to install them and how Waggle checks them. -## Capability Types +Most of the screens below appear at the **Everything** interface level (Settings → General). -| Type | What It Is | Storage | -|------|-----------|---------| -| **Skill** | Markdown file that extends the agent's system prompt with domain knowledge and instructions | `~/.waggle/skills/*.md` | -| **Plugin** | Structured package with a manifest that adds tools and behaviors | `~/.waggle/plugins/` | -| **MCP Server** | Model Context Protocol server that provides external tools | Configured in settings | -| **Workflow Template** | Multi-step agent orchestration (e.g., research-team, review-pair, plan-execute) | Built into `@waggle/agent` | -| **Hook** | Event-driven callback (before/after tool calls, session start/end, etc.) | Registered in agent state | -| **Command** | Slash command (`/research`, `/draft`, etc.) | Built into `@waggle/agent` | +## Kinds of capability -## Built-in Capability Packs +| Kind | What it is | Where it lives | +|---|---|---| +| Skill | A Markdown file with instructions and domain knowledge, added to the agent's prompt when active | `/skills/*.md` | +| Plugin | A package with a manifest that adds tools and skills | `/plugins/` | +| MCP server | A Model Context Protocol server that gives the agent external tools | Settings, MCP Hub | +| Workflow | A multi-agent recipe, such as `research-team`, `review-pair` or `plan-execute` | Built in, or created with `/workflow` | +| Command | A slash command such as `/research` or `/draft` | Built in ([reference](../reference/commands.md)) | -Waggle ships with 5 curated packs. Each pack bundles several skills that work together. +## Starter skills and capability packs -### Research Workflow +Waggle ships with starter skills in seven families: Writing & Docs, Research & Analysis, +Decision Support, Planning & Organization, Communication, Code & Engineering, and Creative & +Ideation. Install them one by one in **Skills Hub**, or as packs: -Skills for investigation and synthesis. Includes `research-synthesis`, `explain-concept`, and the `research-team` multi-agent workflow. +| Pack | Skills | +|---|---| +| Research Workflow | `research-synthesis`, `explain-concept`, `research-team` | +| Writing Suite | `draft-memo`, `compare-docs`, `extract-actions` | +| Planning Master | `daily-plan`, `task-breakdown`, `plan-execute`, `decision-matrix` | +| Team Collaboration | `catch-up`, `status-update`, `meeting-prep`, `review-pair` | +| Decision Framework | `decision-matrix`, `risk-assessment`, `retrospective` | -### Writing Suite +Installing a pack skips skills that are already installed. The "Team Collaboration" pack is a +set of skills for working with other people; it does not need team mode. -Skills for document creation and editing. Includes `draft-memo`, `compare-docs`, and `extract-actions`. +## Writing your own skill -### Planning Master - -Skills for task decomposition and execution. Includes `daily-plan`, `task-breakdown`, and the `plan-execute` workflow. - -### Team Collaboration - -Skills for coordination and communication. Includes `catch-up`, `status-update`, and `meeting-prep`. - -### Decision Framework - -Skills for structured decision-making. Includes `decision-matrix`, `risk-assessment`, and `retrospective`. - -## Installing Packs - -### From the UI - -1. Navigate to **Capabilities** in the sidebar (or click the grid icon) -2. Click **Browse Packs** -3. Each pack shows its skills, install state (available / incomplete / complete), and description -4. Click **Install** to install all skills in the pack at once -5. Already-installed skills are skipped - -### From the API - -```bash -# Install the Research Workflow pack -curl -X POST http://localhost:3333/api/skills/capability-packs/research-workflow - -# Install a single skill from the starter pack -curl -X POST http://localhost:3333/api/skills/starter-pack/draft-memo -``` - -## Installing Individual Skills - -### From the Starter Catalog - -1. Go to **Capabilities** > **Skills** -2. Browse by family: Writing & Docs, Research & Analysis, Decision Support, Planning & Organization, Communication, Code & Engineering, Creative & Ideation -3. Click **Install** on any skill -4. The skill is copied to `~/.waggle/skills/` and loaded immediately - -### Creating Custom Skills - -Skills are markdown files. Create a file in `~/.waggle/skills/`: +A skill is a Markdown file in `/skills/` (by default `~/.waggle/skills/`). Optional YAML +frontmatter gives it a name and other metadata: ```markdown -# My Custom Skill +--- +name: weekly-report +description: Write the weekly status report in our format +--- +# Weekly report -Instructions for the agent when this skill is active. - -## When to Use -- Trigger conditions - -## How to Respond -- Response format guidelines -- Quality standards +When asked for the weekly report: +- Start with the three most important changes. +- List risks with an owner each. +- Keep it under one page. ``` -Or use the API: - -```bash -curl -X POST http://localhost:3333/api/skills \ - -H "Content-Type: application/json" \ - -d '{"name": "my-skill", "content": "# My Skill\n\nInstructions here."}' -``` +You can also create and edit skills in **Skills Hub**. `/skills` lists the active skills in a +chat. ## Marketplace -The marketplace catalog contains 120+ packages across skills, plugins, and MCP servers from curated sources. +**Marketplace** lists skills, plugins and MCP servers from curated sources. Search it, read a +package's details and install it. `/marketplace search ` and +`/marketplace install ` do the same from a chat; `/marketplace sync` refreshes the catalog +from its sources. -### Searching +### Security scan -```bash -# Search by keyword -curl "http://localhost:3333/api/marketplace/search?query=research&limit=10" +Every marketplace install is scanned first for dangerous patterns, such as reading credentials, +unexpected network calls or destructive file operations. The result decides what happens: -# Filter by type -curl "http://localhost:3333/api/marketplace/search?type=skill&category=analysis" -``` +| Severity | Result | +|---|---| +| Clean, low | The install proceeds | +| Medium | The install proceeds with warnings | +| High | Blocked unless you confirm that you accept the risk | +| Critical | Always blocked | -Or use the `/marketplace search ` slash command in any workspace. +Installs, uninstalls and security decisions are recorded in the install audit trail in +`waggle.db`. -### Installing from Marketplace - -```bash -# Install by package ID -curl -X POST http://localhost:3333/api/marketplace/install \ - -H "Content-Type: application/json" \ - -d '{"packageId": 42}' -``` - -Or use `/marketplace install `. - -### Security Scanning - -Every marketplace install goes through the **SecurityGate** scanner. Results determine whether the install proceeds: - -| Severity | Action | -|----------|--------| -| CLEAN | Install proceeds immediately | -| LOW | Install proceeds, logged to audit trail | -| MEDIUM | Install proceeds with warnings in response | -| HIGH | Blocked unless `force=true` (override logged to audit) | -| CRITICAL | Always blocked, cannot override | - -### Syncing the Catalog - -```bash -# Sync from all configured sources -curl -X POST http://localhost:3333/api/marketplace/sync -``` - -Or use `/marketplace sync`. - -## Trust Model - -Waggle uses a multi-layer trust model: - -1. **Source trust** -- starter-pack skills are trusted by default; marketplace and user-created skills get assessed -2. **Content analysis** -- heuristic scan for dangerous patterns (credential access, network calls, file system mutations) -3. **Risk level** -- low, medium, high, critical based on what the capability does -4. **Approval class** -- standard (auto-approve), review (log), force-override (user acknowledged risk), blocked -5. **Content hash** -- SHA-256 hash of each skill file, checked at startup to detect unauthorized modifications - -### Audit Trail - -Every install, uninstall, and security decision is recorded in the audit store. View it: - -- **UI**: Capabilities > Cockpit > Trust Audit section -- **API**: `GET /api/audit/installs` - -### Hash Verification - -At startup, Waggle checks each installed skill's content hash against the stored hash. If a skill file was modified outside of Waggle, it is flagged in the hash status: - -```bash -curl http://localhost:3333/api/skills/hash-status -``` +If a skill or plugin you installed from the marketplace does not show up in the agent, restart +Waggle. The marketplace installer's reload request does not reach the running server yet. ## Plugins -Plugins are structured packages with manifests that can add tools and extend agent behavior. +A plugin is a directory with a manifest. Install one from the marketplace or from a local +directory, then enable or disable it. A disabled plugin stays installed but adds nothing to the +agent. -### Installing a Plugin +## MCP servers -```bash -curl -X POST http://localhost:3333/api/plugins/install \ - -H "Content-Type: application/json" \ - -d '{"sourceDir": "/path/to/my-plugin"}' -``` +**MCP Hub** shows the installed Model Context Protocol servers with their state, lets you start, +stop, test and remove them, and installs new ones from its catalog or from a server you +describe yourself. Catalog installs go through the same security scan as the marketplace. -### Enabling / Disabling +## Trust checks -```bash -# Enable -curl -X POST http://localhost:3333/api/capabilities/plugins/my-plugin/enable - -# Disable -curl -X POST http://localhost:3333/api/capabilities/plugins/my-plugin/disable -``` - -### Uninstalling - -```bash -curl -X DELETE http://localhost:3333/api/plugins/my-plugin -``` - -## Viewing Capability Status - -The **Cockpit** (accessible from the sidebar) shows a live dashboard of all capabilities: - -- **Plugins**: name, state, tool count, skill count -- **MCP Servers**: name, state, healthy status, tool count -- **Skills**: name, content length -- **Tools**: total count (native + plugin + MCP) -- **Commands**: all registered slash commands -- **Hooks**: registered count, recent activity log -- **Workflows**: available templates with step counts - -API equivalent: - -```bash -curl http://localhost:3333/api/capabilities/status -``` +- **Source**: starter skills are trusted; marketplace and your own skills are assessed. +- **Content scan**: the heuristic scan described above. +- **Approval**: capabilities that ask for risky actions go through the same approval rules as + any other tool call (Settings → Permissions). +- **Content hash**: Waggle records a SHA-256 hash of each installed skill and checks it at + startup, so a skill changed outside Waggle is flagged. diff --git a/docs/guides/connectors.md b/docs/guides/connectors.md index b40be2f..ad02591 100644 --- a/docs/guides/connectors.md +++ b/docs/guides/connectors.md @@ -1,195 +1,98 @@ -# Connectors +# Connectors and channels -Connectors let Waggle interact with external services -- GitHub, Slack, Google, Jira, and 25 others. Credentials are stored in the encrypted vault, and sensitive operations go through approval gates. +**Connectors** let the agent work with services you use, such as GitHub, Slack or Google Drive. +**Channels** let you talk to Waggle from a messaging app. Credentials for both are kept in the +encrypted Vault on your computer, and actions that change something go through approvals. -## Vault Setup +## The Vault -Before connecting any service, you need to understand the vault. The vault is Waggle's encrypted credential store, using AES-256-GCM encryption. It lives at `~/.waggle/vault.db` and is the single source of truth for all secrets. +The Vault stores provider keys, connector credentials and other secrets in +`/vault.json`. Each entry is encrypted with AES-256-GCM using a key kept in +`/.vault-key`. Open **Vault** to see the stored secrets (names only), add one, reveal a +value or delete it. -### Adding a Secret to the Vault +Provider keys added in Settings → Models land in the Vault too. -**From the UI:** -1. Go to **Settings** > **Vault** -2. Click **Add Secret** -3. Enter the name (e.g., `GITHUB_TOKEN`) and value -4. Click **Save** +## Native connectors -**From the API:** -```bash -curl -X POST http://localhost:3333/api/vault \ - -H "Content-Type: application/json" \ - -d '{"name": "GITHUB_TOKEN", "value": "ghp_your_token_here"}' -``` +Waggle has 30 built-in connectors: -### Suggested Keys +| Area | Connectors | +|---|---| +| Code | GitHub, GitLab, Bitbucket | +| Chat | Slack, Discord, Microsoft Teams | +| Email and calendar | Gmail, Google Calendar, Outlook (calendar and email), Email via SendGrid | +| Documents and files | Google Docs, Google Drive, Google Sheets, OneDrive, Microsoft OneNote, Dropbox, Notion, Confluence, Obsidian | +| Projects | Jira, Linear, Asana, Trello, Monday.com, Airtable | +| CRM | HubSpot, Salesforce, Pipedrive | +| Data | PostgreSQL | +| Other services | Composio, a bridge to many more services | -The vault suggests common API key names that are not yet configured: +### Connect a service -- `ANTHROPIC_API_KEY` -- LLM provider -- `OPENAI_API_KEY` -- LLM provider -- `GITHUB_TOKEN` -- GitHub API -- `SLACK_BOT_TOKEN` -- Slack Bot -- `JIRA_API_TOKEN` -- Jira Cloud -- `GOOGLE_API_KEY` -- Google services -- `TAVILY_API_KEY` -- Web search -- `BRAVE_API_KEY` -- Web search -- `SENDGRID_API_KEY` -- Email sending -- `GOOGLE_CALENDAR_TOKEN` -- Calendar access +1. Open **Connector Hub** (at the Everything interface level). +2. Choose the service and **Connect**. +3. Enter what the form asks for: a token, an API key, an email and token, a file path for + Obsidian, or a connection string for PostgreSQL. The form links to where the service issues + it. -## Native Connectors +When a service is connected, the agent gets tools for it, for example creating issues and +pull requests on GitHub or searching pages in Confluence. `/connectors` lists the connected +services in a chat. -Waggle registers 29 native connectors at startup. Each connector has a defined auth type and generates agent tools when credentials are available. +**Disconnect** removes the credentials from the Vault. Connector Hub also shows each +connector's health and a history of what was done with it. -| # | Connector | Service | Auth Type | -|---|-----------|---------|-----------| -| 1 | GitHub | GitHub API | Bearer token (PAT) | -| 2 | Slack | Slack Bot API | Bot token | -| 3 | Jira | Jira Cloud | Basic auth (email + API token) | -| 4 | Email (SendGrid) | SendGrid API | API key | -| 5 | Google Calendar | Google Calendar API | OAuth token | -| 6 | Discord | Discord Bot API | Bot token | -| 7 | Linear | Linear API | API key | -| 8 | Asana | Asana API | Bearer token | -| 9 | Trello | Trello API | API key | -| 10 | Monday | Monday.com API | API key | -| 11 | Notion | Notion API | Integration token | -| 12 | Confluence | Atlassian API | Basic auth | -| 13 | Obsidian | Local vault | File path | -| 14 | HubSpot | HubSpot API | Bearer token | -| 15 | Salesforce | Salesforce API | OAuth token | -| 16 | Pipedrive | Pipedrive API | API key | -| 17 | Airtable | Airtable API | Bearer token | -| 18 | GitLab | GitLab API | Bearer token (PAT) | -| 19 | Bitbucket | Bitbucket API | App password | -| 20 | Dropbox | Dropbox API | OAuth token | -| 21 | PostgreSQL | PostgreSQL | Connection string | -| 22 | Gmail | Gmail API | OAuth token | -| 23 | Google Docs | Google Docs API | OAuth token | -| 24 | Google Drive | Google Drive API | OAuth token | -| 25 | Google Sheets | Google Sheets API | OAuth token | -| 26 | MS Teams | Microsoft Graph API | OAuth token | -| 27 | Outlook | Microsoft Graph API | OAuth token | -| 28 | OneDrive | Microsoft Graph API | OAuth token | -| 29 | Composio | Composio Bridge | API key | +Composio keeps the credentials for the services behind it on Composio's side, not in Waggle's +Vault. -## Connecting a Service +## Approvals -### From the UI +When the agent wants to do something with an effect outside the chat, such as sending an email, +posting a message, creating a ticket or pushing a commit, it pauses and shows an approval card +with the tool and its input. Nothing happens until you choose **Approve** or **Deny**. -1. Open the **Cockpit** from the sidebar -2. Scroll to the **Connectors** section -3. Click **Connect** on the service you want -4. Enter your credentials (token, API key, or OAuth details) -5. Click **Save** +**Settings** → **Permissions** sets how often Waggle asks: -The connector status changes from "Disconnected" to "Connected". +| Mode | Behavior | +|---|---| +| Ask every time | Approve every write, edit and other action that changes something | +| Ask only for risky | Writes and edits pass; pushes, installs and cross-workspace actions still ask | +| Never ask | Everything passes except a fixed list of critical actions | -### From the API +## Channels -```bash -# Connect GitHub with a personal access token -curl -X POST http://localhost:3333/api/connectors/github/connect \ - -H "Content-Type: application/json" \ - -d '{"token": "ghp_your_token_here"}' +**Settings** → **Channels** connects messaging apps, so you can message Waggle from them. -# Connect Jira with email + API token -curl -X POST http://localhost:3333/api/connectors/jira/connect \ - -H "Content-Type: application/json" \ - -d '{"apiKey": "your_jira_token", "email": "you@company.com"}' -``` +| Channel | What you need | +|---|---| +| Telegram | A bot token from @BotFather | +| Discord | A bot token from the Discord developer portal | +| Slack | An app token (Socket Mode) and a bot token | +| WhatsApp | A phone to pair by QR code | -### Disconnecting +### WhatsApp -```bash -curl -X POST http://localhost:3333/api/connectors/github/disconnect -``` +WhatsApp runs through `whatsapp-web.js`, an unofficial client that drives WhatsApp Web in a +browser. WhatsApp does not allow this and may block the account: link a secondary number, not +your personal one. -This removes the credentials from the vault and any associated metadata. +- Waggle uses Chrome, Edge or Chromium if one is installed (or the browser named in + `CHROME_PATH`). Otherwise it downloads Chrome for Testing (about 170 MB) into + `/browsers` the first time you start WhatsApp. On Linux on ARM, install Chromium + yourself. +- Start the channel, then scan the pairing QR code with WhatsApp on your phone (Linked + devices). +- The session is kept as a browser profile in `/channels/whatsapp/session`. It is + protected by file permissions, not by the Vault. +- Only text messages are handled. Voice, video and other media are ignored. -## Checking Connector Health +## Troubleshooting -Each connector can report its health status: - -```bash -curl http://localhost:3333/api/connectors/github/health -``` - -Response: -```json -{ - "id": "github", - "name": "GitHub", - "status": "connected", - "lastChecked": "2026-03-19T10:30:00.000Z", - "tokenExpiresAt": null -} -``` - -Possible statuses: `connected`, `disconnected`, `expired`, `error`. - -## Connector Tools - -When a connector is connected (credentials in vault), the agent gains tools for that service. For example, connecting GitHub gives the agent: - -- Create issues -- List repositories -- Read file contents -- Create pull requests - -The exact tools depend on the connector implementation. Tools appear in the agent's tool list and are visible in the Cockpit capabilities dashboard. - -## Approval Gates - -Sensitive connector operations go through approval gates. When the agent wants to: - -- Send an email -- Create a Jira ticket -- Post a Slack message -- Push a git commit - -It pauses and shows an approval card in the chat. You see: - -- **Tool name** (e.g., `send_email`) -- **Input** (e.g., recipient, subject, body) -- **Approve** or **Deny** buttons - -The agent waits for your decision before proceeding. This prevents accidental or unwanted actions. - -### YOLO Mode - -If you trust the agent fully, you can enable **YOLO Mode** in Settings > Permissions. This auto-approves all tool executions without asking. You can also set per-workspace overrides for specific tools. - -## Composio Bridge - -The 29th connector, Composio, is a bridge to 250+ additional services. If you have a Composio account: - -1. Add your Composio API key to the vault -2. Connect the Composio connector -3. The agent gains access to Composio-managed services - -Composio manages its own credential lifecycle. Note: Composio credentials are stored and managed by Composio's infrastructure, not in Waggle's local vault. - -## Enterprise Connectors (KVARK) - -For enterprise deployments, KVARK provides an additional layer of governed connectors with: - -- 28+ document connectors (SharePoint, Box, Google Drive, etc.) -- Permission-aware retrieval (respects source system ACLs) -- Semantic search with reranking -- Full audit trail - -Enterprise connectors require a KVARK connection. See the [API Reference](../reference/api.md) for the `/api/marketplace/enterprise-packs` endpoint. - -## Troubleshooting Connectors - -### "Connector not found" -The connector ID must match exactly (lowercase). Check `GET /api/connectors` for the full list. - -### "Vault not available" -The vault database could not be initialized. Check that `~/.waggle/` is writable and the vault.db file is not corrupted. - -### Token Expired -Some OAuth-based connectors (Google, Microsoft) have tokens that expire. Re-connect the service to refresh. - -### No Tools Appearing -After connecting, the agent tools are regenerated on the next chat message. If tools still do not appear, restart the server. +- **A connector shows as disconnected after a restart.** Check its health in Connector Hub. If + the token expired (common with Google and Microsoft tokens), connect it again with a new one. +- **The agent has no tools for a connected service.** Tools are rebuilt on the next message. If + they still do not appear, restart Waggle. +- **"Vault not available".** Check that the data directory is writable. If `vault.json` cannot + be read, Waggle copies it to `vault.json.bak` and treats the Vault as empty; you then need to + enter the secrets again. diff --git a/docs/guides/getting-started.md b/docs/guides/getting-started.md index 06c5866..e59241c 100644 --- a/docs/guides/getting-started.md +++ b/docs/guides/getting-started.md @@ -1,211 +1,131 @@ -# Getting Started with Waggle +# Getting started -This guide walks you through installing Waggle, configuring your API key, creating your first workspace, having your first conversation, and understanding how memory works. +This guide takes you from a fresh install to your first conversation, and explains how Waggle +remembers things between chats. -## Prerequisites +## 1. Install -- **Node.js 20+** (for local server and CLI) -- **An LLM API key** (Anthropic recommended; OpenAI, Google, and 100+ others supported) -- **Rust toolchain** (only if building the desktop app from source) +Install the desktop app or the server installer as described in [install.md](../install.md). Then +open Waggle: start the desktop app, or run `waggle-server`, which opens +http://localhost:3333 in your browser. -## Installation +## 2. Go through the setup -### Option 1: One-line self-host (Linux / macOS) +The first launch walks you through a short setup: -Best for a VPS or homelab, where you want a headless server rather than the desktop app: +1. **Who are you**: your name, role and the kind of work you do. Waggle keeps this in your profile + on this computer and uses it to tailor answers. +2. **Model**: add a model (next section). You can skip this, but chat stays unavailable until a + model is set up. +3. **Pick a starting point**: a template for your first workspace, such as Research Hub, Sales + Pipeline, Engineering or Legal & Compliance, or **Blank Workspace**. Each template comes with a + matching persona. +4. **What's first?**: type a first task for the agent, or pick a suggestion. -```bash -curl -fsSL /main/install.sh | bash # the public repository location is not decided yet -``` +## 3. Connect a model -The installer checks prerequisites (Node.js 20+, git — no sudo), clones the repo, builds the packages and web UI, then starts the sidecar and prints its URL (`http://127.0.0.1:3333`). A short wizard (all Enter-defaulted) lets you change the install dir, port, and data dir; add `--yes` to accept every default non-interactively. It boots in echo mode with zero API keys, so the UI works right away — add a provider key later under **Settings → API Keys**. +Waggle needs at least one working model. Choose one of two ways, in the setup or later in +**Settings** → **Models**: -Manage the server afterward with `scripts/waggle-server.sh {start|stop|status|logs}`. Windows users should use the desktop app (Option 2) — the one-line installer targets Linux and macOS only. +- **API key**: choose a provider (Anthropic, OpenAI, Google, DeepSeek, xAI, Mistral, Alibaba + Qwen, MiniMax, Zhipu GLM, Moonshot Kimi, Perplexity or OpenRouter) and paste your key. Waggle + checks it with the provider when it can reach it, and stores it encrypted in the **Vault** on + this computer. +- **Local model**: choose **Install private runtime**. Waggle downloads the official Ollama + runtime, checks it and runs it from its own data directory. Then download a model; Waggle + suggests one that fits your computer. No separate Ollama install, Docker or administrator + rights are needed. If you already run Ollama, Waggle finds it. -### Option 2: Desktop App (Recommended) +In **Settings** → **Models** you also choose the default model, a fallback model and an optional +daily budget. -Download the latest release for your platform from the repository's Releases page (the public repository location is not decided yet). +## 4. Your first conversation -**Windows**: Run the `.msi` installer. Waggle appears in your Start menu. +Type in the chat. The agent streams its answer, and you see what it does along the way: -**macOS**: Open the `.dmg` and drag Waggle to Applications. +- **Tool steps**, such as reading a file, searching the web or saving a memory. +- **Approval requests** for actions that change something outside the chat, such as writing a + file or sending a message. The agent waits until you approve or deny. Settings → + **Permissions** sets how often Waggle asks. +- **Memory**: when you tell the agent something worth keeping, it saves it. -The desktop app bundles a Node.js sidecar that runs the local server automatically. +Good first messages: -### Option 3: CLI / Web - -If you prefer a browser-based interface or want to run Waggle without the desktop shell: - -```bash -# Clone the repository -git clone waggle -cd waggle -npm install - -# Start the local server -cd packages/server -npx tsx src/local/start.ts -``` - -The server starts on http://localhost:3333. Open it in any browser. - -### Option 4: CLI REPL - -For a terminal-native experience: - -```bash -cd packages/cli -npx tsx src/index.ts -``` - -## Step 1: Add Your API Key - -Waggle needs at least one LLM provider key to function. Anthropic (Claude) is recommended. - -### Desktop / Web UI - -1. Open **Settings** (gear icon in the sidebar or press `Ctrl+,`) -2. Go to the **Models** tab -3. Click **Add Provider** -4. Select **Anthropic** and paste your API key (starts with `sk-ant-`) -5. Click **Save** - -Your key is stored in the local vault (AES-256-GCM encrypted), never sent to Waggle's servers. - -### CLI / Config File - -Edit `~/.waggle/config.json`: - -```json -{ - "defaultModel": "claude-sonnet-4-6", - "providers": { - "anthropic": { - "apiKey": "sk-ant-your-key-here", - "models": ["claude-sonnet-4-6", "claude-haiku-3"] - } - } -} -``` - -### Verify Your Key - -In Settings > Models, click **Test Key**. A green checkmark means you are ready. - -## Step 2: Create Your First Workspace - -Workspaces are the core organizational unit in Waggle. Each workspace has its own memory, sessions, files, and context. Think of a workspace as a dedicated brain for a project or topic. - -1. Press **Ctrl+N** or click **New Workspace** in the sidebar -2. Enter a name (e.g., "Product Research" or "Q1 Planning") -3. Choose a group (e.g., "Work", "Personal", "Learning") -4. Optionally select a persona (Researcher, Writer, Coder, etc.) -5. Optionally link a directory on disk for file-aware operations -6. Click **Create** - -You land on the **Workspace Home** screen. It shows a summary, suggested prompts, and recent threads. - -## Step 3: Your First Conversation - -Type a message in the input area. Here are good first messages: - -- "Tell me about this project so I can remember it" -- "Help me think through what to work on first" +- "Here is what this project is about: …" +- "Help me plan what to work on first." - "What can you do in this workspace?" -The agent responds with streaming output. You will see: +### Slash commands -- **Tool calls** shown as collapsible cards (file reads, web searches, memory saves) -- **Memory saves** happening automatically when the agent learns something important -- **Approval gates** for sensitive operations (the agent asks before executing risky actions) +Type `/` in the chat to see the commands, for example `/catchup` for a summary of where the +workspace stands, `/research `, `/draft [topic]` or `/plan `. `/help` lists +them all; the [commands reference](../reference/commands.md) describes each one. -### Try a Slash Command +### Files -Type `/catchup` to get a workspace restart summary. Type `/help` to see all 14 commands. +Attach a file to the chat to let the agent read it: PDF, Word (`.docx`), PowerPoint (`.pptx`), +Excel (`.xlsx`), CSV, images, ZIP archives and text or code files, up to 10 MB each. -### Upload a File +## 5. How memory works -Click the paperclip icon or drag a file into the chat. Waggle supports: +Waggle keeps two kinds of memory, both on your computer: -- **Documents**: PDF, DOCX, PPTX -- **Spreadsheets**: XLSX, CSV -- **Images**: PNG, JPG, GIF, WebP, SVG -- **Code**: Any text-based source file -- **Archives**: ZIP (lists contents) +- **Personal memory** follows you across all workspaces: who you are, your preferences, facts + that are not tied to one project. +- **Workspace memory** belongs to one workspace: decisions, findings and project context. -Files are ingested into workspace memory and available to the agent. +Before it answers, the agent recalls what is relevant from personal memory and the current +workspace. Memories get there in three ways: -## Step 4: Understanding Memory +- **You ask**: "Remember that we chose PostgreSQL for the new service." +- **The agent decides** something is worth keeping and saves it. +- **Background extraction**: every few messages, Waggle picks out facts about you, such as your + role or how you like answers, and saves them to personal memory. Turn this off in **Settings** + → **General** → **Memory**. -Memory is a core product primitive, not a side feature. Two memory layers work together: +Open **Memory** to browse, search, correct, confirm or forget memories. `/memory ` searches +from the chat. More detail: [memory.md](../memory.md). -### Personal Mind (`~/.waggle/default.mind`) +## 6. Come back later -Your personal knowledge base. Facts about you, your preferences, recurring context. Shared across all workspaces. +When you return to a workspace: -### Workspace Mind (`~/.waggle/workspaces/{id}/workspace.mind`) +- **Home** shows recent activity and suggests where to continue; +- `/catchup` gives a briefing on where the workspace stands; +- your earlier chats are listed in the workspace, ready to resume; +- the agent recalls the workspace's memory before its first answer. -Project-specific memory. Decisions, research findings, architectural choices, meeting notes. Scoped to one workspace. +## Personas -### How Memory Works +A persona tunes how the agent works: its instructions, preferred tools and suggested commands. +Waggle ships with personas such as Researcher, Writer, Analyst, Coder, Project Manager, Executive +Assistant, Marketer, Legal Counsel and General Purpose. Press `Ctrl+Shift+P` (`Cmd+Shift+P` on +macOS) to switch. See [workspaces.md](workspaces.md#personas). -1. **Auto-save**: The agent automatically saves important information during conversations -- decisions, facts, preferences, and findings. -2. **Explicit save**: You can say "Remember that we decided to use PostgreSQL" and the agent writes it to the appropriate mind. -3. **Memory search**: The agent searches memory before responding, giving you continuity across sessions. -4. **Memory browser**: Open the Memory tab in any workspace to browse, search, and manage stored frames. +## Interface levels -### Memory Frames +**Settings** → **General** offers three interface levels: **Essential**, **Standard** and +**Everything**. Essential shows the basics (Home, Chat, Memory, Files, Vault, Settings). +Everything adds agents, skills, connectors, the marketplace, automations and the system tools. -Each memory unit is a "frame" with: -- **Content**: The actual information -- **Importance**: critical, important, normal, temporary -- **Frame type**: Information (I), Decision (D), Preference (P), etc. -- **Timestamp**: When it was created -- **Access count**: How often it has been retrieved +## Keyboard shortcuts -## Step 5: Return Tomorrow - -When you come back to a workspace, Waggle gives you an instant catch-up: - -1. **Workspace Home** shows a summary of what happened, recent decisions, and suggested next prompts -2. **Type `/catchup`** for a detailed briefing -3. **Resume a thread** by clicking any recent session in the sidebar -4. **Context is automatic** -- the agent loads relevant memory before its first response - -This is the core daily-use loop: - -> Open workspace -> instant context -> real work help -> memory-first response -> visible progress -> return later without losing thread - -## Next Steps - -- **[Workspaces Guide](workspaces.md)** -- Workspace types, switching, home screen, personas -- **[Capabilities Guide](capabilities.md)** -- Install skill packs and browse the marketplace -- **[Connectors Guide](connectors.md)** -- Connect GitHub, Slack, Google, and 26 other services -- **[Commands Reference](../reference/commands.md)** -- All 14 slash commands with examples -- **[Team Mode Guide](team-mode.md)** -- Set up shared workspaces for your team - -## File Locations - -| Item | Path | -|------|------| -| Config | `~/.waggle/config.json` | -| Personal mind | `~/.waggle/default.mind` | -| Workspace minds | `~/.waggle/workspaces/{id}/workspace.mind` | -| Session logs | `~/.waggle/workspaces/{id}/sessions/*.jsonl` | -| Installed skills | `~/.waggle/skills/*.md` | -| Installed plugins | `~/.waggle/plugins/` | -| Marketplace DB | `~/.waggle/marketplace.db` | -| Vault (encrypted) | `~/.waggle/vault.db` | -| Server logs | Console output (stdout) | - -## Keyboard Shortcuts +On macOS use `Cmd` instead of `Ctrl`. | Shortcut | Action | -|----------|--------| -| `Ctrl+N` | New workspace | -| `Ctrl+,` | Settings | -| `Ctrl+K` | Quick switch workspace | -| `Ctrl+Shift+M` | Toggle memory browser | -| `Enter` | Send message | -| `Shift+Enter` | Newline in message | -| `/` | Start slash command | +|---|---| +| `Ctrl+K` | Search everything | +| `Ctrl+Shift+P` | Switch persona | +| `Ctrl+Tab` | Switch workspace | +| `Ctrl+Shift+N` | Open the chat of the active workspace | +| `Ctrl+/` or `?` | Show all shortcuts | +| `Enter` / `Shift+Enter` | Send / new line | +| `Ctrl+Shift+W` | Show or hide the desktop app window (desktop app only) | + +## Next steps + +- [Workspaces](workspaces.md): workspaces, sessions, personas and folders on disk +- [Skills, plugins and the marketplace](capabilities.md) +- [Connectors and channels](connectors.md) +- [Troubleshooting](troubleshooting.md) +- [Where your data lives](../install.md#where-your-data-lives) diff --git a/docs/guides/troubleshooting.md b/docs/guides/troubleshooting.md index 51a21fb..2255eb3 100644 --- a/docs/guides/troubleshooting.md +++ b/docs/guides/troubleshooting.md @@ -1,207 +1,141 @@ # Troubleshooting -Common issues, error codes, and solutions. +Common problems and what to do about them. Paths use `` for the data directory: +`~/.waggle` by default (`%USERPROFILE%\.waggle` on Windows), or `WAGGLE_DATA_DIR`. -## API Key Issues +## Installing -### "No API key configured" +### Windows or macOS refuses to open the installer or the app -The agent cannot run without an LLM provider key. +The first release is not signed. See [install.md](../install.md): on Windows choose **More +info** → **Run anyway**; on macOS use **Open Anyway** in System Settings → Privacy & Security, or +`xattr -dr com.apple.quarantine /Applications/Waggle.app`. -**Fix:** Go to Settings > Models and add your Anthropic key (starts with `sk-ant-`). Or edit `~/.waggle/config.json` directly: +### The AppImage does not start on Linux -```json -{ - "defaultModel": "claude-sonnet-4-6", - "providers": { - "anthropic": { - "apiKey": "sk-ant-your-key-here", - "models": ["claude-sonnet-4-6"] - } - } -} -``` +Install FUSE 2 (`libfuse2` or `libfuse2t64` on Ubuntu, `fuse-libs` on Fedora), or use the `.deb`. -### "API key is too short" / "must start with sk-ant-" +### The server installer stops with an error -The key format validation failed. +| Message | What to do | +|---|---| +| "This build has no download location yet. Pass --from …" | The default download location is not set yet. Pass the archive: `--from ` | +| "There is no Waggle server build for …" | Your system is not supported. Builds exist for Windows x64, macOS (Apple silicon and Intel) and Linux x64 | +| "The archive checksum does not match" | The download is damaged or not the published file. Download it again | +| "No checksum found at ….sha256" | The `.sha256` file is missing next to the archive URL. Pass `--sha256 ` if you have the checksum from another source | +| "This archive is for …; this system needs …" | You downloaded the archive for another system | +| "Windows cannot update the program it is running" | Stop the server, then run `npx @waggle-ai/waggle update` (or `uninstall`) | +| "Waggle needs Node.js 20 or newer to install" | Update Node.js for the `npx` call; Waggle itself runs on its own Node.js | -**Fix:** Anthropic keys must start with `sk-ant-` and be at least 20 characters. OpenAI keys start with `sk-`. Copy the full key from your provider dashboard. +Run `waggle-server doctor` to check an existing installation. -### Key works in API but not in Waggle +## Starting -The built-in Anthropic proxy translates OpenAI-format requests to Anthropic format. If you are using a non-standard provider, you may need LiteLLM as a proxy: +### The desktop app stays on "Starting Waggle…" + +Close the app and look at the server log: `/logs/service.log` (the previous run is in +`service.log.1`). It usually names the problem, such as a data directory that is not writable. + +On Windows, Waggle needs the WebView2 runtime, which ships with current Windows 10 and 11. If the +window stays white, install WebView2 from Microsoft. + +### Port 3333 is in use + +The desktop app picks another free port by itself. For the server installer: ```bash -docker compose up litellm +waggle-server --port 3334 ``` -Then set your LiteLLM URL in Settings > Advanced. +When running from source: `WAGGLE_PORT=3334 npm run dev:server`. -## Server Issues +To see what uses the port: `lsof -i :3333` on macOS and Linux, `netstat -ano | findstr 3333` on +Windows. -### Server won't start +### The page is blank or assets return 404 (running from source) -**Check port availability:** -```bash -# Is port 3333 already in use? -lsof -i :3333 # macOS/Linux -netstat -ano | findstr 3333 # Windows -``` +The sidecar registers the files in `dist/` when it starts. After `npm run build`, restart it. +Use the UI the sidecar serves at http://localhost:3333; the Vite dev server (`npm run dev:web`) +cannot sign in to the sidecar yet. -**Fix:** Kill the existing process or change the port: -```bash -PORT=3334 npx tsx src/local/start.ts -``` +## Models -### "SQLITE_CANTOPEN" error +### "No AI model is ready" -The `.mind` database file cannot be opened. +Chat needs at least one working model. Add a provider key in **Settings** → **Models** (API key +tab), or install the private runtime and a local model there (Local model tab). -**Fix:** -- Check that `~/.waggle/` exists and is writable -- Check that no other process has the `.mind` file locked -- If the file is corrupted, rename it (data will be lost) and let Waggle create a fresh one +### A provider key is rejected -### Server crashes with "out of memory" +Copy the whole key again from the provider's console. Waggle checks the key's format and, when +the provider is reachable, tests it. A key that worked before can stop working when the provider +account runs out of credit or the key is revoked. -Large workspace minds or many concurrent operations can exhaust memory. +### Local model downloads are slow or large -**Fix:** -- Restart the server -- If recurring, check your mind file size: `ls -la ~/.waggle/default.mind` -- Consider running memory consolidation: `POST /api/cron/{consolidation-id}/trigger` +The managed Ollama runtime and the models are large downloads (the Linux runtime alone is about +1.4 GB). They are stored in `/runtimes/ollama` and `/models/ollama`. Keep the window +open until the download is verified. -## Memory Issues +## Memory -### Memory not saving +### Waggle does not remember something -Memories are saved automatically during conversations when the agent determines something is important. +- Say it explicitly: "Remember that …". +- Check the scope. Workspace memories are only recalled in their workspace; personal memories + everywhere. `/search-all ` searches all workspaces. +- Open **Memory** and search for it. If it is there but marked temporary, it is not recalled. -**Check:** -1. Verify the workspace mind path exists: `~/.waggle/workspaces/{id}/workspace.mind` -2. Check that the disk has free space -3. Try saving explicitly: "Remember that [important fact]" +### "Learn facts about me from chats" is not available -### Search returns no results +Background fact extraction needs a model. Set one up in **Settings** → **Models**. -FTS5 search requires at least 3 characters. +### "This memory store was built with embedder …" -**Fix:** -- Use longer queries (3+ characters) -- Check you are searching the right scope (personal, workspace, or all) -- Verify memories exist: open the Memory tab in the right panel +The memory store was created with another embedding model, whose vectors cannot be compared with +the current one. Switch back to that model, or quit Waggle and move `/memory/` aside to +start a new, empty store. -### Memory from another workspace showing up +### Memory needs a network connection the first time -This is expected behavior. The agent searches both personal and workspace minds. Personal mind memories are shared across all workspaces. +The embedding model used by memory is downloaded on first use into `/models/`. After that, +memory works offline. -**Fix:** If a memory should be workspace-specific, it should have been saved to the workspace mind (this happens automatically for project-specific context). +## Connectors and channels -## Connector Issues +See [connectors.md](connectors.md#troubleshooting). -### "Vault not available" +## Your data -The encrypted vault database failed to initialize. +### Where is my data? -**Fix:** -- Check that `~/.waggle/` is writable -- If `~/.waggle/vault.db` exists but is corrupted, rename it and restart -- The vault auto-migrates from plain config.json on first access +In ``; [install.md](../install.md#where-your-data-lives) lists what each file holds. Data +leaves your computer only for the services you set up (model providers, connectors, channels, +marketplace sources) and, if you allowed analytics, one setup event without your chats, files or +memories. -### Connector shows "disconnected" after restart +### Move Waggle to another computer -Credentials persist in the vault across restarts. If a connector shows disconnected: +Quit Waggle and copy the whole data directory. On the new computer, install Waggle and put the +directory in the same place (or point `WAGGLE_DATA_DIR` at it). The Vault key (`.vault-key`) +must travel with `vault.json`, or the stored secrets cannot be read. -1. Check the health endpoint: `GET /api/connectors/{id}/health` -2. If the token expired, re-connect with fresh credentials -3. If the service is unreachable, check your network +### Start over -### "Connector not found" (404) +Quit Waggle and delete the data directory, or use **Settings** → **General** → **Erase all my +data**, which wipes it the next time Waggle starts. Data you already sent to cloud model providers +is not erased by this; ask those providers. -Connector IDs are lowercase. Use `GET /api/connectors` to see all registered IDs. +## Questions -### Approval gate stuck +**Which models work?** Models from Anthropic, OpenAI, Google, DeepSeek, xAI, Mistral, Alibaba, +MiniMax, Zhipu, Moonshot, Perplexity and OpenRouter with your own key, and local models through +Ollama. -If an approval card appears but you cannot click Approve/Deny: +**What does it cost?** Waggle is free. You pay your model provider for what you use. **Usage & +Cost** shows the estimated cost of your model calls, and Settings → Models can set a daily +budget. -**Fix:** -1. Check `GET /api/approval/pending` to see pending approvals -2. Approve via API: `POST /api/approval/{requestId}` with `{"approved": true}` -3. If the request expired, send a new message to the agent - -## Desktop App Issues - -### White screen after launch - -The Tauri WebView2 runtime may not be installed (Windows). - -**Fix:** Download and install the [WebView2 Runtime](https://developer.microsoft.com/en-us/microsoft-edge/webview2/). - -### Sidecar not starting - -The Node.js sidecar needs Node.js 20+ in the system PATH. - -**Fix:** -- Install Node.js 20+ from https://nodejs.org -- Restart the app after installation -- Check the Tauri console for error messages - -### Notifications not appearing (Windows) - -**Fix:** -- Check Windows notification settings for Waggle -- Ensure "Focus Assist" is not blocking notifications -- The app must be running (tray icon visible) - -## Team Mode Issues - -### Cannot connect to team server - -**Check:** -- The server URL is correct (include `https://`) -- The auth token is valid -- The team server is reachable: `curl https://your-server/health` - -### "Team server connection timed out" - -**Fix:** The team server health check has a 5-second timeout. Check that: -- The server is running -- There are no firewall rules blocking the connection -- DNS resolves correctly - -### Tasks not syncing - -Task data is stored per-workspace in JSONL files. In team mode, tasks are stored on the team server. - -**Fix:** Check team connection status: `GET /api/team/status` - -## Common Error Codes - -| Code | Meaning | Fix | -|------|---------|-----| -| 400 | Bad request -- missing or invalid parameters | Check the API reference for required fields | -| 404 | Resource not found | Verify the ID/name exists | -| 409 | Conflict -- resource already exists | The skill/plugin is already installed | -| 413 | File too large | Files must be under 10 MB | -| 503 | Service unavailable | The required service (vault, marketplace, plugin runtime) is not ready | - -## FAQ - -**Q: Where is my data stored?** -A: All data is in `~/.waggle/` on your machine. Nothing is sent to cloud servers unless you explicitly connect a team server or external service. - -**Q: Can I move my data to another machine?** -A: Yes. Copy the entire `~/.waggle/` directory. The `.mind` files are portable SQLite databases. - -**Q: How do I reset everything?** -A: Delete `~/.waggle/` and restart. Waggle creates fresh defaults on startup. - -**Q: Which LLM models work?** -A: Waggle works best with Claude (Anthropic). It also supports OpenAI, Google Gemini, and any model available through LiteLLM. The built-in proxy handles Anthropic natively. - -**Q: How much does it cost to run?** -A: Waggle itself is free. You pay only for LLM API usage. The Cockpit status bar shows estimated cost per session. A typical conversation costs $0.05-0.50 depending on length and model. - -**Q: Can I use it offline?** -A: Waggle requires an LLM API connection for agent responses. Memory, workspace management, and the UI work offline. +**Does it work offline?** With a local model, yes, once the runtime, the model and the embedding +model are downloaded. With a provider key, chat needs a connection to that provider; your +workspaces, files and memory stay available offline. diff --git a/docs/guides/workspaces.md b/docs/guides/workspaces.md index 47cc88a..6675757 100644 --- a/docs/guides/workspaces.md +++ b/docs/guides/workspaces.md @@ -1,149 +1,74 @@ # Workspaces -Workspaces are the core organizational unit in Waggle. Each workspace is a dedicated brain for a project, topic, or area of work. It has its own memory, sessions, files, tasks, and optionally a linked directory on disk. +A workspace holds one project or area of work: its chats, its files, its memory and the persona +the agent uses there. Your personal memory is shared by all workspaces; everything else stays in +its workspace. -## Creating a Workspace +## Creating a workspace -Press **Ctrl+N** or click **New Workspace** in the sidebar. +Create a workspace from **Home** or the workspace list. You choose: -| Field | Required | Description | -|-------|----------|-------------| -| Name | Yes | Display name (e.g., "Q1 Planning", "Product Research") | -| Group | Yes | Category for sidebar grouping (e.g., "Work", "Personal") | -| Icon | No | Emoji or character for visual identification | -| Model | No | Override the default LLM model for this workspace | -| Persona | No | Assign a specialized agent persona | -| Directory | No | Link a folder on disk for file-aware operations | +| Field | Description | +|---|---| +| Name | "What project or area is this for?", for example "Acme Client" or "Q3 Launch" | +| Template | Optional. A starting setup with a matching persona, such as Research Hub or Engineering | +| Group | A category for grouping workspaces in the list, such as Work or Personal | +| Storage type | **Virtual**: Waggle keeps the workspace's files in its data directory. **Local**: the workspace uses a folder on your disk | +| Agent | Optional. The persona to use in this workspace | -When you create your first workspace, Waggle auto-installs the starter skills (draft-memo, research-synthesis, extract-actions, etc.) if they are not already present. +The first setup creates your first workspace from the template you pick. -## Workspace Types +## Storage: virtual or a folder on disk -### Personal Workspaces +- **Virtual** workspaces keep their files in `/workspaces//files`. +- **Local** workspaces point at a folder you choose. The agent's file tools (read, write, edit, + search) work in that folder, and its git tools work on the repository there. Writes go + through the approval rules in Settings → Permissions. Shell commands run with your user's + rights and are not sandboxed; see the [threat model](../../THREAT_MODEL.md). -Created locally, stored on your machine. Memory lives in `~/.waggle/workspaces/{id}/workspace.mind`. Only you can access them. +## Chats (sessions) -### Team Workspaces +Each conversation in a workspace is a session. Sessions are stored as JSON Lines files in +`/workspaces//sessions/`. -Linked to a team server. Created from the Team panel after connecting to a team server. Team workspaces support: - -- Shared memory that syncs between team members -- Task boards visible to the whole team -- Real-time presence (who is online, who is typing) -- Admin-governed capability policies - -A workspace's team status is indicated by a team badge in the sidebar. - -## Workspace Home Screen - -When you open a workspace, you see the Home screen with: - -1. **Summary** -- a narrative description of what this workspace is about, how many memories and sessions it has, and when it was last active. -2. **Recent Decisions** -- key decisions extracted from your conversation history. -3. **Recent Threads** -- your last 5 sessions, clickable to resume. -4. **Suggested Prompts** -- contextual suggestions based on workspace state: - - New workspace: "Tell me about this project", "What can you do?" - - Returning workspace: "Catch me up", "Continue: [last thread]", "What should I do next?" -5. **Progress Items** -- tasks, completions, and blockers extracted from sessions. -6. **Stats** -- memory count, session count, file count. - -The Home screen is never a blank chat. It always gives you a reason to engage. - -## Sessions - -Each conversation thread is a **session**. Sessions are stored as `.jsonl` files in `~/.waggle/workspaces/{id}/sessions/`. - -### Creating Sessions - -- Click the **+** button in the sidebar to start a new session -- Or just start typing -- a session is created automatically - -### Session Metadata - -Each session has: -- **Title** -- derived from the first user message, or set manually -- **Summary** -- auto-generated after 4+ messages (heuristic, no LLM needed) -- **Outcome** -- what changed, open items, next step (extracted at session end) - -### Renaming Sessions - -Click the session title in the sidebar and type a new name, or use the `PATCH /api/sessions/:id` endpoint. - -### Searching Across Sessions - -Use the search bar in the sidebar to find messages across all sessions in the current workspace. Matches show snippets with context. - -### Exporting Sessions - -Right-click a session and choose **Export as Markdown** to get a clean document with timestamps and formatted messages. - -## Switching Workspaces - -- Click any workspace in the left sidebar -- Press **Ctrl+K** for the quick-switch dialog -- Workspaces are grouped by their category (Work, Personal, etc.) - -Each workspace in the sidebar shows a hue-colored dot for visual differentiation. +- Start a new chat from the workspace, or just type: a session is created on the first message. +- A session is titled from its first message; you can rename it. +- Search finds messages across the sessions of the current workspace. +- A session can be exported as Markdown. ## Personas -Personas are specialized agent configurations. Assigning a persona to a workspace tunes the agent's behavior without changing the core capabilities. +A persona sets how the agent works in a workspace: its instructions, the tools it prefers, the +commands it suggests and, for some, a default workflow. Waggle ships with 22 personas: General +Purpose, Researcher, Writer, Analyst, Coder, Planner, Verifier, Coordinator, Project Manager, +Senior PM, Executive Assistant, Sales Rep, Marketer, Customer Support, Operations Manager, Data +Engineer, HR Manager, Recruiter, Legal Counsel, Business Finance, Strategy Consultant and +Creative Director. -| Persona | Focus | Default Workflow | -|---------|-------|-----------------| -| Researcher | Deep investigation, multi-source synthesis | research-team | -| Writer | Document drafting, editing, tone adaptation | -- | -| Coder | Software development, debugging, code review | -- | -| Analyst | Data analysis, decision matrices, pattern recognition | -- | -| Project Manager | Task tracking, status reports, planning | plan-execute | -| Executive Assistant | Email drafting, meeting prep, correspondence | -- | -| Sales Rep | Lead research, outreach, pipeline management | research-team | -| Marketer | Content creation, campaign planning, SEO | -- | +Switch the persona with `Ctrl+Shift+P` (`Cmd+Shift+P` on macOS). Some personas are read-only: +they cannot use tools that write. You can also create your own agents in **Agents** (at the +Everything interface level). -### Changing Persona +## Workspace memory -Open workspace settings (gear icon on the workspace home) and select a different persona. The change takes effect on the next message. +Each workspace has its own memory, separate from your personal memory: -### Persona Tool Presets +- **Workspace memory** holds project context: decisions, findings, notes. +- **Personal memory** holds what is true across workspaces: your preferences, your role, + recurring facts. -Each persona has a curated tool set. The Coder persona enables git tools; the Researcher enables web search; the Executive Assistant enables document generation. All tools remain available -- the persona just adjusts defaults and suggestions. +In a workspace the agent reads both. It does not read other workspaces' memory unless you ask +it to search all workspaces (`/search-all `). See [memory.md](../memory.md). -## Directory Association +Open **Memory** to browse, search, correct, confirm or forget memories. -Linking a workspace to a directory on disk enables: +## Switching workspaces -- `read_file`, `write_file`, `edit_file` operate relative to the linked directory -- `search_files` and `search_content` scan the directory tree -- `git_status`, `git_diff`, `git_log`, `git_commit` work on the repo in that directory -- File listings appear in the workspace context +- Pick a workspace from **Home** or the workspace list. +- `Ctrl+Tab` opens the workspace switcher; `Ctrl+K` searches everything, workspaces included. -To link a directory, set it during workspace creation or update it in workspace settings. +## Deleting a workspace -## Workspace Memory - -Each workspace has its own `.mind` database separate from your personal mind. - -- **Workspace mind** stores project-specific context: decisions, research, architecture choices -- **Personal mind** stores cross-workspace knowledge: your preferences, name, recurring facts - -When the agent searches memory, it queries both minds and merges results, prioritizing workspace-specific memories when relevant. - -### Memory Browser - -Click the **Memory** tab in the right panel to browse all frames in the current workspace. You can: - -- Search by keyword (FTS5 full-text search) -- Filter by importance level -- View the knowledge graph of entities and relationships -- Delete outdated or incorrect frames - -## Deleting a Workspace - -Right-click a workspace in the sidebar and choose **Delete**. This removes: - -- The workspace entry from the workspace list -- Session files -- The workspace `.mind` file -- Task and file registry data - -Personal mind memories are not affected. +Deleting a workspace removes its memory, its sessions and its virtual files +(`/workspaces//`). A folder on disk linked as Local storage is not deleted. Personal +memory is not affected. diff --git a/docs/install.md b/docs/install.md new file mode 100644 index 0000000..9bd97f8 --- /dev/null +++ b/docs/install.md @@ -0,0 +1,279 @@ +# Installing Waggle + +There are two ways to install Waggle. Both run the same server and the same web UI, and both +keep your data in the same place. + +| | Desktop app | Server installer | +|---|---|---| +| What you get | A Waggle window, started from the Start menu, Applications or your launcher | A `waggle-server` command; you use Waggle in your browser | +| Needs | Nothing else | Node.js 20 or newer for the first `npx` call only | +| Formats | Windows `.exe` and `.msi`, macOS `.dmg`, Linux `.deb` and `.AppImage` | One archive per system, downloaded by the installer | + +Supported systems in this release: Windows x64, macOS on Apple silicon (arm64) and Intel +(x64), and Linux x64 with glibc (for example Debian, Ubuntu, Fedora). Other systems, such as +Windows on ARM or Linux on ARM, have no build. + +Neither way needs Python, Docker or a compiler. Each build carries its own Node.js runtime +(22.23.2) and the native modules that match it. + +> **Unsigned first release.** The installers of the first community release are not signed. +> Windows and macOS warn about them. The sections below show how to get past the warning. +> Download Waggle only from the project's release page. + +## Desktop app + +### Windows + +1. Download `Waggle__x64-setup.exe`. A `.msi` package is also available; use the + `.exe` unless you deploy with MSI tools. The `.exe` installs for the current user and does + not need administrator rights. +2. If the browser warns that the file is not commonly downloaded, choose to keep it (in Edge: + **...** → **Keep**). +3. Run the installer. SmartScreen shows "Windows protected your PC". Choose **More info**, then + **Run anyway**. +4. Start Waggle from the Start menu. + +Waggle uses the WebView2 runtime, which ships with current Windows 10 and 11. + +### macOS + +1. Download the `.dmg` for your Mac: `Waggle__aarch64.dmg` for Apple silicon, + `Waggle__x64.dmg` for Intel. (Apple menu → **About This Mac** shows the chip.) +2. Open the `.dmg` and drag **Waggle** to **Applications**. +3. Open Waggle. Because the app is not signed, macOS blocks the first launch: + - macOS 15 and later: after the warning, open **System Settings** → **Privacy & Security**, + scroll to the message about Waggle and choose **Open Anyway**. + - Earlier versions: Control-click (or right-click) Waggle in Applications, choose **Open**, + then **Open** again in the dialog. + - If macOS says the app "is damaged" or still refuses, remove the download quarantine flag + in Terminal and open it again: + + ```bash + xattr -dr com.apple.quarantine /Applications/Waggle.app + ``` + +You only need to do this once. + +### Linux + +Debian, Ubuntu and other distributions with apt: + +```bash +sudo apt install ./Waggle__amd64.deb +``` + +apt installs the libraries the app needs (WebKitGTK and GTK). Start Waggle from your +application launcher. + +AppImage, for other distributions: + +```bash +chmod +x Waggle__amd64.AppImage +./Waggle__amd64.AppImage +``` + +An AppImage needs FUSE 2. If it fails with a FUSE error, install it (`libfuse2` on Ubuntu 22.04, +`libfuse2t64` on Ubuntu 24.04 and later, `fuse-libs` on Fedora). + +Linux packages are not signed. If the release page lists SHA-256 checksums, compare them before +you install: + +```bash +sha256sum Waggle__amd64.deb +``` + +### What the desktop app does on start + +The window shows "Starting Waggle…" while the app starts its own server in the background. The +server listens on `127.0.0.1` port 3333, or on another free port if 3333 is taken. Its log is in +`~/.waggle/logs/service.log` (on Windows `%USERPROFILE%\.waggle\logs\service.log`). + +### Update the desktop app + +The desktop app does not update itself in this release. Quit Waggle, then install the new +version over the old one: + +- Windows: run the new installer. +- macOS: drag the new Waggle to Applications and replace the old one. +- Linux: `sudo apt install ./Waggle__amd64.deb`, or replace the AppImage file. + +Your data is not touched. + +### Uninstall the desktop app + +- Windows: **Settings** → **Apps** → **Installed apps** (**Apps & features** on Windows 10) → + **Waggle** → **Uninstall**. The uninstaller always keeps your data, even if you tick its + "Delete app data" box. +- macOS: quit Waggle and move **Waggle** from Applications to the Trash. +- Linux: remove the package (`sudo apt remove waggle`; `dpkg -l | grep -i waggle` shows its + name), or delete the AppImage file. + +Your data stays in the data directory; see [Where your data lives](#where-your-data-lives). + +## Server installer + +The server installer puts the Waggle server on your computer and starts it. You then use Waggle +in your browser at http://localhost:3333. + +> **Temporary names.** The package name `@waggle-ai/waggle` and the command `waggle-server` are +> working names until the product name is decided. The installer also has no default download +> location yet, because the location of the public repository is not decided. Until then, pass +> the archive with `--from`, as shown below. + +### Install + +You need Node.js 20 or newer for this one command. After that, Waggle runs on its own Node.js +runtime and does not use yours. + +```bash +npx @waggle-ai/waggle install --from +``` + +The archive for your system is named `waggle-server--.tar.gz`, where +`` is `windows-x64`, `macos-arm64`, `macos-x64` or `linux-x64`. A `.sha256` file is +published next to it. + +The installer: + +1. checks the archive against its SHA-256 (for a URL it reads `.sha256`; for a local file it + reads `.sha256` if one is next to it). A mismatch stops the install. Pass + `--sha256 ` to give the checksum yourself; +2. unpacks it into a program directory; +3. writes the `waggle-server` command. + +| System | Program directory | `waggle-server` command | +|---|---|---| +| Windows | `%LOCALAPPDATA%\Waggle\server` | `%LOCALAPPDATA%\Waggle\server\bin\waggle-server.cmd` | +| macOS | `~/Library/Application Support/Waggle/server` | `~/.local/bin/waggle-server` | +| Linux | `~/.local/share/waggle/server` (or `$XDG_DATA_HOME/waggle/server`) | `~/.local/bin/waggle-server` | + +The installer never edits your `PATH`. If the command's directory is not on your `PATH`, it +prints the full path to run and the directory to add. + +Once a download location exists, `npx @waggle-ai/waggle install` without `--from` downloads +the archive for your system. `WAGGLE_RELEASE_BASE_URL` points it at a mirror. + +### Run + +```bash +waggle-server +``` + +This is the same as `waggle-server run`. It starts the server, waits until it answers and opens +http://localhost:3333 in your browser. The server runs in that terminal; press Ctrl+C to stop +it. It listens on this computer only (`127.0.0.1`). + +| Option | Meaning | +|---|---| +| `--port `, `-p` | Use another port (default 3333) | +| `--no-open` | Do not open the browser | +| `--data-dir ` | Keep data in another directory (default `$WAGGLE_DATA_DIR` or `~/.waggle`) | +| `--dir ` | Use a program directory other than the default | + +If Waggle already runs on the port, `waggle-server` only opens the browser. + +### Check the installation + +```bash +waggle-server doctor +``` + +`doctor` checks that this system has a build, the installed files, the bundled Node.js runtime, +that SQLite loads, that the data directory is writable, the port, and whether +`waggle-server` is on your `PATH`. It exits with an error code if a check fails. + +### Update + +`update` installs the version of the installer you run it with: + +```bash +npx @waggle-ai/waggle@latest update # once a download location exists +waggle-server update --from # a specific archive +``` + +Your data is not touched. On Windows a running program cannot be replaced: stop the server +first, and run `update` through `npx`, not through the installed `waggle-server`. + +### Uninstall + +```bash +waggle-server uninstall +``` + +This removes the program directory and the `waggle-server` command. Your data stays in the data +directory. On Windows, stop the server and run `npx @waggle-ai/waggle uninstall`. + +All commands also take `--help` and `--version`. + +## First launch + +Waggle opens a short setup: + +1. **Welcome** and **Who are you**: your name, role and the kind of work you do, kept on this + computer in your profile. +2. **Model**: choose how Waggle thinks (see below). You can skip this step, but chat stays + unavailable until a model is set up. +3. **Pick a starting point**: a template for your first workspace (for example Research Hub, + Sales Pipeline or Engineering), or **Blank Workspace**. +4. **What's first?**: give the agent its first task, or pick a suggestion. + +### Choose a model + +- **API key.** Paste a key from a provider you already use: Anthropic, OpenAI, Google, + DeepSeek, xAI, Mistral, Alibaba (Qwen), MiniMax, Zhipu (GLM), Moonshot (Kimi), Perplexity or + OpenRouter. Waggle checks the key with the provider and stores it encrypted on this computer. + Requests go from your computer straight to that provider. +- **Local model.** Choose **Install private runtime**. Waggle downloads the official Ollama + runtime into its data directory, checks the download's checksum and runs it for you. Then + download a model (Waggle suggests one that fits your hardware). Nothing else needs to be + installed, and chats stay on your computer. The downloads are large: the Linux runtime alone + is about 1.4 GB, and a model adds several GB. If you already run Ollama yourself, Waggle uses + it. + +You can change both later in **Settings** → **Models**. Keys are also listed in the **Vault**. + +### Analytics + +Release builds may carry a key for product analytics. If yours does, Waggle asks once whether it +may send usage analytics. Nothing is sent until you choose **Allow**. If you allow it, Waggle +sends one event when you finish the setup, with the template, persona and model you chose. Your +chats, files and memories are never sent. You can turn analytics off again in **Settings** → +**General** → **Privacy & Telemetry**. Builds from source have no key: they never ask and never +send. + +## Where your data lives + +Both ways of installing use one data directory: `~/.waggle` (on Windows +`%USERPROFILE%\.waggle`). Set `WAGGLE_DATA_DIR` to use another directory; the server installer +also takes `--data-dir`. + +| Path in the data directory | What it holds | +|---|---| +| `config.json` | Settings | +| `vault.json`, `.vault-key` | Provider keys and connector credentials, encrypted with AES-256-GCM | +| `waggle.db` | Automations, notifications, cost tracking, your profile card and task list | +| `memory/` | Your memories (see [memory.md](memory.md)) | +| `workspaces//` | Each workspace: settings, chat sessions (`sessions/*.jsonl`) and files | +| `skills/`, `plugins/` | Installed skills and plugins | +| `marketplace.db` | The marketplace catalog | +| `models/` | The embedding model used by memory, downloaded on first use; local model weights in `models/ollama/` | +| `runtimes/ollama/` | The Waggle-managed Ollama runtime | +| `channels/`, `browsers/` | Messaging channel sessions, and a browser downloaded for WhatsApp if none is installed | +| `logs/` | The desktop app's server log | + +Uninstalling Waggle, either way, keeps this directory. To remove your data, quit Waggle and +delete the directory, or use **Settings** → **General** → **Erase all my data**, which wipes it +the next time Waggle starts. + +To move Waggle to another computer, quit it and copy the whole directory. + +The commercial edition of Waggle uses the same default directory. If you use both on one +computer, give the community edition its own directory with `WAGGLE_DATA_DIR`. + +## Run from source + +See the [README](../README.md#run-from-source) and [CONTRIBUTING.md](../CONTRIBUTING.md). + +## Problems + +See [troubleshooting](guides/troubleshooting.md). diff --git a/docs/memory.md b/docs/memory.md new file mode 100644 index 0000000..f48d141 --- /dev/null +++ b/docs/memory.md @@ -0,0 +1,176 @@ +# Memory + +Waggle remembers facts, decisions and preferences across chats and keeps them on your computer. +This page describes how memory works in the community edition, for users who want to know what +is kept and for contributors who change memory code. + +The decisions behind it are in [ADR-0002](oss/adr/0002-memory-contract-local-mem0.md) and +[ADR-0007](oss/adr/0007-memory-contract.md). + +## The memory contract + +Waggle reaches memory only through one interface, `MemoryStore`, in the package +`@waggle/memory-contract` (`packages/memory-contract/src/types.ts`). No other code queries a +memory database directly. The package also holds an in-memory store for development and tests, +and a compatibility suite that every implementation must pass. + +The implementation that ships is `@waggle/memory-mem0` (`packages/memory-mem0/src/index.ts`). +It runs the open-source [mem0](https://github.com/mem0ai/mem0) library (`mem0ai`, Apache-2.0, +pinned to 3.3.1) inside the Waggle server. The hosted Mem0 Platform is not used, and nothing is +sent to a cloud memory service. + +### Records + +A memory record has: + +| Field | Values | +|---|---| +| `text` | The memory itself | +| `scope` | `personal`, or one workspace | +| `kind` | `fact`, `event`, `preference`, `profile`, `decision`, `summary`, `note` | +| `importance` | `critical`, `important`, `normal`, `temporary` | +| `source` | `user_stated`, `tool_verified`, `agent_inferred`, `import`, `system` | +| `origin` | Which part of Waggle wrote it, for example `chat`, `agent-run`, `loop` or `extraction` | +| `sessionId` | The chat it came from, if any | +| `createdAt`, `updatedAt` | ISO 8601 timestamps | +| `metadata` | Free fields set by the caller (title, tags and so on) | + +Ids are unique across all scopes. + +### Scopes + +- **Personal** memory follows you across all workspaces: who you are, how you like to work, + facts that are not tied to one project. +- **Workspace** memory belongs to one workspace: its decisions, findings and files. + +In a workspace, the agent reads both its personal memory and that workspace's memory. It never +reads another workspace's memory unless you ask it to search all workspaces (for example with +`/search-all`). + +### Operations + +| Operation | What it does | +|---|---| +| `remember` | Stores a text as one memory, or extracts facts from a conversation with a model. Identical text in the same scope is not stored twice. | +| `search` | Returns the memories most relevant to a query, best first (default 10) | +| `list` | Returns memories newest first (default 50) | +| `count` | Counts the memories that match a filter | +| `get` | Reads one memory by id | +| `update` | Changes the text, kind, importance or metadata of a memory | +| `forget` | Deletes one memory | +| `forgetMatching` | Deletes every memory that matches a filter | +| `history` | Lists the changes to one memory, oldest first | + +Filters select by scopes, kinds, minimum importance, origin, chat session and a creation-time +window. + +### Capabilities + +A store reports what it can do, and the web UI hides what it cannot: + +| Capability | mem0 store | Meaning | +|---|---|---| +| `extraction` | Yes, when a model is set up | `remember` can extract facts from a conversation | +| `semanticSearch` | Yes | Search ranks by meaning, not only by shared words | +| `history` | Yes | The change log of each memory is kept | +| `knowledgeGraph` | No | Entities and relations between memories. Not part of this release. | + +## Where mem0 keeps its data + +Everything is in `memory/` inside the data directory (`~/.waggle/memory/` by default): + +| File | Content | +|---|---| +| `vectors.db` | Memories and their embeddings (SQLite) | +| `history.db` | The change log, and the last messages used for fact extraction (SQLite) | +| `vectors_entities.db` | A second store mem0 creates next to `vectors.db` (SQLite) | +| `waggle-memory.json` | Which embedding model built the store | +| `config.json` | mem0's own settings | + +All memories belong to one local user. The scope is stored as mem0's agent id (`personal` or +`workspace:`), so mem0 never compares facts across scopes. + +mem0's own telemetry is turned off before the library loads, so it makes no network requests +of its own. A test checks this. + +When you forget a memory, Waggle also deletes its old text from `history.db`, so the text does +not survive in the change log. + +### Embeddings + +Search compares embeddings: number vectors that capture the meaning of a text. Waggle computes +them on your computer with a small model (`all-MiniLM-L6-v2`), downloaded on first use into +`models/` in the data directory. After that, memory works without a network connection. + +The store records which embedding model built it and refuses to open with a different one, +because their vectors cannot be compared. To start over with another model, move the +`memory/` directory aside. + +## How the agent uses memory + +### Recall + +Before it answers, the agent recalls what is relevant to your message: + +- it searches personal memory and the current workspace's memory (mem0 combines vector and + keyword search); +- it adds the most important memories of the workspace, even if they do not match the words; +- it adds facts about you (kind `profile`) and recent events, each in its own section; +- a question about a period ("what did we do last week") limits the search to that period; +- a catch-up question ("where were we?", "what did we decide?") recalls the workspace's + important memories and decisions instead of searching for the words; +- `temporary` memories are left out. + +The recalled text passes the prompt-injection scanner first. If it looks like an attempt to +give the agent instructions, the whole recall is dropped for that turn. + +An optional reranker can reorder search results; it is off by default +(`WAGGLE_RERANKER=1` turns it on). + +### Writing + +Memories are written: + +- when you ask ("Remember that we chose PostgreSQL") or the agent decides something is worth + keeping, through its `save_memory` tool; +- from finished agent runs, scheduled loops, connector fetches and indexed workspace files; +- from summaries when a long chat is compacted; +- by background fact extraction (next section). + +### Background fact extraction + +While you chat, Waggle can pick out facts about you, such as your role or how you like answers, +and save them to personal memory as kind `profile`. + +- Finished chat turns are collected per chat. After six messages, or after a two-minute pause, + they are handed to mem0, which asks a model to extract facts. +- The model is the one you chat with. mem0 calls it through the Waggle server's built-in + provider proxy, so a local model keeps this on your computer, and a provider key sends these + messages to that provider, as chat does. +- Long messages are cut to their first 4,000 characters. +- A failed extraction is logged and never blocks a chat. +- Turn it off in **Settings** → **General** → **Memory** ("Learn facts about me from chats"). + It is on by default, and unavailable when no model is set up. + +## Managing memory + +Open **Memory** in Waggle to browse, search, correct, confirm or forget what is kept. +`/memory ` searches from a chat. + +Not in this release: importing chat history from other AI tools, the knowledge graph, the wiki, +and the memory server for external tools (MCP). + +## For contributors + +- Reach memory only through `MemoryStore`. Do not query a memory database with SQL and do not + import mem0 outside `@waggle/memory-mem0`. +- Waggle's own data (automations, cost tracking, notifications, your profile card, the task + list) is not memory. It lives in `waggle.db`, opened by `WaggleDB` in `@waggle/core`. +- A new implementation of the contract must pass the compatibility suite in + `@waggle/memory-contract`. +- `list`, `count` and `forgetMatching` read every memory of the scopes involved. That is fine at + personal scale. +- Upgrading `mem0ai` means re-running the compatibility suite and re-checking that mem0 sends + no telemetry (ADR-0007). +- Tests use a word-based embedder that needs no model download: + `WAGGLE_MEMORY_EMBEDDER=words`. diff --git a/docs/reference/api.md b/docs/reference/api.md index b929cd1..238b17d 100644 --- a/docs/reference/api.md +++ b/docs/reference/api.md @@ -1,22 +1,35 @@ -# API Reference +# Local API reference -The Waggle local server runs on `http://localhost:3333`. All endpoints accept and return JSON unless otherwise noted. +The Waggle server runs on `http://localhost:3333` and listens on `127.0.0.1` only. The web UI +uses this API; it is not a stable public interface and can change between releases. Endpoints +accept and return JSON unless noted otherwise. The routes live in +`packages/server/src/local/routes/`. + +## Authentication + +Every `/api/*` request needs the session token as `Authorization: Bearer `. Server-Sent +Event streams that cannot send headers accept it as `?token=`. The web UI gets the token from +`GET /api/auth/session-token`, which answers only same-origin requests from the page the server +serves (the desktop app gets it through its own launch credential). Requests with a `Host` +header other than a local one are refused. + +This page lists the main route groups, not every route. ## Chat | Method | Path | Description | |--------|------|-------------| -| POST | `/api/chat` | Send a message and receive an SSE stream of agent responses. Body: `{ message, workspaceId, sessionId, model? }` | +| POST | `/api/chat` | Send a message and receive an SSE stream of the agent's reply. Body: `{ message, workspaceId?, sessionId?, model?, persona? }` | | GET | `/api/history` | Get conversation history. Query: `?session=ID&workspace=ID` | -The chat endpoint streams responses via Server-Sent Events (SSE). Events include `message`, `tool_call`, `tool_result`, `approval_request`, `done`, and `error`. +The chat endpoint streams the reply as Server-Sent Events; see [Chat stream events](#chat-stream-events). ## Workspaces | Method | Path | Description | |--------|------|-------------| | GET | `/api/workspaces` | List all workspaces | -| POST | `/api/workspaces` | Create a workspace. Body: `{ name, group, icon?, model?, personaId?, directory?, teamId? }` | +| POST | `/api/workspaces` | Create a workspace. Body: `{ name, group, icon?, model?, personaId?, templateId?, storageType?: 'virtual' \| 'local', storagePath? }` | | GET | `/api/workspaces/:id` | Get workspace by ID | | PUT | `/api/workspaces/:id` | Update workspace. Body: `{ name?, group?, icon?, model?, personaId? }` | | DELETE | `/api/workspaces/:id` | Delete workspace | @@ -27,20 +40,40 @@ The chat endpoint streams responses via Server-Sent Events (SSE). Events include | Method | Path | Description | |--------|------|-------------| -| GET | `/api/workspaces/:wid/sessions` | List sessions for a workspace, sorted by last active | -| POST | `/api/workspaces/:wid/sessions` | Create a new session. Body: `{ title? }` | -| GET | `/api/workspaces/:wid/sessions/search` | Search across sessions. Query: `?q=query&limit=20` | -| GET | `/api/workspaces/:wid/sessions/:sid/export` | Export session as Markdown. Returns `text/markdown` | -| PATCH | `/api/sessions/:id` | Rename a session. Body: `{ title }`. Query: `?workspace=ID` | -| DELETE | `/api/sessions/:id` | Delete a session. Query: `?workspace=ID` | +| GET | `/api/workspaces/:workspaceId/sessions` | List sessions for a workspace, sorted by last active | +| POST | `/api/workspaces/:workspaceId/sessions` | Create a new session. Body: `{ title? }` | +| GET | `/api/workspaces/:workspaceId/sessions/search` | Search across sessions. Query: `?q=query&limit=20` | +| GET | `/api/workspaces/:workspaceId/sessions/:sessionId/export` | Export a session as Markdown (`text/markdown`) | +| GET | `/api/workspaces/:workspaceId/sessions/:sessionId/timeline` | Tool events of a session | +| PATCH | `/api/sessions/:sessionId` | Rename a session. Body: `{ title }`. Query: `?workspace=ID` | +| DELETE | `/api/sessions/:sessionId` | Delete a session. Query: `?workspace=ID` | ## Memory +All memory routes go through the memory contract ([memory.md](../memory.md)). Some paths still +say "frames", the name used by the old memory engine; they return memory records. + | Method | Path | Description | |--------|------|-------------| -| GET | `/api/memory/search` | Search memory. Query: `?q=query&scope=all|personal|workspace&limit=20&workspace=ID` | -| GET | `/api/memory/frames` | Recent frames without search. Query: `?workspace=ID&limit=50` | -| GET | `/api/memory/graph` | Knowledge graph entities and relations. Query: `?workspace=ID` | +| GET | `/api/memory/capabilities` | What the memory store supports, and the extraction setting | +| PUT | `/api/memory/settings` | Turn background fact extraction on or off. Body: `{ extraction: boolean }` | +| GET | `/api/memory/search` | Search memory. Query: `?q=query&scope=all\|personal\|workspace\|global&workspace=ID&limit=20&since=&until=` (`all` is personal plus the given workspace; `global` is personal plus every workspace) | +| GET | `/api/memory/frames` | Newest personal and workspace memories. Query: `?workspace=ID&limit=50&since=&until=` | +| POST | `/api/memory/frames` | Save a memory. Body: `{ content, workspaceId?, importance?, kind?, source? }` | +| PUT | `/api/memory/frames/:id` | Edit a memory's text or importance | +| DELETE | `/api/memory/frames/:id` | Delete a memory | +| GET | `/api/memory/stats` | Memory counts. Query: `?workspace=ID` | +| GET | `/api/memory` | List memories. Query: `?mind=personal\|workspace&workspace=ID&kind=&status=&q=&limit=` | +| GET | `/api/memory/:id` | One memory | +| POST | `/api/memory` | Create a memory stated by the user | +| PATCH | `/api/memory/:id` | Edit text, importance, kind, title, tags, evidence or status | +| DELETE | `/api/memory/:id` | Delete a memory | +| POST | `/api/memory/:id/archive` | Archive a memory (reversible) | +| POST | `/api/memory/:id/confirm` | Mark a memory as confirmed by the user | +| GET | `/api/memory/:id/trace` | The execution that produced a memory ("why did you do that?") | +| POST | `/api/memory/erase` | Forget one memory (`{ id }`) or every memory saved from one chat (`{ sessionId }`) | +| POST | `/api/memory/merge` | Merge several memories of one scope into one | +| GET | `/api/memory/graph` | Always returns an empty graph: the knowledge graph is not part of this release | ## Agent @@ -67,9 +100,9 @@ The chat endpoint streams responses via Server-Sent Events (SSE). Events include | GET | `/api/settings` | Read config: default model, providers (masked keys), paths | | PUT | `/api/settings` | Update config. Body: `{ defaultModel?, providers? }` | | POST | `/api/settings/test-key` | Validate API key format. Body: `{ provider, apiKey }` | -| POST | `/api/settings/probe-provider` | Live-probe a STORED provider key by id. Body: `{ provider }` → `{ configured, valid, verified, error? }` | -| GET | `/api/settings/permissions` | Read permission settings (YOLO mode, external gates) | -| PUT | `/api/settings/permissions` | Update permissions. Body: `{ yoloMode?, externalGates?, workspaceOverrides? }` | +| POST | `/api/settings/probe-provider` | Test a stored provider key with the provider. Body: `{ provider }` → `{ configured, valid, verified, error? }` | +| GET | `/api/settings/permissions` | Read the approval settings | +| PUT | `/api/settings/permissions` | Update the approval settings. Body: `{ defaultAutonomy?, externalGates?, workspaceOverrides? }` (`yoloMode` is still accepted) | ## Vault @@ -120,7 +153,6 @@ The chat endpoint streams responses via Server-Sent Events (SSE). Events include | GET | `/api/marketplace/search` | Search catalog. Query: `?query=text&type=skill&category=cat&limit=20&offset=0` | | GET | `/api/marketplace/packs` | List all capability packs | | GET | `/api/marketplace/packs/:slug` | Pack detail with packages | -| GET | `/api/marketplace/enterprise-packs` | Enterprise packs (requires KVARK) | | POST | `/api/marketplace/install` | Install package. Body: `{ packageId, installPath?, settings?, force? }` | | POST | `/api/marketplace/uninstall` | Uninstall package. Body: `{ packageId }` | | GET | `/api/marketplace/installed` | List installed packages | @@ -176,23 +208,16 @@ The chat endpoint streams responses via Server-Sent Events (SSE). Events include | Method | Path | Description | |--------|------|-------------| -| POST | `/api/ingest` | Ingest files. Body: `{ files: [{ name, content (base64) }], workspaceId? }`. Max 15MB. | +| POST | `/api/ingest` | Ingest files. Body: `{ files: [{ name, content (base64) }], workspaceId? }`. Up to 10 MB per file; the request body is limited to 15 MB. | Supported types: images (PNG, JPG, GIF, WebP, SVG), documents (PDF, DOCX, PPTX), spreadsheets (XLSX, CSV), text/code (50+ extensions), archives (ZIP). -## Import - -| Method | Path | Description | -|--------|------|-------------| -| POST | `/api/import/preview` | Preview import from ChatGPT or Claude export. Body: `{ data, source: "chatgpt"|"claude" }` | -| POST | `/api/import/commit` | Import and save to personal memory. Body: `{ data, source }` | - ## Feedback | Method | Path | Description | |--------|------|-------------| -| POST | `/api/feedback` | Record feedback. Body: `{ sessionId, messageIndex, rating: "up"|"down", reason?, detail? }` | -| GET | `/api/feedback/stats` | Improvement stats: positive rate, top issues, corrections, trend | +| POST | `/api/feedback` | Record thumbs up or down on a reply. Body: `{ sessionId, messageIndex, rating: "up"\|"down", reason?, detail? }` | +| GET | `/api/feedback/stats` | Feedback statistics | ## Notifications @@ -206,19 +231,11 @@ Supported types: images (PNG, JPG, GIF, WebP, SVG), documents (PDF, DOCX, PPTX), |--------|------|-------------| | GET | `/api/audit/installs` | Recent install audit trail. Query: `?limit=20` | -## Team +## Built-in provider proxy | Method | Path | Description | |--------|------|-------------| -| POST | `/api/team/connect` | Connect to team server. Body: `{ serverUrl, token }` | -| POST | `/api/team/disconnect` | Disconnect from team server | -| GET | `/api/team/status` | Current team connection status | - -## Anthropic Proxy - -| Method | Path | Description | -|--------|------|-------------| -| POST | `/v1/chat/completions` | OpenAI-compatible endpoint that translates to Anthropic API. Used by internal agent loop. | +| POST | `/v1/chat/completions` | OpenAI-compatible endpoint used by the agent loop and memory extraction. Translates Claude requests to Anthropic's Messages API and forwards other providers' models to their OpenAI-compatible APIs | ## Commands @@ -226,17 +243,20 @@ Supported types: images (PNG, JPG, GIF, WebP, SVG), documents (PDF, DOCX, PPTX), |--------|------|-------------| | POST | `/api/commands/execute` | Execute a slash command. Body: `{ command, workspaceId, sessionId }` | -## SSE Streaming +## Chat stream events -The `/api/chat` endpoint uses Server-Sent Events for real-time streaming. Event types: +`POST /api/chat` answers with Server-Sent Events. The event names the chat route sends: | Event | Description | |-------|-------------| -| `message` | Agent text chunk | -| `tool_call` | Agent is calling a tool (shows tool name and input) | -| `tool_result` | Tool execution result | -| `approval_request` | Agent needs approval before proceeding | -| `approval_resolved` | Approval was granted or denied | -| `memory_saved` | A memory frame was auto-saved | -| `done` | Agent turn complete | -| `error` | Error occurred | +| `token` | A chunk of the agent's text | +| `step` | A progress step of the turn (recall, thinking, tool use and so on) | +| `tool` | The agent calls a tool | +| `tool_result` | The result of a tool call | +| `approval_required` | A tool call waits for the user's approval | +| `approval_held` | A tool call was held for later approval | +| `model_switch` | The turn moved to another model | +| `file_created` | A tool created a file | +| `notification` | A notification for the user | +| `done` | The turn is complete | +| `error` | The turn failed | diff --git a/docs/reference/commands.md b/docs/reference/commands.md index fe1ca42..720ae4b 100644 --- a/docs/reference/commands.md +++ b/docs/reference/commands.md @@ -1,27 +1,41 @@ -# Commands Reference +# Slash commands -Waggle provides 14 slash commands for quick actions. Type `/` in any workspace to see autocomplete suggestions. +Type `/` in a chat to see the commands with autocomplete. A command is `/name` followed by its +arguments; aliases work the same way as the name. Many commands hand a prepared request to the +agent, which then uses its normal tools and approvals. -## Command List +The commands are defined in `packages/agent/src/commands/` (`workflow-commands.ts` and +`marketplace-commands.ts`). -| Command | Aliases | Args | Description | -|---------|---------|------|-------------| -| `/catchup` | `/catch-up`, `/recap` | -- | Workspace restart summary. Shows recent activity, memories, and state so you can get up to speed instantly. | -| `/now` | `/current`, `/where` | -- | Current workspace state. What is happening right now -- active threads, recent changes, open items. | -| `/research` | `/investigate` | `` | Launch multi-agent research. Spawns the `research-team` workflow with multiple agents investigating the topic in parallel. | -| `/draft` | `/write` | ` [topic]` | Start a drafting workflow. Supports any document type: blog, report, email, proposal, memo. Uses the `review-pair` workflow for drafts with review. | -| `/decide` | `/decision`, `/weigh` | `` | Create a structured decision matrix with options, pros/cons, risk, and effort columns. Provides an evaluation framework for systematic decision-making. | -| `/review` | `/critique`, `/check` | -- | Review the last output with a critic agent. Spawns the `review-pair` workflow to analyze the most recent response for accuracy and completeness. | -| `/spawn` | `/agent`, `/summon` | ` [task]` | Spawn a specialist sub-agent. Available roles: `researcher`, `writer`, `coder`, `analyst`, `reviewer`, `planner`. | -| `/skills` | `/abilities`, `/tools` | -- | Show active skills in this workspace. Lists all loaded skill files with their names. | -| `/status` | `/report`, `/progress` | -- | Project status summary. Shows workspace state including tasks, progress, blockers, and recent activity. | -| `/memory` | `/remember`, `/recall` | `[query]` | Search workspace memory. Without arguments, shows usage help. With a query, searches both personal and workspace minds. | -| `/plan` | `/decompose`, `/break-down` | `` | Break a goal into actionable tasks. Uses the `plan-execute` workflow to create a structured task list with dependencies. | -| `/focus` | `/narrow`, `/scope` | `` | Narrow agent focus to a specific topic. Subsequent responses prioritize this context. Use again to change focus. | -| `/marketplace` | `/mp`, `/market` | ` [args]` | Marketplace operations. Sub-commands: `search`, `install`, `packs`, `installed`, `sync`. See details below. | -| `/help` | `/commands`, `/?` | -- | List all available commands with descriptions. | +## Command list -## Detailed Examples +| Command | Aliases | Arguments | What it does | +|---|---|---|---| +| `/catchup` | `/catch-up`, `/recap` | | Summary of where the workspace stands: recent activity, memories and state | +| `/now` | `/current`, `/where` | | What is happening in the workspace right now | +| `/status` | `/report`, `/progress` | | Project status: tasks, progress, blockers and recent activity | +| `/research` | `/investigate` | `` | Runs the `research-team` workflow on a topic | +| `/draft` | `/write` | ` [topic]` | Drafts a document (memo, report, email, proposal…) with a review cycle | +| `/decide` | `/decision`, `/weigh` | `` | Builds a decision matrix with options, pros, cons, risk and effort | +| `/review` | `/critique`, `/check` | | Has a critic agent review the last answer | +| `/plan` | `/decompose`, `/break-down` | `` | Breaks a goal into a task list | +| `/spawn` | `/agent`, `/summon` | ` [task]` | Starts a specialist sub-agent: `researcher`, `writer`, `coder`, `analyst`, `reviewer` or `planner` | +| `/focus` | `/narrow`, `/scope` | `` | Narrows the agent's focus to a topic until you change it | +| `/memory` | `/remember`, `/recall` | `` | Searches personal and workspace memory | +| `/search-all` | `/find-all` | `` | Searches personal memory and all workspaces | +| `/skills` | `/abilities`, `/tools` | | Lists the active skills | +| `/plugins` | | | Lists installed plugins and capabilities | +| `/connectors` | `/integrations`, `/connections` | | Lists connected services and their status | +| `/marketplace` | `/mp`, `/market` | ` [args]` | Marketplace: `search`, `install`, `packs`, `installed`, `sync` | +| `/workflow` | `/wf` | ` [args]` | Creates, lists or runs multi-agent workflows | +| `/export` | | `` | Exports workspace data with the agent's help | +| `/import` | | `` | Reads a file, URL or pasted text and saves what matters as workspace memories | +| `/cli` | `/cli-tools` | `[allow\|deny\|discover] [name]` | Shows or changes which command-line programs the agent may run | +| `/pr` | `/pull-request`, `/merge-request` | `` | Creates a pull request from the current branch of the workspace's repository | +| `/settings` | | | Shows the current workspace and agent settings | +| `/help` | `/commands`, `/?` | | Lists all commands | + +## Examples ### /catchup @@ -29,72 +43,55 @@ Waggle provides 14 slash commands for quick actions. Type `/` in any workspace t /catchup ``` -Output: A structured briefing with what this workspace is about, recent sessions, decisions, memory count, and suggested next steps. Use this every time you return to a workspace after a break. +A briefing for when you come back to a workspace: what it is about, recent sessions, decisions +and suggested next steps. ### /research ``` -/research quantum computing applications in drug discovery +/research applications of quantum computing in drug discovery ``` -Spawns the `research-team` multi-agent workflow: -1. A researcher agent investigates the topic -2. Findings are synthesized and cross-referenced -3. Results are presented with citations and confidence levels +Runs the `research-team` workflow: research agents investigate the topic and their findings are +combined into one answer. ### /draft ``` -/draft blog post about AI safety best practices -/draft report quarterly metrics for Q1 2026 -/draft email to client about project timeline changes +/draft report quarterly metrics for Q1 +/draft email to the client about the timeline change ``` -The agent determines the document type from the first word and adapts its formatting accordingly. If the `review-pair` workflow is available, the draft goes through a review cycle. +The first word is the document type. ### /decide ``` -/decide Should we use PostgreSQL or MongoDB for the new project? +/decide Should we use PostgreSQL or MongoDB for the new service? ``` -Produces a decision matrix template: - -| Option | Pros | Cons | Risk | Effort | -|--------|------|------|------|--------| -| PostgreSQL | | | | | -| MongoDB | | | | | - -Ask the agent to fill it in: "Please analyze this decision for me." +Produces a matrix with a row per option and columns for pros, cons, risk and effort. Ask the +agent to fill it in. ### /spawn ``` -/spawn researcher Find recent papers on transformer architectures -/spawn coder Implement a rate limiter middleware -/spawn analyst Compare our metrics against industry benchmarks +/spawn researcher Find recent papers on retrieval-augmented generation +/spawn coder Write a rate limiter middleware ``` -The sub-agent runs with the specified role and task, then returns its results to the main conversation. +The sub-agent works on the task and returns its result to the conversation. -### /memory +### /memory and /search-all ``` /memory architecture decisions -/memory last meeting notes -/memory project goals +/search-all project deadlines ``` -Searches both personal and workspace memory using FTS5 full-text search. Results include content, importance level, and dates. - -### /plan - -``` -/plan Build a user dashboard with analytics -/plan Migrate the database to PostgreSQL -``` - -Creates a structured plan with ordered steps, dependencies, and success criteria. The agent can then execute steps sequentially. +`/memory` searches personal memory and the current workspace's memory. `/search-all` also +searches every other workspace and groups the results by workspace. See +[memory.md](../memory.md). ### /marketplace @@ -106,33 +103,32 @@ Creates a structured plan with ordered steps, dependencies, and success criteria /marketplace sync ``` -| Sub-command | Description | -|-------------|-------------| -| `search <query>` | Search the marketplace catalog by keyword. Returns up to 10 results with name, type, category, and description. | -| `install <name>` | Install a package by name. Searches for the best match, runs SecurityGate scan, and installs. | -| `packs` | List all capability packs grouped by priority tier. | -| `installed` | Show all currently installed packages with status and install date. | -| `sync` | Trigger a manual sync from all configured marketplace sources. Reports added/updated counts. | +| Subcommand | What it does | +|---|---| +| `search <query>` | Searches the catalog, up to 10 results | +| `install <name>` | Finds the best match, runs the security scan and installs it | +| `packs` | Lists the capability packs | +| `installed` | Lists installed packages | +| `sync` | Refreshes the catalog from its sources | -## Command Parsing +### /workflow -Commands follow the format `/commandName arg1 arg2 ...`. The parser: +``` +/workflow list +/workflow run research-team +/workflow create Research competitors, summarize findings and draft an executive brief +``` -1. Strips the leading `/` -2. Splits on the first space to get the command name and arguments -3. Resolves aliases (e.g., `/mp` resolves to `/marketplace`) -4. Executes the handler with the argument string and workspace context +## Adding a command -Partial matching is supported for autocomplete but not for execution. You must type the full command name or alias. +Commands are `CommandDefinition` objects registered with the `CommandRegistry` +(`packages/agent/src/commands/command-registry.ts`): -## Adding Custom Commands +- `name`: the command name, lowercase, without `/`; +- `aliases`: other names, also without `/`; +- `description` and `usage`: shown in `/help`; +- `handler(args, context)`: returns the reply text. A reply that starts with + `AGENT_LOOP_REROUTE_PREFIX` is sent to the agent as a request instead of shown directly. -Commands are registered in the `CommandRegistry`. To add a custom command, create a `CommandDefinition` with: - -- `name` -- the primary command name (lowercase) -- `aliases` -- alternative names -- `description` -- shown in `/help` output -- `usage` -- usage string with argument placeholders -- `handler` -- async function receiving `(args: string, context: CommandContext)` - -See `packages/agent/src/commands/` for implementation examples. +Register a new command in `registerWorkflowCommands` or in a registration function of your +own, called from the sidecar (`packages/server/src/local/index.ts`).