From e0836388f10632b8e9be5ab94727f748291789cc Mon Sep 17 00:00:00 2001 From: "shumeiko.it" Date: Wed, 4 Nov 2026 10:31:47 +0100 Subject: [PATCH] docs: record the separate data directory (ADR-0011) Add ADR-0011: the community edition defaults to ~/.waggle-community, a temporary name that follows the product-name decision (Q8), and copies nothing from the commercial ~/.waggle. It supersedes point 4 of ADR-0009 and closes the pending "separate data directory" decision. Update the install guide, troubleshooting, capabilities, memory, architecture, the Windows build notes, THREAT_MODEL, README, .env.example and the changelog. Log UF-068 (Open): marketplace skills and plugins ignore a custom data directory. --- .env.example | 7 +-- AGENTS.md | 4 ++ CHANGELOG.md | 5 +- README.md | 3 +- THREAT_MODEL.md | 5 +- docs/ARCHITECTURE.md | 3 +- docs/guides/capabilities.md | 2 +- docs/guides/troubleshooting.md | 3 +- docs/install.md | 17 +++++-- docs/memory.md | 2 +- docs/oss/adr/0009-installer-server-archive.md | 2 +- docs/oss/adr/0011-separate-data-directory.md | 47 +++++++++++++++++++ docs/oss/adr/README.md | 4 +- docs/oss/build-windows.md | 4 +- docs/oss/research/memory-search-quality.md | 2 +- docs/oss/upstream-fixes.md | 10 ++++ 16 files changed, 99 insertions(+), 21 deletions(-) create mode 100644 docs/oss/adr/0011-separate-data-directory.md diff --git a/.env.example b/.env.example index 94f4f8e..ad7b971 100644 --- a/.env.example +++ b/.env.example @@ -1,6 +1,6 @@ # Optional settings for running the sidecar from source. A basic local run # needs none of them: keys are entered in the app and kept in the encrypted -# vault (~/.waggle/vault.json). +# vault (~/.waggle-community/vault.json). # ── Provider API keys ────────────────────────────────────────────── # Set keys in the app under Settings > Models (Provider Keys) or in the Vault @@ -15,7 +15,8 @@ # VOYAGE_API_KEY= # embeddings only (EMBEDDING_PROVIDER=voyage) # ── Data directory and port ──────────────────────────────────────── -# WAGGLE_DATA_DIR=~/.waggle +# Default: ~/.waggle-community, kept apart from the commercial Waggle's ~/.waggle. +# WAGGLE_DATA_DIR=~/.waggle-community # WAGGLE_PORT=3333 # ── LiteLLM (optional) ───────────────────────────────────────────── @@ -30,7 +31,7 @@ # EMBEDDING_PROVIDER=auto # In-process model: downloads about 23 MB on first use, cached in -# ~/.waggle/models/, then works offline. +# ~/.waggle-community/models/, then works offline. # EMBEDDING_MODEL=Xenova/all-MiniLM-L6-v2 # Ollama: needs Ollama running (https://ollama.com). diff --git a/AGENTS.md b/AGENTS.md index e464a8f..cb12e6a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -37,6 +37,10 @@ The community edition: packages, without Python, Docker or a compiler on the user's machine; - has no monetization: no tiers, trials, payments, license checks or upsell (ADR-0001); - keeps memory on the user's machine through a memory contract backed by local mem0 (ADR-0002); +- keeps its data in `~/.waggle-community` or `WAGGLE_DATA_DIR`, apart from the commercial + edition's `~/.waggle` (ADR-0011). The name is temporary; code takes it from + `DEFAULT_DATA_DIR_NAME` in `@waggle/shared` (the Rust shell, the launcher and the + marketplace keep a checked copy); - sends analytics only after the user opts in (ADR-0006). ### Stack (verified 2026-10-15) diff --git a/CHANGELOG.md b/CHANGELOG.md index 43565ab..f2e2780 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -39,11 +39,14 @@ lists what changed for users compared with that product. The reasons are recorde ### Changed - **Memory runs on local mem0** behind a memory contract (ADR-0002, ADR-0007). Memories are kept - in `~/.waggle/memory/`, embedded by a model that runs on your computer, and searched with + in `~/.waggle-community/memory/`, embedded by a model that runs on your computer, and searched with mem0's combined vector and keyword search. Nothing goes to a cloud memory service. Memories from the old `.mind` files are not carried over. See [docs/memory.md](docs/memory.md). - **Waggle's own data** (automations, notifications, cost tracking, your profile card and task list) moved to a separate `waggle.db`, apart from memory. +- **The data directory is `~/.waggle-community`** (ADR-0011), so the community edition never + opens the databases of the commercial Waggle in `~/.waggle`. Nothing is copied from + `~/.waggle`; set `WAGGLE_DATA_DIR` to use another directory. The name is temporary. - **Analytics are opt-in** (ADR-0006). Waggle asks once; nothing is sent until you choose **Allow**, and Settings → General → Privacy & Telemetry turns it off again. When allowed, Waggle sends one event at the end of setup with the template, persona and model you chose. diff --git a/README.md b/README.md index c72f281..0a23707 100644 --- a/README.md +++ b/README.md @@ -65,7 +65,8 @@ sidecar: it registers the files in `dist/` when it starts. The Vite dev server (`npm run dev:web`) cannot sign in to the sidecar yet. Workspace packages are resolved from their TypeScript source, so no package build is needed -before running or testing. Data goes to `~/.waggle` unless you set `WAGGLE_DATA_DIR`. +before running or testing. Data goes to `~/.waggle-community` unless you set `WAGGLE_DATA_DIR`; +the commercial Waggle's `~/.waggle` is not used. Add a model in the app (Settings → Models). `.env.example` lists optional environment variables; a basic local run needs none of them. diff --git a/THREAT_MODEL.md b/THREAT_MODEL.md index afc3ea6..a034d74 100644 --- a/THREAT_MODEL.md +++ b/THREAT_MODEL.md @@ -80,7 +80,10 @@ when, why, and by whom. machine-local key file; each entry is independently encrypted. Runtime API keys belong in the vault or a local `.env` (never committed). `.env.example` contains names plus non-secret development defaults, never live credentials. Callers must -not persist secret values into prompts, logs, or memory frames. +not persist secret values into prompts, logs, or memory frames. The vault and its +key file live in the data directory, `~/.waggle-community` by default, which is +separate from the commercial edition's `~/.waggle` (ADR-0011): neither edition +opens the other's vault. ### 6. MCP tool scope gate (removed) The memory MCP servers this gate protected are not part of the community edition diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 0c42cd1..32aec32 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -128,7 +128,8 @@ loads the `apps/web` build. The shell: ## Data -One data directory per user: `WAGGLE_DATA_DIR`, or `~/.waggle`. [install.md](install.md#where-your-data-lives) +One data directory per user: `WAGGLE_DATA_DIR`, or `~/.waggle-community`, separate from the +commercial edition's `~/.waggle` (ADR-0011). [install.md](install.md#where-your-data-lives) lists what it contains. - **Memory** goes only through `MemoryStore` (`@waggle/memory-contract`). The shipped store keeps diff --git a/docs/guides/capabilities.md b/docs/guides/capabilities.md index 22b2823..73769f2 100644 --- a/docs/guides/capabilities.md +++ b/docs/guides/capabilities.md @@ -34,7 +34,7 @@ set of skills for working with other people; it does not need team mode. ## Writing your own skill -A skill is a Markdown file in `/skills/` (by default `~/.waggle/skills/`). Optional YAML +A skill is a Markdown file in `/skills/` (by default `~/.waggle-community/skills/`). Optional YAML frontmatter gives it a name and other metadata: ```markdown diff --git a/docs/guides/troubleshooting.md b/docs/guides/troubleshooting.md index 2255eb3..15bac55 100644 --- a/docs/guides/troubleshooting.md +++ b/docs/guides/troubleshooting.md @@ -1,7 +1,8 @@ # Troubleshooting Common problems and what to do about them. Paths use `` for the data directory: -`~/.waggle` by default (`%USERPROFILE%\.waggle` on Windows), or `WAGGLE_DATA_DIR`. +`~/.waggle-community` by default (`%USERPROFILE%\.waggle-community` on Windows), or +`WAGGLE_DATA_DIR`. ## Installing diff --git a/docs/install.md b/docs/install.md index 4af2d83..bdfdf7e 100644 --- a/docs/install.md +++ b/docs/install.md @@ -87,7 +87,8 @@ sha256sum Waggle__amd64.deb The window shows "Starting Waggle…" while the app starts its own server in the background. The server listens on `127.0.0.1` port 3333, or on another free port if 3333 is taken. Its log is in -`~/.waggle/logs/service.log` (on Windows `%USERPROFILE%\.waggle\logs\service.log`). +`~/.waggle-community/logs/service.log` (on Windows +`%USERPROFILE%\.waggle-community\logs\service.log`). ### Update the desktop app @@ -168,7 +169,7 @@ it. It listens on this computer only (`127.0.0.1`). |---|---| | `--port `, `-p` | Use another port (default 3333) | | `--no-open` | Do not open the browser | -| `--data-dir ` | Keep data in another directory (default `$WAGGLE_DATA_DIR` or `~/.waggle`) | +| `--data-dir ` | Keep data in another directory (default `$WAGGLE_DATA_DIR` or `~/.waggle-community`) | | `--dir ` | Use a program directory other than the default | If Waggle already runs on the port, `waggle-server` only opens the browser. @@ -244,9 +245,15 @@ send. ## Where your data lives -Both ways of installing use one data directory: `~/.waggle` (on Windows -`%USERPROFILE%\.waggle`). Set `WAGGLE_DATA_DIR` to use another directory; the server installer -also takes `--data-dir`. +Both ways of installing use one data directory: `~/.waggle-community` (on Windows +`%USERPROFILE%\.waggle-community`). Set `WAGGLE_DATA_DIR` to use another directory; the server +installer also takes `--data-dir`. + +The community edition does not touch `~/.waggle`, the data directory of the commercial Waggle, +unless you point `WAGGLE_DATA_DIR` at it, so the two editions do not read or write each other's +databases. Nothing is copied from +`~/.waggle`: the community edition starts with an empty data directory. The directory name is +temporary until the product name is decided. | Path in the data directory | What it holds | |---|---| diff --git a/docs/memory.md b/docs/memory.md index f48d141..39a5180 100644 --- a/docs/memory.md +++ b/docs/memory.md @@ -77,7 +77,7 @@ A store reports what it can do, and the web UI hides what it cannot: ## Where mem0 keeps its data -Everything is in `memory/` inside the data directory (`~/.waggle/memory/` by default): +Everything is in `memory/` inside the data directory (`~/.waggle-community/memory/` by default): | File | Content | |---|---| diff --git a/docs/oss/adr/0009-installer-server-archive.md b/docs/oss/adr/0009-installer-server-archive.md index 7390709..9ac59fb 100644 --- a/docs/oss/adr/0009-installer-server-archive.md +++ b/docs/oss/adr/0009-installer-server-archive.md @@ -1,6 +1,6 @@ # ADR-0009: The installer delivers a prebuilt server archive -- Status: Accepted +- Status: Accepted; point 4 superseded by ADR-0011 - Date: 2026-11-01 ## Context diff --git a/docs/oss/adr/0011-separate-data-directory.md b/docs/oss/adr/0011-separate-data-directory.md new file mode 100644 index 0000000..9bc00ec --- /dev/null +++ b/docs/oss/adr/0011-separate-data-directory.md @@ -0,0 +1,47 @@ +# ADR-0011: The community edition has its own data directory + +- Status: Accepted +- Date: 2026-11-04 +- Supersedes: point 4 of ADR-0009 + +## Context + +The community edition was made from the commercial Waggle and kept its default data directory, +`~/.waggle` (`%USERPROFILE%\.waggle` on Windows). On a computer with both editions, the two +would open the same `config.json`, vault, `waggle.db`, marketplace catalog, skills and logs. Their +schemas and features have diverged: the community edition ignores the commercial keys in +`config.json` (ADR-0001) and the commercial `.mind` memory files (ADR-0007), and the commercial +edition does not know the mem0 store. Sharing one directory risks one edition changing data the +other relies on. + +## Decision + +1. **The default data directory is `~/.waggle-community`** (`%USERPROFILE%\.waggle-community` on + Windows) for the sidecar, the CLI, the marketplace, the desktop shell and the server + installer. The project owner chose this on 2026-11-04. +2. **`WAGGLE_DATA_DIR` still overrides the default** wherever it did before. +3. **Nothing is migrated.** The community edition does not read, copy or move anything from + `~/.waggle`. The owner explicitly does not want a migration: the product is still in + development and nobody has data to bring over. +4. **The name is temporary.** The final name follows the product-name decision (Q8). It is defined + once per runtime so that the rename is one change in each: + - TypeScript: `DEFAULT_DATA_DIR_NAME` and `DEFAULT_DATA_DIR_LABEL` in `@waggle/shared`, with + `defaultDataDir()` in `@waggle/core` for Node code; + - the Tauri shell: `DEFAULT_DATA_DIR_NAME` in `app/src-tauri/src/lib.rs`; + - the server installer, which uses only Node's built-in modules: `DATA_DIR_NAME` in + `packages/launcher/src/platform.ts`; + - the marketplace, which installs on its own without the workspace packages: + `DATA_DIR_NAME` in `packages/marketplace/src/data-dir.ts`. + + `packages/shared/tests/data-dir.test.ts` fails when these copies differ. The CLI help text, + which loads no dependency, spells the path out and `packages/cli/tests/cli-runtime.test.ts` + checks it. + +## Consequences + +- A user who ran an earlier build of the community edition finds an empty data directory and + sets Waggle up again. +- The commercial keys stay untouched in `config.json` (ADR-0001), which still matters when a user + points `WAGGLE_DATA_DIR` at `~/.waggle`. +- The marketplace security scan flags skills that reference the skills folder of either + directory. diff --git a/docs/oss/adr/README.md b/docs/oss/adr/README.md index 1743a52..14f3d07 100644 --- a/docs/oss/adr/README.md +++ b/docs/oss/adr/README.md @@ -14,8 +14,9 @@ decision, add a new record that supersedes the old one. | [0006](0006-opt-in-analytics.md) | Analytics are opt-in | Accepted, one open item | | [0007](0007-memory-contract.md) | The memory contract and how it maps onto mem0 | Accepted | | [0008](0008-native-modules.md) | Native modules ship with a pinned Node runtime | Accepted | -| [0009](0009-installer-server-archive.md) | The installer delivers a prebuilt server archive | Accepted | +| [0009](0009-installer-server-archive.md) | The installer delivers a prebuilt server archive | Accepted, point 4 superseded by ADR-0011 | | [0010](0010-marketplace-catalog-in-repo.md) | The marketplace catalog stays in the repository | Accepted | +| [0011](0011-separate-data-directory.md) | The community edition has its own data directory | Accepted | ## Pending decisions @@ -28,5 +29,4 @@ work packages wait for the answer. | Product name and system identifiers | Temporary names | P1.3d, P2.6 | | Location of the public repository | `release.yml` is ready but has not run; the installer has no default download location (`--from`) | P2.6, P2.7 | | Installer signing | First release may ship unsigned with launch instructions | P2.5 | -| Separate data directory from the commercial edition | Not separated yet | P1.3c | | PostHog project for the community edition | See ADR-0006 | P1.3a | diff --git a/docs/oss/build-windows.md b/docs/oss/build-windows.md index e5de1b2..92300ed 100644 --- a/docs/oss/build-windows.md +++ b/docs/oss/build-windows.md @@ -113,10 +113,10 @@ node scripts/release-smoke.mjs desktop --app "$env:LOCALAPPDATA\Waggle\waggle.ex 2. Start Waggle from the Start menu. The window should move past "Starting Waggle…" to the Home screen. 3. If it stays on "Starting Waggle…", close it and take the log from - `%USERPROFILE%\.waggle\logs\service.log`. + `%USERPROFILE%\.waggle-community\logs\service.log`. 4. Chat needs a model: an API key in Settings → Models (Provider Keys), or a running Ollama. -Waggle keeps its data in `%USERPROFILE%\.waggle`. +Waggle keeps its data in `%USERPROFILE%\.waggle-community`. ## Troubleshooting diff --git a/docs/oss/research/memory-search-quality.md b/docs/oss/research/memory-search-quality.md index f4883da..9ec8cc0 100644 --- a/docs/oss/research/memory-search-quality.md +++ b/docs/oss/research/memory-search-quality.md @@ -329,7 +329,7 @@ npx tsx packages/memory-mem0/bench/search-quality.ts --engines mem0,cosine npx tsx packages/memory-mem0/bench/search-quality.ts --out /tmp/search-quality.json ``` -The in-process model must be in `~/.waggle/models` (the sidecar downloads it on first start) +The in-process model must be in `~/.waggle-community/models` (the sidecar downloads it on first start) or in the directory named by `WAGGLE_MODELS_DIR`; the bench refuses network access and does not download it. diff --git a/docs/oss/upstream-fixes.md b/docs/oss/upstream-fixes.md index 04a4579..1857583 100644 --- a/docs/oss/upstream-fixes.md +++ b/docs/oss/upstream-fixes.md @@ -96,6 +96,7 @@ Baseline: commit `b20b138` of the main Waggle repository. | UF-065 | The Windows uninstaller names a "Data & Privacy" settings page that does not exist | Fixed | Desktop, Windows installer | | UF-066 | Installed builds have no capability packs and no starter skills | Fixed | Sidecar bundle, SDK | | UF-067 | The status bar names a default model that chats cannot use | Fixed | Sidecar, Web UI | +| UF-068 | Marketplace skills and plugins ignore the configured data directory | Open | Marketplace, sidecar | ## Entries @@ -675,3 +676,12 @@ Entries UF-010 to UF-019 come from the UI audit `anti-slop/audit-001-2026-10-14. - Cause: the chat route resolves the configured model to a usable one (`resolveUsableModel`), but the status endpoint returns the configured model as is. - What we did: the status endpoint resolves the model the same way, without changing the configured one. A test covers it. - Recommendation: same change in the main project. + +### UF-068. Marketplace skills and plugins ignore the configured data directory +- Status: Open +- Found: 2026-11-04 +- Where: at commit `b20b138`, `packages/marketplace/src/installer.ts:47-50` (`SKILLS_DIR`, `PLUGINS_DIR`, `REGISTRY_PATH`); the sidecar reads `/skills` and `/plugins` (`packages/server/src/local/routes/skills.ts:43`) +- Symptom: with `WAGGLE_DATA_DIR` or a custom data directory, a skill or plugin installed from the Marketplace is written to the default directory in the home folder, so the sidecar does not list or load it. +- Cause: the installer builds these paths from `os.homedir()` when its module loads. Only `.mcp.json` honors `WAGGLE_DATA_DIR`, and the MCP route also saves the entry in the server's data directory to make up for it. +- What we did: nothing yet. The community edition changed the default directory name only (ADR-0011). +- Recommendation: pass the server's data directory to `MarketplaceInstaller`, or resolve the paths at call time from `WAGGLE_DATA_DIR` like `mcpConfigPath()`.