Commit Graph

16 Commits

Author SHA1 Message Date
shumeiko.it
101523e463 test(server): cover background extraction, the legacy memory copy and run results 2026-10-24 12:00:00 +02:00
shumeiko.it
2d507dc457 fix(agent): make the recall reranker opt-in and use the word embedder in tests
The reference-set comparison showed the cross-encoder adds nothing to mem0's
ranking and costs about 130 ms per search.
2026-10-24 11:57:18 +02:00
shumeiko.it
625545aa0f feat(server): run the WhatsApp channel on whatsapp-web.js (P1.7)
Baileys depends on libsignal (GPL-3.0), which the sidecar bundled. The
WhatsApp adapter now drives WhatsApp Web through whatsapp-web.js and
keeps its public shape: start, stop, status with the pairing QR, send
and inbound text. LocalAuth stores the session as a browser profile in
<dataDir>/channels/whatsapp; the old Baileys state is left untouched.

The browser comes from whatsapp-browser.ts: CHROME_PATH or
PUPPETEER_EXECUTABLE_PATH, then an installed Chrome, Edge or Chromium
(Windows, macOS, Linux), then a Chrome for Testing build downloaded on
the first start into <dataDir>/browsers, with progress in the status.
The root .puppeteerrc.cjs stops npm install from downloading a browser.
The sidecar build keeps whatsapp-web.js external: bundled, it fails on
__dirname and would inline LGPL node-webpmux. Text only, so ffmpeg is
never needed.

New dependencies, pinned: whatsapp-web.js 1.34.7 (Apache-2.0) and
@puppeteer/browsers 2.13.0 (Apache-2.0). Transitive: puppeteer,
puppeteer-core, chromium-bidi, webdriver-bidi-protocol (Apache-2.0),
devtools-protocol (BSD-3-Clause), extract-zip (BSD-2-Clause),
fluent-ffmpeg and the rest (MIT or ISC), and node-webpmux
(LGPL-3.0-or-later, shipped as a separate package; needs the license
review). Removed: @whiskeysockets/baileys and libsignal.

UF-020 is now Fixed; ADR-0005 records the consequences.
2026-10-24 11:56:38 +02:00
shumeiko.it
f046a20b34 test(memory): compare search quality of the old engine and mem0 on a reference set (P1.5)
Add a reference set (100 invented memories across personal memory and two
workspaces, 67 queries: paraphrase, keyword, multi-fact, scope, no-answer)
and a tsx benchmark that loads it into the old engine (MindDB, FrameStore,
HybridSearch, with and without the cross-encoder reranker), the mem0 store
and a plain-cosine baseline, all on Waggle's in-process embedder. It reports
recall@1, recall@5, MRR, latency and misses per engine and query type. The
mem0 half runs on its own (--engines mem0,cosine) once the old engine is
removed. Ollama and the user's own key are optional embedders; the bench
refuses network access to anything else and never reads OPENAI_API_KEY.

Results in docs/oss/research/memory-search-quality.md: mem0 matches the old
engine's chat-recall path (recall@1 0.72, recall@5 0.89 vs 0.90, MRR 0.82
vs 0.83) at a fraction of its latency and is acceptable for v1.

No new dependencies.
2026-10-24 11:56:35 +02:00
shumeiko.it
784ca770b1 refactor(memory): run agent recall, memory tools and the server on the memory contract (P1.5)
- Orchestrator, memory tools, context loader and pattern write-back use a
  MemoryStore and a workspace scope instead of MindDB layers; the knowledge
  graph tools, cognify, memory linker and feedback handler are gone (graph
  hidden in v1, ADR-0002).
- The server opens one mem0 store in <dataDir>/memory/, embedded by the
  in-process model on first use, and extracts facts about the user in the
  background with the model the user chose (memoryExtraction setting).
- Legacy .mind memories are copied into the store once, in the background;
  the legacy files are only read.
- Weaver consolidation, harvest, wiki, memory import, memory lanes, vector
  backfill and index reconciliation are removed; agent runs, loops, sub-agent
  results, connector fetch and the failed-turn capture write through the
  contract. Workspace sessions no longer hold a mind handle.
- FileIndexer and the install audit move to waggle.db; recall date helpers and
  the reranker are copied into core.
- The CLI REPL opens the same memory store.
2026-10-24 11:56:26 +02:00
shumeiko.it
0325c4eb11 fix(memory-mcp): keep reading profile and awareness from the .mind file
The memory MCP server serves the .mind file directly, so it uses the
substrate's IdentityLayer and AwarenessLayer instead of the waggle.db ones
that @waggle/core now exports. The WaggleDB-only constructors caught the
mismatch at build time.
2026-10-23 14:51:20 +02:00
shumeiko.it
ad2ca875cb feat(core): give Waggle its own database, apart from memory (P1.5)
Waggle kept its own tables in the memory database. Before the memory
engine can be replaced by mem0 they need a home of their own.

- add WaggleDB (<dataDir>/waggle.db). Automations, notifications, held
  actions, skill hashes, thumbs feedback, cost tracking, the profile card
  and awareness items live there. Their stores take a WaggleDB, so passing
  a memory database is a type error.
- the profile card and awareness items are now per scope ('personal' or
  'workspace:<id>') in one table each, instead of one table per .mind file
- on the first start, copy these tables from personal.mind and every
  workspace.mind (importLegacyMindData). The legacy files are only read,
  each scope is copied once and a table only while its target is empty.
- copy the logger, injection scanner, external-memory ingress guard,
  WorkspaceManager and the embedders from @waggle/hive-mind-core into
  @waggle/core, which now re-exports its own copies
- point the server, agent orchestrator and CLI at waggle.db; update the
  tests that seeded these tables in a memory database

ADR-0007 records the layout and the licensing question about the copied
files.
2026-10-23 14:31:28 +02:00
shumeiko.it
5f088698dd chore: remove internal documents and service directories (P2.2 part 1)
The commercial repository carries the product's internal history: audits,
screenshots, briefs, decisions, pricing, publication drafts, personal data
and research runs. None of it belongs in the community edition.

- delete judging/, "UAT 3/", benchmarks/ (including LoCoMo data under
  CC BY-NC), preflight-results/, gepa-phase-5/, notes/, decisions/,
  .understand-anything/, .agents/, .lovable/, the legacy sidecar/ package
  and waggle-cowork/ (an analysis of Claude Code sources)
- delete docs/ except docs/oss, the user guides (without the team-mode
  guide), the reference, methodology, architecture, getting-started and
  contributing pages, which P2.4 rewrites
- delete render.yaml, .parity-allowlist, the retired mind-parity and
  sync-mind workflows and 29 one-off scripts nothing references
- delete tests that only checked removed files (sidecar package, Render
  blueprint, UAT capture, benchmark harness isolation, internal audit and
  signing docs); drop sidecar/ and benchmarks/ from the hive-mind subtree
  split guard; prune ignore, attribute, lint and test globs
- replace the commercial README with an interim one for this edition

Source comments still point at some deleted docs; P2.2 part 2 cleans them
together with names and paths.
2026-10-20 01:02:18 +02:00
shumeiko.it
2e1999a3f6 refactor: remove self-learning and the optimizer package (P1.4)
Self-learning is excluded from the community edition (ADR-0004); the
project owner reworks it separately. Cost tracking stays.

- agent: delete behaviour evolution (evolve-schema, iterative GEPA,
  compose-evolution, evolution gates, orchestrator and deploy),
  improvement and correction detection, AgentLearning, the eval-dataset
  builder, optimization-capture and the agent_insights tool. Secret
  detection moves from eval-dataset to secret-patterns. The LLM wiring
  keeps only the Haiku adapter and judge call used by skill audit and
  hygiene, so @ax-llm/ax is now declared by agent instead of server.
- server: delete /api/evolution/*, the evolution and optimizer services,
  GEPA expansion of the first chat message, the prompt_optimization and
  monthly_assessment jobs, the idle-session reviewer and its
  session-reviewer persona, prompt injection of past corrections and
  thumbs-down recorded as correction signals. Existing job rows of the
  removed types are ignored, not deleted.
- core: delete OptimizationLogStore; stop re-exporting the improvement
  signal and evolution run stores.
- web: delete the Evolution screen and the two job types in the
  automation editor.
- delete packages/optimizer.

Also logs UF-027 (self-learning loops read data nothing collects) and
UF-028 (the Settings prompt-shape choice is ignored by chat).
2026-10-20 00:54:58 +02:00
shumeiko.it
b734c1d3d2 refactor: remove KVARK, enterprise packs and the compliance subsystem (P1.4, P1.6)
KVARK is the commercial enterprise product and is not part of the community
edition. The agent's KVARK tools were never wired into the server, so
removing them changes no runtime behaviour beyond the routes listed here.

- agent: delete kvark-tools; CombinedRetrieval and formatCombinedResult now
  merge workspace and personal memory only (no KVARK lane, no source
  conflict note); connector substrate is always 'waggle'
- agent: promote_skill stops at workspace scope; team and enterprise scopes
  answer "not available in this edition" instead of an upgrade prompt
- server: delete src/kvark and GET /api/marketplace/enterprise-packs
- marketplace: delete ENTERPRISE_PACKS
- compliance (EU AI Act dashboard, its templates and the KVARK template
  seeded on every boot): delete core/src/compliance, the /api/compliance/*
  routes, agent compliance-pdf and the Cockpit compliance panel. The
  ai_interactions table lives in the memory substrate and goes with P1.5.
2026-10-20 00:27:16 +02:00
shumeiko.it
cd6553061b feat(memory): add the memory contract and the local mem0 store (P0.3)
@waggle/memory-contract defines MemoryStore: remember (text, or a
conversation to extract facts from), search, list, count, get, update,
forget, forgetMatching and history over personal and workspace scopes,
with kinds, importance, source, origin and session in place of the old
content prefixes and named sessions. It ships an in-memory store and a
compatibility suite that every implementation runs.

@waggle/memory-mem0 implements the contract on mem0ai 3.3.1 in-process:
SQLite files in one directory, scope as mem0's agent_id, Waggle's own
embedder through mem0's langchain provider, fact extraction through any
OpenAI-compatible endpoint. mem0's telemetry and remote notice download
are switched off before the module loads; a test asserts that no request
is made. Memory survives a restart (tested).

New dependencies: mem0ai 3.3.1 (Apache-2.0, pinned exactly) with its
required peers compromise (MIT) and @cloudflare/workers-types (MIT OR
Apache-2.0); transitive openai (Apache-2.0), axios (MIT) and others under
MIT or BSD-2-Clause.
2026-10-18 14:28:13 +02:00
shumeiko.it
d2b6506f8b build: declare workspace dependencies and check them in CI (P0.2)
Packages relied on npm hoisting for imports they never declared, so a
package installed on its own fails (UF-024).

- server: + @waggle/marketplace, @waggle/sdk, @waggle/weaver, zod (MIT),
  @anthropic-ai/sdk (MIT), @ax-llm/ax (Apache-2.0);
  - @waggle/hive-mind-shim-core and stripe (unused)
- agent: + pdf-parse (Apache-2.0); playwright-core (Apache-2.0) as an
  optional peer, since it is loaded only when available
- worker: + @waggle/waggle-dance; web: + @waggle/shared
- launcher: + @waggle/server as a dev dependency (bundled by tsup);
  - stripe; cli: - @waggle/weaver (unused); root: - stripe

scripts/check-workspace-deps.mjs (npm run check:deps) compares the
imports in each community-edition package's src/ with its manifest and
runs in CI. Two imports wait for P1.4 (optimizer, evolution wiring).
2026-10-15 19:31:50 +02:00
shumeiko.it
fe2eb0f648 refactor: remove the billing tier model (P0.1)
The community edition has no monetization (ADR-0001). Remove the tier
model (packages/shared/src/tiers.ts), the requireTier middleware and every
tier gate, GET/PATCH /api/tier and the trial start, and the Stripe
integration, which depends on the tier model.

Personal features a tier used to limit are open to everyone: unlimited
workspaces and connectors, every embedding provider including LiteLLM,
per-workspace costs, Approvals. The workspace session cap is one constant
(100). Team and enterprise surfaces are hidden instead of unlocked (team
dock zone, Settings Team/Enterprise/Plan tabs, workspace Team tab);
cloud sync reports unavailable, as it did for free users.

The sidecar ignores the tier keys in config.json but does not delete them:
both editions may still share ~/.waggle.

Web billing UI is deleted: plan cards, upgrade and trial modals, payment
success route, the KVARK sales block, tier 403 handling.
2026-10-15 19:31:50 +02:00
shumeiko.it
3e1a188c03 fix(server): stream Ollama model pulls so long downloads finish
A non-streaming /api/pull holds the response headers until the download
ends, and Node fetch gives up on headers after five minutes, cancelling
every larger model pull. Request stream:true and read the NDJSON progress,
failing on an in-stream error or a missing success status (UF-008).
2026-10-15 18:13:34 +02:00
Oleg Maslov
b20b138fe4 moving 2026-09-02 10:14:22 +02:00
Oleg Maslov
0c3e2ead3b moving
Some checks failed
Installer Smoke / installer-smoke (push) Has been cancelled
2026-09-02 10:10:29 +02:00