242 lines
13 KiB
Markdown
242 lines
13 KiB
Markdown
# Memory Sync Audit — waggle-os ↔ hive-mind Divergence
|
|
|
|
**Date:** 2026-04-26
|
|
**Author:** PM
|
|
**Status:** Audit complete; 3-step sync repair plan ratified by Marko
|
|
**Decision:** Sync repair authorized (Step 1 → Step 2 → Step 3) via parallel CC-2 sesija
|
|
|
|
---
|
|
|
|
## §1 — Trigger
|
|
|
|
Marko 2026-04-26 zatražio sync verifikaciju nakon agent fix sprint kickoff-a:
|
|
> "memorija u waggle os i u hive memo mora biti uvek sinhronizovana, trebalo bi da bude da je to reseno kroz CI CD ali... proveri sve... Mozda moze cak i paralelno u paralelnoj cc sesiji"
|
|
|
|
PM verifikacija: **CI/CD sync mehanizam ne postoji** ni u jednoj formi. EXTRACTION.md je dao static map koji file mapuje gde, ali sync workflow nikada nije implementiran. Bidirektivni gap-ovi između repoa su otkriveni file-by-file (file size diff) i potvrđeni git log analizom.
|
|
|
|
---
|
|
|
|
## §2 — Realna istorija (git log evidence)
|
|
|
|
**Hive-mind je extraction artifact + par post-extraction fixes:**
|
|
|
|
Wave 1A → Wave 6 chronological extraction iz waggle-os (15-19. april 2026), zatim:
|
|
- Wave 1A 87562eb: schema.ts (scrubbed OSS subset)
|
|
- Wave 1B caae523: db.ts (MindDB standalone) + smoke tests
|
|
- Wave 2A cd61dca: embedding providers (scrubbed)
|
|
- Wave 2B aae8bbd: FrameStore + SessionStore (verbatim)
|
|
- Wave 2C a4d82df: HybridSearch + scoring layer
|
|
- Wave 2D 4d86015: KnowledgeGraph + Ontology + ConceptTracker
|
|
- Wave 2E 29afce3: IdentityLayer + AwarenessLayer
|
|
- Wave 2F 24a41bb: reconcile.ts (mind substrate complete)
|
|
- Wave 3A → 3C: harvest foundation + 9 adapters + claude-code adapter
|
|
- Wave 4 9f774f7: @hive-mind/wiki-compiler
|
|
- Wave 5A 74f2b76: WorkspaceManager + MultiMindCache
|
|
- Wave 5B a30d04a: @hive-mind/mcp-server
|
|
- Wave 6 6c32987: @hive-mind/cli + harvest dedup fix (H-34)
|
|
- Release prep d85f290: v0.1.0 ship-prep
|
|
- CI fixes b66c151 / 2db2f5f / 9d681d4: package-lock, lint script removal, typecheck removal
|
|
- 471a840: Node 22→24 + cross-platform matrix + first-run smoke
|
|
- Persona facing: 6c0752c, f04434d (CLI commands)
|
|
- 9ec75e6: **Stage 0 root cause fix** (timestamp persist)
|
|
- 0bbdf7a: **Stage 0 Task 0.5 fix** (content preview cap 2000→10000)
|
|
- 742ed75: awareness expiry ISO-8601 fix
|
|
- c363257: ClaudeAdapter 2026-04-22 export streams coverage
|
|
|
|
**Waggle-os je live development:**
|
|
|
|
Most recent mind/ commits (newest top):
|
|
- a748f8f: C5 atomic PATCH /api/memory/frames/:id/access (server feature)
|
|
- 3d43a26: awareness expiry ISO-8601 fix (manual cherry-pick from hive-mind)
|
|
- fc5d728: PA v5 WAGGLE_EVAL_MODE tier bypass (Waggle-only)
|
|
- 8666883: e2e fix
|
|
- 61651e2: AI Act compliance ship-blockers (Waggle-only per EXTRACTION.md)
|
|
- b8ffe8e: Day-1-PM correctness cluster
|
|
- 21621c6: Skills 2.0 scope + promotion foundations (agent feature)
|
|
- 2f1f067: evolution run store (NOT extracted per EXTRACTION.md)
|
|
- 65faecb: execution trace recorder (NOT extracted per EXTRACTION.md)
|
|
- db1c257: shared team memory (Teams tier, Waggle-only)
|
|
- 63ef881: findDuplicate JS trim fix
|
|
- dd9eb9a: harvest parent session creation fix
|
|
- 803c6f6: harvest_import dedup uses frame id (memory-mcp)
|
|
|
|
---
|
|
|
|
## §3 — Real production bugs (Marko Y/N input needed)
|
|
|
|
### 3.1 — Stage 0 root cause: timestamp not persisted ❌
|
|
|
|
Hive-mind ima fix (`9ec75e6`):
|
|
```
|
|
fix(harvest-local): persist item.timestamp to memory_frames.created_at (Stage 0 root cause)
|
|
```
|
|
|
|
Waggle-os **NEMA** taj fix. Production Tauri desktop koji koristi `packages/core/src/mind/frames.ts` + harvest pipeline ima isti bug. To znači:
|
|
|
|
- Harvest-ovani items nemaju ispravan `created_at` field u `memory_frames` table
|
|
- Temporal queries protiv harvest-ovanih frames vraćaju netačne timestamps
|
|
- Bitemporal KG queries (event-time vs state-time) potencijalno daju netačne rezultate
|
|
- Stage 0 LoCoMo benchmark koji je radio sa waggle-os mind/ je radio sa ovim bug-om
|
|
|
|
Severity: **production bug, ne demonstration bug**. Fix je trivijalan (1 line change verovatno) ali nije back-portovan.
|
|
|
|
### 3.2 — Content preview cap too low ⚠
|
|
|
|
Hive-mind ima fix (`0bbdf7a`):
|
|
```
|
|
fix(harvest-local): raise content preview cap 2000 → 10000 chars (Stage 0 Task 0.5)
|
|
```
|
|
|
|
Waggle-os **NEMA** taj fix. Harvest content preview je truncated na 2000 chars u waggle-os-u. Manje kritično ali utiče na harvest UX.
|
|
|
|
### 3.3 — Manualno back-portovan fix (✓ partial credit)
|
|
|
|
Awareness expiry ISO-8601 fix postoji u oba repoa sa istim commit message-om ali različitim SHA:
|
|
- waggle-os 3d43a26
|
|
- hive-mind 742ed75
|
|
|
|
Manualno cherry-picked. Što potvrđuje da NIJE bilo automated sync — neko je rukom kopirao. Mehanizam fragile.
|
|
|
|
### 3.4 — Waggle-os fixes koji možda treba u hive-mind (PM rec audit)
|
|
|
|
Dva commit-a u waggle-os koja touch-uju "shared substrate" patterns:
|
|
- `63ef881`: "fix(frames): findDuplicate must use JS trim, not SQLite trim" — POTENCIJALNO bug i u hive-mind frames.ts (hive-mind frames.ts je 16969 bytes, ima više content-a, treba check)
|
|
- `803c6f6`: "fix(memory-mcp): harvest_import / ingest_source dedup detection uses frame id, not timestamp" — touch-uje memory-mcp koji JESTE extracted u hive-mind kao mcp-server package
|
|
|
|
Plus possibly:
|
|
- `b8ffe8e`: "fix(agent,core): Day-1-PM correctness cluster from orchestrator review" — touches agent + core, treba diff audit šta tačno u core/mind/
|
|
|
|
---
|
|
|
|
## §4 — Test coverage gap
|
|
|
|
`hive-mind/packages/core/src/mind/` ima test coverage:
|
|
- awareness.test.ts
|
|
- concept-tracker.test.ts
|
|
- db.test.ts
|
|
- embedding-provider.test.ts
|
|
- entity-normalizer.test.ts
|
|
- frames.test.ts (7383 bytes)
|
|
- identity.test.ts
|
|
- inprocess-embedder.test.ts
|
|
- knowledge.test.ts
|
|
- litellm-embedder.test.ts
|
|
- ontology.test.ts
|
|
- reconcile.test.ts
|
|
- scoring.test.ts
|
|
- search.test.ts
|
|
- sessions.test.ts
|
|
|
|
`waggle-os/packages/core/src/mind/` **NEMA** test files (verified via Get-ChildItem listing — only .ts source files, no .test.ts files in mind/ folder; tests možda postoje na drugoj lokaciji).
|
|
|
|
**Implikacije:**
|
|
|
|
1. Production code (Tauri desktop konzumira waggle-os mind/) nema test coverage tu gde memorija stvarno radi
|
|
2. Tests su rađene u hive-mind što je extraction artifact, ne live production code
|
|
3. Pilot 2026-04-26, Stage 3 v6 LoCoMo, sva production validation = waggle-os mind/ runtime BEZ test coverage
|
|
4. "7 dana testova" Marko-vog priznanja = realan rad ali primenjen na pogrešnu code path
|
|
|
|
**Tests treba portovati u waggle-os.** Gde tests pass = APIs match hive-mind, dobri smo. Gde tests fail = ili waggle-os ima bug koji hive-mind nema (fix), ili waggle-os ima Waggle-specific extension koja je legitno drugačija (dokumentuj).
|
|
|
|
---
|
|
|
|
## §5 — Zašto file size diff nije katastrofa per se
|
|
|
|
`schema.ts` waggle-os 11150 bytes vs hive-mind 5882 bytes — **OČEKIVANO**. Wave 1A commit message: "extract schema.ts (scrubbed OSS subset)". Filtering tokom ekstrakcije je uklonio Waggle-specific schema (compliance, tier, telemetry tabele). Razlika 90% u size-u je posledica filtering-a, ne bug-ova.
|
|
|
|
`embedding-provider.ts` waggle-os 16694 vs hive-mind 11047 — **OČEKIVANO**. Wave 2A commit: "extract embedding providers (scrubbed)". Waggle-specific config + telemetry uklonjen.
|
|
|
|
`frames.ts` hive-mind 16969 vs waggle-os 14825 — **NIJE očekivano**. Wave 2B: "extract FrameStore + SessionStore (verbatim)" — verbatim ekstrakcija znači size-ovi su trebali da ostanu blizu. Razlika od 14% u korist hive-mind-a sugerišje da hive-mind ima post-extraction additions ili bug fixes koji nisu vraćeni u waggle-os. Treba diff audit.
|
|
|
|
`knowledge.ts` hive-mind 11393 vs waggle-os 9531 — verovatno isti razlog kao frames.ts. Verbatim ekstrakcija + post-extraction fixes only u hive-mind.
|
|
|
|
---
|
|
|
|
## §6 — 3-step sync repair plan (RATIFIED 2026-04-26)
|
|
|
|
### Step 1 — Back-port 2 hive-mind bug fixes u waggle-os ✅ ratified
|
|
|
|
Cherry-pick u waggle-os mind/ + harvest/:
|
|
1. `9ec75e6` — Stage 0 root cause (timestamp persist)
|
|
2. `0bbdf7a` — Content preview cap 2000→10000
|
|
|
|
Plus audit: da li findDuplicate JS trim fix (waggle-os 63ef881) treba u hive-mind, da li harvest_import frame-id dedup (waggle-os 803c6f6) treba u hive-mind. Bidirektioni audit.
|
|
|
|
Effort: 1-3h CC-2 work
|
|
Acceptance: tests pass on both repos posle back-port; commit dual-PR or coordinated commits
|
|
|
|
### Step 2 — Port hive-mind tests u waggle-os ✅ ratified
|
|
|
|
Kopiraj svih `*.test.ts` iz `hive-mind/packages/core/src/mind/` u waggle-os equivalent location (treba odlučiti convention: `__tests__/` adjacent ili separate `tests/mind/` folder per existing waggle-os pattern).
|
|
|
|
Run vitest. Klasifikuj outcomes:
|
|
- PASS: API match, test je validno za waggle-os
|
|
- FAIL — bug u waggle-os: cherry-pick fix iz hive-mind ako postoji, ili identifikuj kao standalone bug
|
|
- FAIL — API mismatch: waggle-os ima Waggle-specific extension; dokumentuj test ne primenjuje se ili treba prilagoditi
|
|
- FAIL — test depends on hive-mind-only utility: skip ili adapter
|
|
|
|
Effort: 4-8h CC-2 work
|
|
Acceptance: dokumentovano per-test outcome; svaki FAIL ima ratifikaciju da li je bug, extension, ili adapter need
|
|
|
|
### Step 3 — CI/CD sync workflow ✅ ratified
|
|
|
|
Dva GitHub Actions deliverable:
|
|
|
|
**3.1 — `mind-parity-check` u waggle-os/.github/workflows/ci.yml:**
|
|
- Trigger: PR + push na main koji touch-uje `packages/core/src/mind/` ili `packages/core/src/harvest/`
|
|
- Job: clone hive-mind master, copy hive-mind tests u waggle-os checkout, run vitest na waggle-os
|
|
- Pass criteria: hive-mind tests pass na waggle-os codebase (or explicit allowlist of failing tests dokumented kao Waggle-specific extensions)
|
|
- Fail = block merge
|
|
|
|
**3.2 — `mind-sync-pr` u waggle-os/.github/workflows/sync-mind.yml (new file):**
|
|
- Trigger: push na main u waggle-os koji touch-uje `packages/core/src/mind/` ili `packages/core/src/harvest/`
|
|
- Job: filter diff exluding Waggle-specific files (per EXTRACTION.md "NOT extracted" list — vault.ts, evolution-runs.ts, execution-traces.ts, improvement-signals.ts, compliance/), open PR u marolinik/hive-mind sa diff applied to hive-mind/packages/core/src/mind/
|
|
- PR review: manual approval u hive-mind (GitHub PR review process)
|
|
- Auto-tag PR sa "auto-sync from waggle-os@{sha}"
|
|
|
|
Effort: 3-5h CC-2 work
|
|
Acceptance: oba workflows running u CI; test sync end-to-end (push test commit u waggle-os mind/ on test branch, verify auto-PR opens u hive-mind)
|
|
|
|
### Sequencing
|
|
|
|
Step 1 ide odmah (parallel sa agent fix Phase 1.x koja ne touch-uje mind/). Step 2 sledi posle Step 1 verifikacije. Step 3 sledi posle Step 2.
|
|
|
|
CC-2 sesija paralelna sa CC-1 agent fix sprintom — independent code paths, no merge conflict expected.
|
|
|
|
---
|
|
|
|
## §7 — Šta NE radimo (rejected alternatives)
|
|
|
|
- **Radikalan refactor** (waggle-os consumes @hive-mind/core via npm dep, Opcija B from prior PM proposal): rejected. Ima legitimne razloge da postoje obe verzije (Waggle-specific schema + compliance + telemetry u waggle-os/mind/, scrubbed OSS subset u hive-mind/mind/). Refactor bi bio 2-4 nedelje rada za marginal benefit. Sync workflow daje 90% benefita za 10% effort-a.
|
|
|
|
- **Git submodule** (Opcija C from prior PM proposal): rejected. Sprečava clean Apache-2.0 npm publish; submodule nameni se za internal dependencies, ne za public release artifacts.
|
|
|
|
- **Auto-merge bidirectional** (no manual review): rejected. Waggle-specific code može slučajno da uđe u hive-mind public release; manual PR review u hive-mind je essential safety check.
|
|
|
|
---
|
|
|
|
## §8 — Implications za 14-step launch plan (post-sync repair)
|
|
|
|
- **Korak 1.5 (Memory sync repair)** je sad konkretizovan: 3-step plan, ratified, paralelni CC-2 sesija u progress
|
|
- **Korak 2 (retrieval V2)** će se raditi u oba repoa simultaneously preko sync workflow-a (Step 3 enabled)
|
|
- **Korak 8 (open source repos public)** — hive-mind v0.1.0 release artifact ostaje validan, ali Step 3 sync workflow garantuje da budući commits konsistentno propagiraju
|
|
- **Korak 12 (substrate integrity audit)** mora da uključi sync verification: test parity check + git divergence check pre arxiv submission
|
|
|
|
---
|
|
|
|
## §9 — Audit trail
|
|
|
|
- Git log evidence: `D:\Projects\PM-Waggle-OS\tmp_git_audit.txt` (temporary, biće obrisan po završetku audit-a)
|
|
- File size comparison: prikazana u prethodnom PM message-u (svih 19 mind/ files DIFF, 3 waggle-only files, 0 hive-mind-only files)
|
|
- File comparison metodologija: SHA256 hash + size + LastWriteTime, no content read (preserved confidentiality)
|
|
- EXTRACTION.md: `D:\Projects\hive-mind\EXTRACTION.md` (16-line static map, no sync mechanism)
|
|
|
|
---
|
|
|
|
## §10 — Cross-references
|
|
|
|
- `D:\Projects\PM-Waggle-OS\decisions\2026-04-26-pilot-verdict-FAIL.md` (pilot used waggle-os in-tree mind/)
|
|
- `D:\Projects\PM-Waggle-OS\research\2026-04-26-arxiv-paper\01-paper-skeleton.md` (substrate claim — 74% oracle ceiling — must be pinned to specific waggle-os HEAD SHA + sync verified pre arxiv submission)
|
|
- `D:\Projects\PM-Waggle-OS\briefs\2026-04-26-memory-sync-repair-cc2-brief.md` (CC-2 paste-ready brief, authored next)
|
|
- `D:\Projects\hive-mind\BACKLOG.md` (P1 entry: ClaudeAdapter coverage — sync timing TBD)
|