# OSS Drift Triage — waggle-os ↔ marolinik/hive-mind (2026-06-11)

**Trigger:** the first `scripts/oss-drift-check.sh` run (§7.5 policy ratification arc)
surfaced drift far beyond the W4.2 reranker incident: 3 OSS-only source files +
42 differing files. This doc is the file-by-file direction triage (4-agent recon,
monorepo `HEAD` vs OSS `origin/master`) and the reverse-port execution record.

**Key discovery:** the OSS repo was never a byte-identical subtree-split product —
it was a **hand-extraction (May 2026) with deliberate transforms** (extraction
headers, `[hive-mind:]` branding, import rewrites, scrubbed prompt examples), and
both sides evolved independently since. Blob-equality checks are useless;
semantic per-file triage was required. The §7.5 "monorepo sole source" policy
now governs; this triage is the one-time consolidation.

## Verdict matrix (45 files)

### PACKAGING-ONLY — no action (19)
perplexity/markdown/pdf/plaintext/url adapters, chunk-utils, prompts (scrubbed
examples), index (harvest barrel), run-store, source-store, sessions, awareness,
identity, concept-tracker, reconcile, api/inprocess/litellm/ollama embedders,
multi-mind-cache.

### MONO-AHEAD — mirror is stale, fixed by next re-split (6)
chatgpt-adapter, gemini-adapter, universal-adapter (W4.4 captions + raw-types),
dedup (harvestSetHash), types (HARVEST_FRAME_CONTENT_CAP), scoring (W4.2
created_at decay — **OSS still has the last_accessed decay bug**).
Plus all 11 ONLY-IN-MONO modules (temporal stack, raw-turns/raw-detail-lane,
extract-memory-lanes, multi-mind, proprietary-excluded evolution/traces/signals).

### REVERSE-PORT — OSS has real functionality the monorepo lacks

| # | Source | What | Size | Status |
|---|---|---|---|---|
| R1 | logger.ts | **stderr routing for info/debug** — mono logger writes to stdout; hive-mind-mcp-server is stdio-transport, so pipeline `log.info` calls can corrupt the MCP stream (LIVE latent bug). + `CoreLogger` type export. | S | PORTED |
| R2 | knowledge.ts | `findEntityByName()` exact-match dedup fix (LIKE top-K drops exact match → runaway dups, e.g. 3506 "Phase" rows) + `dedupeByName()` transactional merge + `safeParseProps()`. | M | PORTED |
| R3 | entity-normalizer.ts | Write-time noise filter: `isNoiseName()`/`isLikelyAcronym()` + STOP_TOKENS + TECH_ALLOWLIST — blocks low-signal names entering the KG. | S | PORTED |
| R4 | workspace-manager.ts | `ensure(id, options)` idempotent create-by-trusted-id — mono MCP save_memory to a not-yet-created workspace has no auto-create path. | S | PORTED |
| R5 | embedding-provider.ts | `maxEmbedCharsForModel()`/`capEmbedText()` (input capping) + `reembedPerText()` (per-text batch-failure recovery — mono degrades whole batch to mock noise). | M | PORTED |
| R6 | claude-adapter.ts | 2026-04-22 Claude export streams: `memories` (conversations_memory + project_memories) + `design_chats[]` + enriched project-doc parsing — mono silently drops two whole export streams. | M | PORTED |
| R7 | db.ts | Embedding-fingerprint guard (`ensureEmbeddingFingerprint` + `EmbeddingDimMismatchError` + `recreateVecTables`) — refuses loudly on dim mismatch; mono has zero protection against mixed-dim vector corruption. | M | PORTED |

### D-items — founder GO 2026-06-11 ("do all 3"), ALL PORTED same session

| # | What | Status |
|---|---|---|
| D1 | **Chunk-level retrieval stack** — chunker.ts + `memory_frame_chunks`(+`_vec`, dim-parameterized) schema + `indexChunksForFrame` + chunk-vec lane in HybridSearch (over-fetch ×5, best-chunk-per-frame dedup, clean fallback to whole-frame vectors) + `rechunkAllFrames` backfill. | PORTED + **DEFAULT-ON (2026-06-12)** after the long-frame needle probe (kill switch `WAGGLE_CHUNK_RETRIEVAL=0`). See probe record below. |
| D2 | **LLM KG entity extraction** (replaces the capitalized-n-gram regex as the KG quality path) | PORTED — prompt/parser/batching from OSS llm-extractor; executors rehomed onto `LLMCallFn` 'fast'. Runs in the daily memory-lane cron AFTER the lane pass, same frame window + shared watermark. Writes dedup via `findEntityByName` (R2) + filter via `isNoiseName` (R3); injection-scanned. |
| D3 | content_hash indexed dedup column | PORTED — **with MONO semantics** (sha256 over `stripHmPrefix(content).trim()`, content-hash.ts): the OSS trim-only hash would have regressed OQ-6 provenance-insensitive dedup. `findDuplicate` now O(1) indexed, NO recency window (old LIMIT-500 scan silently missed older dups). Idempotent migration + backfill. |

### Forward-port queue (mono → OSS, next re-split)
W4 arc (all of it), scoring created_at fix, since/until fencepost fixes,
likeFallbackSearch, escapeLikeTerm (knowledge), W4.4 captions + raw-types,
HARVEST_FRAME_CONTENT_CAP, harvestSetHash. Note: mono adapters depend on
`harvest/raw-types.ts` which the OSS tree lacks — re-split must carry it.

## Execution record
- 2026-06-11: triage run (4 parallel agents over monorepo HEAD vs oss origin/master).
- 2026-06-11: R1-R7 reverse-ported (see commits on main).
- 2026-06-11 (later): founder GO "do all 3" → D1+D2+D3 ported same session
  (3 implementation agents + direct work; D1 flag-gated default-OFF).
- 2026-06-12 — **D1 eval gate run → DEFAULT-ON.** Pre-registered probe
  (`benchmarks/chunk-probe/run-probe.mjs`, $0, all-local):
  - **LoCoMo REJECTED as the ruler** — its frames max ~1,000 chars (below the
    2,000-char chunk threshold): every frame yields 1 chunk ≡ whole frame, so
    a LoCoMo A/B measures noise by construction. Epistemic check before spend.
  - Honest corpus: COPY of the real personal mind (471 frames, 129 >2k chars,
    max 25.7k), both cells re-embedded from scratch (Ollama
    nomic-embed-text-8k/1024; original vectors were mock-fingerprinted).
  - Verbatim deep-position needles (50–90% frame depth), n=52, paired cells:
    whole-frame `vectorSearch` vs chunk `vectorSearchChunks`.
  - **Result: hit@5 chunk 46/52 vs whole-frame 17/52; discordant pairs 30-vs-1
    (McNemar p≈2e-8).** Within-embed-cap stratum: chunk 17/20 vs 13/20 (wins
    on fair ground); beyond-cap: 29/32 vs 4/32 (whole-frame is structurally
    blind past the embedder's token window).
  - **Side-finding (R5 hardening):** model names lie — `nomic-embed-text-8k`
    is architecture-capped at 2048 tokens (nomic-bert); the OSS 24k-char
    heuristic 400s and mock-poisons long frames. `maxEmbedCharsForModel`
    8k-branch reduced 24k→8k chars.
  - Per-needle detail: `benchmarks/chunk-probe/data/probe-result.json`
    (data dir gitignored — contains a personal-mind copy).
- 2026-06-12 — **backfill wiring SHIPPED** (`packages/server/src/local/
  vector-backfill.ts`): one-time per-mind vector repair (mock-fingerprint
  cure: recreate + re-embed) + `rechunkAllFrames`, wired at boot (personal)
  and in the daily memory_lane_extract cron (all minds); idempotent meta
  flag, skip-without-flag while the embedder is mock (daily retry).
- 2026-06-12 — **forward-port SHIPPED as
  [marolinik/hive-mind PR #14](https://github.com/marolinik/hive-mind/pull/14)**
  (branch `feature/mono-parity-2026-06-12`; merge = founder gate). Three
  port agents: core fixes (scoring created_at decay, search date-window +
  LIKE-fallback + chunk flag `HIVE_MIND_CHUNK_RETRIEVAL` + auto-index +
  rechunkAllFrames, content-hash hm-stripped semantics + one-time rehash
  migration, embed-cap 24k→8k), harvest parity (captions ×4 + raw-types +
  content cap + harvestSetHash + extract-memory-lanes + raw-turns), new
  modules (temporal stack + raw-detail-lane). OSS gates: tsc 0, full repo
  654/654 (+~90 new co-located tests).
- **Drift-check after the port (against the parity branch):** structural
  gaps CLOSED. Residual ONLY-IN-OSS = `llm-extractor.ts` (deliberately
  kept — the OSS CLI consumes it; mono's D2 rehomed it as
  extract-kg-entities). Residual ONLY-IN-MONO = proprietary exclusions
  (evolution-runs/execution-traces/improvement-signals) + mono-specific
  modules (multi-mind, extract-kg-entities). The long DIFFERS list is
  PERMANENT-BY-DESIGN cosmetics (branding, env-var names, scrubbed
  examples, extraction headers) — the check's actionable signals are the
  ONLY-IN-* buckets plus manual DIFFERS inspection after substrate arcs.