Art.17 Frame + Index + KG Erasure Companion (2026-07-01)

Continues the #7 verbatim-provenance arc. The S2 handoff (06c8574b) shipped redaction-only erasure of the raw_archive provenance rows. That is provenance-only: the DERIVED memory_frames (whose summaries quote source PII) plus their FTS / vector / chunk-vector / KnowledgeGraph projections stay searchable and recall-able. This slice closes that gap.

Erasure surface (verified 2026-07-01)

PRAGMA foreign_keys = ON (db.ts:50) → ON DELETE CASCADE fires.

Store	Holds PII?	Cascaded by frame DELETE?	Handled by `FrameStore.delete()` today
`memory_frames.content`	yes (summary quotes source)	n/a (the row itself)	DELETE ✔
`memory_frames_fts` (fts5)	yes (indexed content)	no (virtual)	manual DELETE ✔
`memory_frames_vec` (vec0)	yes (embedding)	no (virtual)	manual DELETE ✔
`memory_frame_chunks`	yes (chunk text)	yes (FK CASCADE)	via cascade ✔
`memory_frame_chunks_vec` (vec0)	yes (chunk embedding)	no (virtual)	✗ LEAK
`kg_entity_frames`	link only	yes (FK CASCADE)	manual DELETE ✔
`knowledge_entities`	yes (`name`/`properties`)	no (shared)	not touched — needs orphan sweep
`knowledge_relations`	yes (`properties`)	no	not touched — needs orphan sweep
`raw_archive`	yes (verbatim)	no (append-only)	redacted by `RawArchive.erase` ✔

Leak #1 — memory_frame_chunks_vec is a vec0 virtual table (no FK); its rowid is the chunk id (memory_frame_chunks.id). Frame DELETE cascades the chunk rows away but leaves the chunk EMBEDDINGS keyed by orphaned ids, and search() queries memory_frame_chunks_vec first. → still recall-able.

Leak #2 — an entity derived solely from an erased frame becomes an orphan (zero remaining kg_entity_frames links) but its name/properties PII persists and is still returned by searchEntities / contextual recall.

FK gotcha — knowledge_relations references knowledge_entities WITHOUT ON DELETE CASCADE; hard-deleting an entity with live relations raises SQLITE_CONSTRAINT. Relations must go first.

Decisions

Derived frames → DELETE (not redact). The raw_archive skeleton already is the founder-ratified audit record ("item existed, erased at T for reason R"). A redacted frame in the corpus would need its own PII-stripping and would pollute recall. DELETE is the simpler, more compliant choice.
Orphaned entities → hard-delete (relations first, then entity). Retire (bitemporal valid_to) hides from active recall but leaves name/properties PII physically present → not Art.17-compliant. Only delete entities with ZERO surviving frame links (never shared entities).
Fix the chunks_vec leak in FrameStore.delete() — it is a latent correctness bug. delete() now purges memory_frame_chunks_vec, and compact()'s three prune/merge sites are routed THROUGH delete() so every deletion path is covered (the adversarial review confirmed compact() bypassed delete() and still leaked _vec/_fts/_chunks_vec). Strictly additive (removes stale vectors that should be gone); cannot degrade recall of surviving frames. SOTA-neutral.
All-or-nothing — every multi-table erasure runs in one db.transaction(). A partial erasure is a compliance failure.
archive_uid re-identification residual → DEFERRED. archive_uid = sha256(source∥sourceRef∥content) stays frozen post-erasure → re-id vector for low-entropy content. Rotating it breaks the frame→archive link; distinct arc.

API (new module `mind/erasure.ts`)

interface EraseResult {
  framesDeleted: number;
  archiveRedacted: number;      // raw_archive rows redacted
  chunkVectorsPurged: number;   // memory_frame_chunks_vec rows removed
  entitiesErased: number;       // orphaned KG entities hard-deleted
  relationsErased: number;      // relations of those entities removed
}

class MindErasure {
  constructor(db, frameStore, rawArchive, knowledgeGraph)
  eraseFrame(frameId, reason): EraseResult            // one frame + provenance + orphans
  eraseBySourceRef(source, sourceRef, reason): EraseResult  // subject sweep
}

eraseFrame (single-frame primitive): collect linked entity ids → redact linked archive rows (RawArchive.eraseByFrame) → purge chunk vectors → FrameStore.delete (fts/vec/chunks/kg-bridge cascade) → for each previously-linked entity now at zero links, hard-delete its relations then the entity. One transaction.

eraseBySourceRef (subject sweep): reaches the subject's derived corpus through THREE keys, because one harvested item fans out into differently-keyed frames:

(a) the distilled summary frame — metadata.archiveUids reverse lookup;
(b) the verbatim [mind-rawturn …] frames — content-prefix keyed by sanitize(source∥sourceRef) (they carry NO archive link — the review's CRITICAL);
(c) synthesized B-frames — swept by content.references ∩ erased-ids (fixpoint). Then redact any subject provenance row no frame reached. One transaction.

Adversarial review (2026-07-01, 4-lens, each finding verified)

Raised 14, confirmed 4 — all fixed in this arc:

CRITICAL verbatim raw-turn frames survived (link-only sweep) → (b) above.
HIGH harvest KG entities were born unlinked (createEntity with no linkEntityToFrame) so the orphan sweep couldn't reach them → centralized into KnowledgeGraph.importEntitiesForFrame (creates + links); both MCP harvest handlers routed through it. Existing unlinked entities are covered by the boot-time backfillKgEntityFrames string-match.
MEDIUM compact() bypassed delete() → routed through it (decision #3).
LOW B-frames quoting entity-name PII → (c) above.

Out of scope (documented residuals)

archive_uid opaque-id rotation (decision 5) — re-id vector for low-entropy content survives; interacts with frame→archive link stability. Distinct arc.
eraseFrame is single-frame — it does NOT sweep raw-turns/B-frames; full data-subject erasure must go through eraseBySourceRef. Documented on the method.
Server route / MCP tool / Memory-Center "erase" button (substrate first).
OSS forward-port of this delta → marolinik/hive-mind (after it lands + PR #21 merges).

6.3 KiB Raw Permalink Blame History