# Master Data Model (two ER diagrams) Waggle OS keeps data in two physically separate stores. The **per-workspace memory layer** is a single SQLite file (`*.mind`) per workspace — 14 base tables plus 2 virtual search tables — holding one user's private frames, knowledge graph, identity, awareness, and audit. The **team/cloud relational layer** is a shared PostgreSQL database (20 tables, Drizzle ORM) holding users, teams, agents, tasks, jobs, and governance. The two stores share **no cross-database foreign keys**; the only bridge is the `users.mind_path` text column in Postgres, which points at where a given user's local SQLite `*.mind` file lives. The note after the diagrams explains how one workspace mind relates to a user/team row. --- ## Diagram 1 — Per-workspace memory layer (SQLite, one `*.mind` file) One isolated SQLite file per workspace. Schema version `'1'`. 14 base tables plus `memory_frames_fts` (FTS5 keyword index) and `memory_frames_vec` (vec0, 1024-dim embeddings), both keyed by `rowid = memory_frames.id`. Foreign keys exist only inside this file (frames to sessions, frames self-ref, relations to entities). `ai_interactions` is append-only (DB triggers reject UPDATE/DELETE). The knowledge graph is bitemporal: active rows have `valid_to IS NULL`. ```mermaid erDiagram sessions ||--o{ memory_frames : "gop_id FK" memory_frames ||--o{ memory_frames : "base_frame_id self-FK" memory_frames ||--|| memory_frames_fts : "rowid=id FTS5" memory_frames ||--|| memory_frames_vec : "rowid=id vec0" knowledge_entities ||--o{ knowledge_relations : "source_id FK" knowledge_entities ||--o{ knowledge_relations : "target_id FK" meta { TEXT key PK TEXT value } identity { INTEGER id PK "CHECK id=1, single row" TEXT name TEXT role TEXT department TEXT personality TEXT capabilities TEXT system_prompt TEXT created_at TEXT updated_at } awareness { INTEGER id PK TEXT category "task action pending flag" TEXT content INTEGER priority TEXT metadata "JSON" TEXT created_at TEXT expires_at "nullable" } sessions { INTEGER id PK TEXT gop_id UK "join key for frames" TEXT project_id "nullable" TEXT status "active closed archived" TEXT started_at TEXT ended_at "nullable" TEXT summary "nullable" } memory_frames { INTEGER id PK TEXT frame_type "I P B" TEXT gop_id FK INTEGER t "per-gop ordinal" INTEGER base_frame_id FK "nullable self" TEXT content "JSON for B-frames" TEXT importance "critical..deprecated" TEXT source "user_stated..system" INTEGER access_count TEXT created_at TEXT last_accessed } memory_frames_fts { TEXT content "FTS5 rowid=id" } memory_frames_vec { FLOAT embedding "float 1024 rowid=id" } knowledge_entities { INTEGER id PK TEXT entity_type TEXT name TEXT properties "JSON" TEXT valid_from TEXT valid_to "nullable=active" TEXT recorded_at } knowledge_relations { INTEGER id PK INTEGER source_id FK INTEGER target_id FK TEXT relation_type REAL confidence TEXT properties "JSON" TEXT valid_from TEXT valid_to "nullable=active" TEXT recorded_at } procedures { INTEGER id PK TEXT name TEXT model TEXT template INTEGER version REAL success_rate REAL avg_cost TEXT created_at TEXT updated_at } improvement_signals { INTEGER id PK TEXT category "capability_gap..skill_promotion" TEXT pattern_key TEXT detail INTEGER count TEXT first_seen TEXT last_seen INTEGER surfaced "0 or 1" TEXT surfaced_at "nullable" TEXT metadata "JSON" } install_audit { INTEGER id PK TEXT timestamp TEXT capability_name TEXT capability_type "native..marketplace" TEXT source TEXT version "nullable" TEXT risk_level "low medium high" TEXT trust_source TEXT approval_class TEXT action TEXT initiator "agent user system" TEXT detail } ai_interactions { INTEGER id PK "APPEND-ONLY" TEXT timestamp TEXT workspace_id "nullable" TEXT session_id "nullable" TEXT model TEXT provider INTEGER input_tokens INTEGER output_tokens REAL cost_usd TEXT tools_called "JSON" TEXT human_action "nullable" TEXT risk_context "nullable" TEXT imported_from "nullable" TEXT persona "nullable" TEXT input_text "nullable" TEXT output_text "nullable" } execution_traces { INTEGER id PK TEXT session_id "nullable" TEXT persona_id "nullable" TEXT workspace_id "nullable" TEXT model "nullable" TEXT task_shape "nullable" TEXT outcome "success..pending" TEXT trace_json "JSON" REAL cost_usd INTEGER duration_ms TEXT created_at TEXT finalized_at "nullable" } evolution_runs { INTEGER id PK TEXT run_uuid UK TEXT target_kind TEXT target_name "nullable" TEXT baseline_text TEXT winner_text TEXT winner_schema_json "nullable" REAL delta_accuracy TEXT gate_verdict "pass fail" TEXT gate_reasons_json "JSON" TEXT status "proposed..failed" TEXT artifacts_json "nullable" TEXT user_note "nullable" TEXT failure_reason "nullable" TEXT created_at TEXT decided_at "nullable" TEXT deployed_at "nullable" } harvest_sources { INTEGER id PK TEXT source UK TEXT display_name TEXT source_path "nullable" TEXT last_synced_at "nullable" INTEGER items_imported INTEGER frames_created INTEGER auto_sync "0 or 1" INTEGER sync_interval_hours TEXT last_content_hash "nullable" TEXT created_at } ``` > Standalone tables (no DB-level FKs to others): `meta`, `identity`, `awareness`, `procedures`, `improvement_signals`, `install_audit`, `ai_interactions`, `execution_traces`, `evolution_runs`, `harvest_sources`. Their `session_id` / `workspace_id` columns are plain TEXT, not enforced foreign keys. A `kg_entity_frames` link table is referenced by `frames.delete()` but is not defined in `schema.ts`, so it may be absent — omitted here. --- ## Diagram 2 — Team/cloud relational layer (PostgreSQL + Drizzle) Shared multi-user store. All primary keys are server-generated UUIDs (`gen_random_uuid()`); all timestamps are `timestamp with time zone`. Every foreign key is `ON DELETE no action` — there are **no cascades**, so deleting a parent is blocked while children reference it. Two junction tables use composite PKs (`team_members`, `agent_group_members`). `users.mind_path` is the only pointer out to the SQLite memory layer. ```mermaid erDiagram users ||--o{ teams : "owns owner_id" users ||--o{ team_members : "is" teams ||--o{ team_members : "has" users ||--o{ agents : "owns user_id" teams ||--o{ agents : "scopes team_id" users ||--o{ agent_groups : "owns" agent_groups ||--o{ agent_group_members : "contains" agents ||--o{ agent_group_members : "joins" teams ||--o{ tasks : "has" users ||--o{ tasks : "creates created_by" users ||--o{ tasks : "assigned assigned_to" teams ||--o{ messages : "channel" users ||--o{ messages : "sends sender_id" teams ||--o{ team_entities : "owns" users ||--o{ team_entities : "shares shared_by" teams ||--o{ team_relations : "owns" team_entities ||--o{ team_relations : "source_id" team_entities ||--o{ team_relations : "target_id" teams ||--o{ team_resources : "owns" users ||--o{ team_resources : "shares" teams ||--o{ team_capability_policies : "governs" users ||--o{ team_capability_policies : "updates" teams ||--o{ team_capability_overrides : "governs" users ||--o{ team_capability_overrides : "decides" teams ||--o{ team_capability_requests : "scopes" users ||--o{ team_capability_requests : "requests-decides" teams ||--o{ agent_jobs : "owns" users ||--o{ agent_jobs : "runs" teams ||--o{ cron_schedules : "owns" users ||--o{ cron_schedules : "creates" users ||--o{ scout_findings : "for-user" teams ||--o{ scout_findings : "for-team" proactive_patterns ||--o{ suggestions_log : "fires" users ||--o{ suggestions_log : "receives" users ||--o{ agent_audit_log : "acts" teams ||--o{ agent_audit_log : "scopes" users { uuid id PK text clerk_id UK text display_name text email UK text avatar_url "nullable" text mind_path "nullable to SQLite mind" timestamptz created_at timestamptz updated_at } teams { uuid id PK text name text slug UK uuid owner_id FK timestamptz created_at } team_members { uuid team_id PK_FK uuid user_id PK_FK text role "default member" text role_description "nullable" jsonb interests "nullable" timestamptz joined_at } agents { uuid id PK uuid user_id FK uuid team_id FK "nullable" text name text role "nullable" text system_prompt "nullable" text model "default claude-haiku-4-5" jsonb tools "default empty" jsonb config "default empty" timestamptz created_at } agent_groups { uuid id PK uuid user_id FK text name text description "nullable" text strategy "default parallel" timestamptz created_at } agent_group_members { uuid group_id PK_FK uuid agent_id PK_FK text role_in_group "default worker" integer execution_order "default 0" } tasks { uuid id PK uuid team_id FK text title text description "nullable" text status "default open" text priority "default normal" uuid created_by FK uuid assigned_to FK "nullable" uuid parent_task_id "no FK logical self-ref" timestamptz created_at timestamptz updated_at } messages { uuid id PK uuid team_id FK uuid sender_id FK text type text subtype jsonb content uuid reference_id "no FK" jsonb routing "nullable" timestamptz created_at } team_entities { uuid id PK uuid team_id FK text entity_type text name jsonb properties "default empty" uuid shared_by FK timestamptz valid_from timestamptz valid_to "nullable open-ended" timestamptz created_at } team_relations { uuid id PK uuid team_id FK uuid source_id FK uuid target_id FK text relation_type real confidence "default 1.0" jsonb properties "default empty" timestamptz created_at } team_resources { uuid id PK uuid team_id FK text resource_type text name text description "nullable" jsonb config uuid shared_by FK real rating "default 0" integer use_count "default 0" timestamptz created_at } team_capability_policies { uuid id PK uuid team_id FK text role jsonb allowed_sources "default empty" jsonb blocked_tools "default empty" text approval_threshold "default none" uuid updated_by FK "nullable" timestamptz created_at timestamptz updated_at } team_capability_overrides { uuid id PK uuid team_id FK text capability_name text capability_type text decision text reason "default empty" uuid decided_by FK timestamptz created_at timestamptz decided_at } team_capability_requests { uuid id PK uuid team_id FK uuid requested_by FK text capability_name text capability_type text justification text status "default pending" uuid decided_by FK "nullable" text decision_reason "nullable" timestamptz created_at timestamptz decided_at "nullable" } agent_jobs { uuid id PK uuid team_id FK uuid user_id FK text job_type text status "default queued" jsonb input jsonb output "nullable" timestamptz started_at "nullable" timestamptz completed_at "nullable" timestamptz created_at } cron_schedules { uuid id PK uuid team_id FK uuid created_by FK text name text cron_expr text job_type jsonb job_config "default empty" boolean enabled "default true" timestamptz last_run_at "nullable" timestamptz next_run_at "nullable" timestamptz created_at } scout_findings { uuid id PK uuid user_id FK "nullable" uuid team_id FK "nullable" text source text category text title text summary "nullable" real relevance_score "default 0" text url "nullable" text status "default new" timestamptz created_at } proactive_patterns { uuid id PK "config-only no FK" text name jsonb trigger text suggestion_type text template boolean enabled "default true" } suggestions_log { uuid id PK uuid user_id FK uuid pattern_id FK jsonb context text status "default pending" timestamptz created_at } agent_audit_log { uuid id PK uuid user_id FK uuid team_id FK "nullable" text agent_name text action_type text description jsonb before_state "nullable" jsonb after_state "nullable" boolean requires_approval "default false" boolean approved "nullable tri-state" uuid approved_by FK "nullable" timestamptz created_at } ``` --- ## How a workspace mind relates to a user/team row A **workspace** is, physically, one SQLite `*.mind` file on the local machine (e.g. `personal.mind`). It is self-contained: switching workspace means opening a different file, and nothing inside that file joins across workspaces. There is no `workspace` table in Postgres — workspaces live as files, not relational rows. The single connection point between the two layers is the **`users.mind_path`** column in Postgres: a nullable `text` pointer to where that user's private SQLite mind file lives. There are **no cross-database foreign keys**; the relationship is resolved only in application code. Concretely: - A **user** row in Postgres (`users.id`, Clerk-backed via `clerk_id`) owns at most one `mind_path`. Following that path opens the user's personal `*.mind` SQLite file (Diagram 1), whose `identity` table (single CHECK-pinned row) describes that same user from the memory side. - Inside the mind file, `ai_interactions.workspace_id` and `execution_traces.workspace_id` are plain TEXT labels for the originating workspace — they are **not** foreign keys to anything in Postgres. The same is true of `session_id`: it is a logical label, not an enforced cross-DB reference. - **Team-shared knowledge** is duplicated in concept but not in storage: each user keeps a private SQLite `knowledge_entities` / `knowledge_relations` graph (Diagram 1), while the team keeps a separate Postgres `team_entities` / `team_relations` graph (Diagram 2). They are distinct tables in distinct engines; promoting a private fact to the team graph is an application-level copy, not a join. - The relational layer **never stores memory frames**. All `memory_frames`, embeddings, awareness, and harvest tracking stay local in the workspace mind file; Postgres only holds the team/collaboration/governance metadata and the `mind_path` breadcrumb back to each local file.