oleg.m/waggle-os
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
master
waggle-os/docs/ux-refactor/gap-cards/S07-connector-hub.md
Oleg Maslov 0c3e2ead3b
Some checks failed
Installer Smoke / installer-smoke (push) Has been cancelled
Details
moving
2026-09-02 10:10:29 +02:00

14 KiB
Raw Permalink Blame History

Gap Card — S07 · Connector Hub

Execution model: in-place incremental refactor of apps/web + targeted backend extensions. PRD acceptance criteria win over the (directional) mockup. Every claim below is grounded in real files. PRD = docs/Waggle_OS_UX_Refactor_Master_Handoff_Package/Waggle_OS_UX_Refactor_PRD.md.

1. Screen & purpose

Connector Hub (PRD §12.7, blueprint Screen 7, dock app id connectors). Purpose: "Connect external tools and data." It is the Extend layer surface for service connectors (distinct from S08 MCP Hub — PRD §12.8 — which the current ConnectorsApp folds in as a second "MCP Servers" tab via McpCatalog; the refactor should keep the catalog but the two are separate PRD screens).

PRD §12.7 acceptance criterion (the bar this card must hit):

"Users see exactly what tools are connected and whether data is flowing."

The "whether data is flowing" half is the load-bearing gap — today there is no sync state, no last-sync timestamp, and no recent-activity surface anywhere in the stack.

2. Required states (PRD/Blueprint)

PRD §12.7 functional requirements + blueprint Screen-7 state list (_blueprint_extracted.txt:326-331, state list :467):

  1. Connected connectors with status and last sync.
  2. Available connectors grouped by category.
  3. Health surface: health, recent sync activity, token expiry, and errors.
  4. Actions: connect, sync now, manage, revoke, reconnect.
  5. Recommended connectors based on onboarding + workspace needs.
  6. State machine per blueprint: connected · disconnected · expired token · syncing · failed · recommended (plus list-level loading / empty / populated / error).
  7. RBAC/trust (PRD §17.3, §18.1): "Connectors require consent and revocation path" + "Connector/MCP install audit" (append-only trust trail).

Mockup (screen_07_connector_hub.png, directional) adds visual chrome the PRD does not mandate but implies: top tabs (Connected / Available / Recommended / Built by Waggle), a Connector Summary donut (connected/syncing/error/disconnected counts), a System Health tile (API/sync/error-rate gauges), a Recent Activity feed, and a "Need Help?" card. Treat these as direction, not a contract.

3. Current state in repo (disposition: rework)

Frontend — apps/web/src/components/os/apps/ConnectorsApp.tsx (356 LOC):

  • Two-tab sidebar: Services (native connectors) + MCP Servers (McpCatalog). Filter rail = all / connected / available.
  • Renders connectors from adapter.getConnectors() grouped by a hardcoded CATEGORIES map (:41-50)
    • hardcoded SETUP_HINTS (:52-60). Per-row expand reveals capability chips, setup steps, and a token/API-key input.
  • Connect flow (handleConnect, :113-135): writes credential via adapter.addVaultSecret('connector:<id>') then calls adapter.connectConnector(id) (note: the adapter's connectConnector POSTs no body — adapter.ts:1161; the credential is pre-seeded in vault). Disconnect = adapter.disconnectConnector(id).
  • shouldResetCredentialInputs (:69-71) is an exported, regression-tested pure guard (phase5b-connectors.test) — keep, it prevents credential cross-submission.
  • Subcomponents (apps/web/src/components/os/apps/connectors/): BrandTile.tsx, brand-identity.ts (logo/colour identity — keep, reuse), McpCatalog.tsx + McpServerCard.tsx + mcp-registry.ts (the MCP tab — belongs to S08, leave for that card).
  • What's MISSING in the UI vs PRD: no last-sync, no health probe call, no "sync now", no "revoke" (only disconnect), no recommended tab, no recent-activity feed, no summary/health tiles, no token-expiry surfacing. adapter.getConnectorHealth exists (adapter.ts:1156) but is never called anywhere (grep-confirmed) — the UI shows only the coarse list status.

Backend — packages/server/src/local/routes/connectors.ts (121 LOC): 4 routes — GET /api/connectors, GET /api/connectors/:id/health, POST /api/connectors/:id/connect, POST /api/connectors/:id/disconnect. Backed by fastify.connectorRegistry (ConnectorRegistry in packages/agent/src/connectors/connector-registry.ts; 30 connectors registered via setup-connectors.ts; per-connector healthCheck() impls exist in every packages/agent/src/connectors/*-connector.ts).

Disposition: rework — the shell, brand tiles, category grouping, and connect/disconnect flow are solid and reusable, but the screen is missing the entire "is data flowing?" dimension (sync/health/ activity/recommended/audit). This is additive rework on a good base, not a rebuild.

4. Frontend work

Rework ConnectorsApp.tsx into the PRD Connector Hub (keep MCP tab in place for S08):

  • Add tabs matching PRD: Connected · Available · Recommended (mockup also shows "Built by Waggle" — optional). The existing all/connected/available filter rail collapses into these tabs.
  • Per-row health + last sync: lazy-call adapter.getConnectorHealth(id) for connected rows (already in adapter, currently dead) to render status (connected/expired/error), tokenExpiresAt (token-expiry badge), lastChecked, and (new) lastSyncAt. Extract a new ConnectorCard.tsx component (blueprint names ConnectorCard, :488) from the inline row JSX (:262-340) — status pill, last-sync, health dot, actions menu (Connect / Sync now / Manage / Revoke / Reconnect).
  • Recommended tab: reuse the existing recommendConnectors(personaId) from @waggle/shared (connector-recommendations.ts:146) — today wired only into the MCP tab (McpCatalog.tsx:69), not service connectors. Forward personaId (already a prop, :28) to a recommended-connectors section.
  • Summary + health + activity tiles (mockup, directional): a small ConnectorSummary donut from client-side status counts (no new endpoint), a System Health tile from the new GET /api/connectors/health aggregate (§5), and a Recent Activity feed from the new GET /api/connectors/activity (sync/connect/revoke events). The blueprint's reusable Timeline and EvidencePanel (:488) can host the activity list.
  • Actions wiring: add syncConnector(id) → POST /api/connectors/:id/sync; add revokeConnector(id) → POST /api/connectors/:id/revoke (PRD verb; aliases existing disconnect); "Reconnect" reuses the existing connect flow. Surface a syncing row state + toasts on failed.
  • State: extend the local Connector interface (:31-38) with optional lastSyncAt, tokenExpiresAt, health, category, recommended. Move category off the hardcoded map onto the ConnectorDefinition.category field that already exists in @waggle/shared (types.ts:299) but is not yet emitted by the row data — reconcile so the UI stops carrying its own CATEGORIES.

Adapter (apps/web/src/lib/adapter.ts) — add 2 methods, reuse 1:

  • syncConnector(id): POST /api/connectors/:id/sync (NEW).
  • revokeConnector(id): POST /api/connectors/:id/revoke (NEW; or alias to existing disconnectConnector).
  • getConnectorHealth(id) already exists (:1156) — start calling it; optionally add getConnectorsHealth(): GET /api/connectors/health aggregate + getConnectorActivity().

5. Backend work (PRD §16.9 connectors subset)

§16.9 spans Connectors + MCPs + Marketplace. This card scopes the Connectors rows only; MCP rows (/api/mcps*) and /api/marketplace* belong to S08 / S13.

PRD §16.9 endpoint Status Action
GET /api/connectors EXISTS connectors.ts:6 → registry.getDefinitions(). EXTEND the definition payload to carry category (already on the type, types.ts:299) + lastSyncAt so the UI drops its hardcoded CATEGORIES/sync-shim.
POST /api/connectors/:id/connect EXISTS connectors.ts:55. EXTEND: write an install-audit entry (see audit gap below) on success.
POST /api/connectors/:id/sync MISSING NET-NEW route in connectors.ts. No sync substrate exists — WaggleConnector (connector-sdk.ts) has connect/healthCheck/execute but no sync(). Minimum viable: probe healthCheck() + record a lastSyncAt timestamp (persist in vault sub-key connector:<id>:lastSync or a small store) + emit an activity event. Full sync (re-pull data) is a larger connector-SDK addition — flag as phased.
POST /api/connectors/:id/revoke PARTIAL Closest is POST /api/connectors/:id/disconnect (connectors.ts:107, removes connector:<id> + sub-keys). EXTEND: add a /revoke alias (or rename) that additionally writes an install-audit action:'rejected'/revoke entry. Same intent, PRD verb.

Additional backend work the PRD requires but §16.9 does not enumerate (implied by §18.1 / §12.7):

  • Install/consent audit on connect & revoke (MISSING write path). connectors.ts does NOT call fastify.auditStore.record(...) (grep-confirmed — only marketplace.ts:224-319 writes audit today). PRD §18.1 mandates "Connector/MCP install audit". The substrate exists: InstallAuditStore (packages/core/src/install-audit.ts) with AuditCapabilityType including 'connector' (:22), actions installed/rejected (:15). EXTEND connect/revoke to record entries. ⚠ Latent enum-drift bug to avoid: TS AuditRiskLevel allows 'critical' (install-audit.ts:16) but both DDL CHECKs only allow low/medium/high (install-audit.ts:65, schema.ts:130) — pass 'high', never 'critical'.
  • Audit read route (MISSING). No HTTP surface lists the audit trail (getRecent/getByCapability have no route). The Connector Hub's "Recent Activity" + the Extend trust-trail view need a NEW read, e.g. GET /api/connectors/activity (or a shared GET /api/extend/audit filtered to type:'connector'). Net-new but reads existing install_audit rows — no migration.
  • System-health aggregate (optional, mockup). A GET /api/connectors/health that fans registry.healthCheck() over connected connectors for the System-Health tile. Net-new thin aggregator; no new substrate.

.mind migration: None required. install_audit already exists in schema.ts:119; connector credentials live in the vault, not .mind. lastSyncAt can live as a vault sub-key or a tiny store — no schema change. (The audit table's pre-existing risk_level CHECK drift is a code-discipline note, not a migration.)

6. Shared types needed (PRD §15 vs lib/types.ts)

  • ConnectorDefinition / ConnectorHealth / ConnectorStatus already exist in packages/shared/src/types.ts:266-312 and are the canonical contract — the FE should import these instead of the ad-hoc local Connector interface in ConnectorsApp.tsx:31-38 (a lossy duplicate).
  • lib/types.ts has a thin Connector (frontend mirror, §(d) substrate inventory) — reconcile it to the shared ConnectorDefinition shape (add category, tokenExpiresAt, lastSyncAt, optional health).
  • New optional fields: ConnectorDefinition.lastSyncAt?: string (and surface category which is already declared but not always populated). PRD §15.2 lists an ExtensionType union — connectors are one member; not blocking for S07 but should be defined when S13 (Marketplace) lands.
  • ConnectorRecommendation already exists (connector-recommendations.ts:38) — reuse as-is for the Recommended tab.

7. Dependencies (screens/phases first)

  • Onboarding (S12, PRD §12.12 Tool Discovery / §16.1) feeds the Recommended tab's "based on onboarding" signal. recommendConnectors(personaId) works standalone today, so S07 is not blocked — but the richer "based on workspace needs" recommendation improves once onboarding tool-discovery answers are persisted.
  • S08 MCP Hub shares this app shell (the MCP tab). Coordinate the tab split so reworking S07 doesn't regress the MCP catalog; ideally S07 and S08 land in the same phase or S07 first with the MCP tab untouched.
  • Extend trust-trail / Marketplace (S13) shares the install-audit read route — build the audit write + read once, consume in both. Sequence the audit endpoint before/with S07's Recent Activity.
  • No hard dependency on Home/Workspace screens.

8. Effort: M

Frontend rework on a solid existing shell (new tabs, ConnectorCard extraction, wire two dead/easy adapter paths) is modest. The backend lift is small-but-real: /sync and /revoke routes + audit write on connect/revoke + an audit read route + a health aggregate — all over existing substrate (registry, vault, install_audit), zero migration. The one true unknown (real data-pull sync in the connector SDK) is phaseable to a health-probe-only v1, which keeps this M rather than L.

9. Open questions

  1. Sync semantics. Does "sync now" mean (a) re-probe health + stamp lastSyncAt (v1, cheap), or (b) re-pull/refresh cached connector data (needs a new sync() on WaggleConnector + a cache)? PRD §12.7 says "show recent sync activity" but connectors are currently runtime tools (execute(action)), not background data-syncers. Recommend v1 = health-probe + timestamp.
  2. lastSyncAt storage. Vault sub-key (connector:<id>:lastSync) vs a small dedicated store vs an install_audit derived value? Vault sub-key is lowest-risk and matches the existing connector:<id>:email pattern.
  3. revoke vs disconnect. Add /revoke as a true alias of disconnect, or have revoke additionally purge OAuth tokens (<provider>_oauth_token, oauth.ts) + write a stronger audit entry? PRD §17.3 ("revocation path") implies the latter for OAuth connectors.
  4. Audit scope. Connector-specific GET /api/connectors/activity vs a shared GET /api/extend/audit?type=connector? The latter serves S07 + S08 + S13 with one route (preferred).
  5. Audit .mind location. install_audit is per-.mind (per-workspace). Connectors are global (vault-scoped, not workspace-scoped). Confirm the audit-store decorator binds to personal.mind so the global Connector Hub trail is consistent (substrate inventory §(d) flags this exact ambiguity).
  6. Token-expiry source. ConnectorHealth.tokenExpiresAt is populated only on the fallback path (connectors.ts:49); confirm each connector's healthCheck() surfaces real expiry, else the expired-token state will never show for SDK-backed connectors.
Reference in New Issue View Git Blame Copy Permalink