oleg.m/waggle-os
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
b20b138fe485817e8d69d0c60e376e3c53af149a
waggle-os/docs/ux-refactor/gap-cards/S21-marketplace-extend.md
Oleg Maslov 0c3e2ead3b
Some checks failed
Installer Smoke / installer-smoke (push) Has been cancelled
Details
moving
2026-09-02 10:10:29 +02:00

20 KiB
Raw Blame History

Gap Card — S21 · Marketplace / Extend

Execution model: in-place incremental refactor of apps/web + targeted backend extensions. Mockups directional only (PRD §24). PRD acceptance criteria win over pixels. Every claim grounded in real files (paths + lines below).

1. Screen & purpose

Screen 21 — Marketplace / Extend Waggle (PRD §12.13, lines 648-662; blueprint catalog row 21, _blueprint_extracted.txt:432-439).

The central, power-user extension surface. PRD §10.4 names the Extend layer as Connectors, MCPs, Models, External tools, Marketplace — S21 is the Marketplace node of that layer: a single place to discover, install, update, risk-approve, and audit extensions of six kinds (Skills, Agents, Connectors, MCPs, Models, Templates) without "hunting through settings" (PRD §12.13 acceptance criterion). It is the funnel for the PRO upgrade trigger (skills + connectors + marketplace are gated to PRO; 05g-subsystem-skills-marketplace-wiki.md:3).

This card scopes the Marketplace browse/install/audit surface specifically. The sibling Extend surfaces — Connector Hub (S? / §12.7), MCP Hub (§12.8), Skills Hub (§12.6) — are separate screens that this one cross-links to; their per-domain CRUD lives in their own cards. S21 is the unified catalog + governance view across all extension types.

2. Required states (PRD / Blueprint)

Functional requirements (PRD §12.13):

  • Categories: Skills, Agents, Connectors, MCPs, Models, Templates (6 facets).
  • Search / filter / sort by category, trust, popularity, source, risk.
  • Show per-item lifecycle: installed, update available, risk approval (required), install failed.
  • Install actions write to install audit.
  • Support workspace / team scoping of installs.

Extension lifecycle states (PRD §14.7, lines 906-914): Available · Installed · Update available · Installing · Failed install · Risk approval required · Disabled/revoked.

Acceptance criterion (PRD §12.13): "Power users can extend Waggle without hunting through settings." Cross-cuts to §12.8 MCP acceptance ("powerful but always visible, scoped, auditable, and reversible") and the install-audit governance theme (_blueprint_extracted.txt:43-45,95: install audit is the "best basis for Extend governance").

Journeys touching S21: J8 install connector, J9 install MCP (audit recorded), J25 marketplace update → review changelog/risk (_blueprint_extracted.txt:203,206,252).

Mockup (directional): screens_18_21_builders_and_marketplace.png shows S21 as a card-grid catalog with a left category rail and per-card install affordances + trust badges — consistent with the spec; not a pixel target.

3. Current state in repo

Disposition: rework (the existing surfaces are reusable substrate but neither satisfies §12.13 — the catalog is hardwired to 3 of 6 categories and there is no unified Extend view with trust/risk/popularity/scope filters).

Marketplace is doubly represented today (frontend.md §f flags this as an IA cleanup point):

  1. apps/web/src/components/os/apps/MarketplaceApp.tsx (258 lines, fully read) — standalone browser. Two tabs ('search' | 'installed', :26). Calls adapter.searchMarketplace / getMarketplaceInstalled / installMarketplacePackage / uninstallMarketplacePackage. Renders a flat package list with a scanBadge (passed/failed Shield/AlertTriangle, :141-145), installed check, type/category/source chips (:223-227), and 403→waggle:tier-insufficient UpgradeModal routing (:108-117). Registered in Desktop.tsx appConfig marketplace (Desktop.tsx:93) + renderAppContent case (:347), but NO dock entry points at it — dock-tiers.ts:67 comment: "Marketplace already a tab inside Skills & Apps — no separate dock entry". So MarketplaceApp is currently reachable only via waggle:open-app events, effectively orphaned.

  2. apps/web/src/components/os/apps/CapabilitiesApp.tsx (478 lines, fully read) — "Skills & Apps", the live Extend surface (dock key skills → appId capabilities, dock-tiers.ts:64). 5 tabs: installed | starter | marketplace | tools | audit (:80,358). The marketplace tab lists SkillPacks via getMarketplacePacks; the audit tab (AuditTab, :15-58) reads GET /api/audit/installs and renders name/source/outcome/timestamp rows. It has a pack-detail drawer (PackDetail, :249-334) with trust label + bundled-skills + install. This is pack-oriented (curated bundles), not the faceted package catalog of §12.13.

Backend already present (more than the inventory claimed):

  • Full marketplace route family: packages/server/src/local/routes/marketplace.ts — search (FTS5 + faceted, :56), packs, packs/:slug, enterprise-packs (ENTERPRISE), install (PRO + SecurityGate, :180), uninstall (:377), installed (:396), security-check, sources GET/POST/DELETE (:446,457,533), categories (:567 → PACKAGE_CATEGORIES), sync, security-status, publish (PRO, :685). Sort facet supported via ?sort= (:72).
  • Install-audit READ route EXISTS — GET /api/audit/installs at packages/server/src/local/routes/skills.ts:685-703 (returns normalized { entries: [{ capabilityName, capabilityType, source, riskLevel, trustSource, approvalClass, action, initiator, detail }] }). This corrects substrate-types.md §d gap #1, which claimed "No HTTP endpoint surfaces the audit trail." The write path is live too (marketplace.ts:224-319 records every SecurityGate verdict; skills.ts:212,315,475). adapter.getAuditInstalls already wires it (frontend.md, adapter Misc).
  • Install-audit store: packages/core/src/install-audit.ts — AuditCapabilityType = native|skill|plugin|mcp|connector|marketplace (:22), full enums for action/trust/approval/risk; read API getRecent/getByCapability/getByAction/getAll.

Critical limitation (grounded): the marketplace catalog only supports 3 install kinds — InstallationType = 'skill' | 'plugin' | 'mcp' (packages/marketplace/src/types.ts:185), confirmed by 05g-subsystem-skills-marketplace-wiki.md:11 ("three installable package kinds — skill, plugin, mcp"). PRD §12.13 demands six categories incl. Agents, Connectors, Models, Templates, which have no catalog representation today (connectors are a separate /api/connectors registry with only connect/disconnect; templates are /api/workspace-templates; agents are personas/groups; models are /api/litellm/models + providers). So S21's "one catalog, six categories" is the central rework: a federated catalog read over the existing marketplace catalog + connector registry + template store + persona/agent catalog + model list, NOT a new package table.

4. Frontend work

Strategy: Consolidate the two marketplace surfaces into ONE Extend/Marketplace screen, retire the orphaned standalone, and give the new screen a federated faceted catalog. Keep CapabilitiesApp's Skills-pack flow as the Skills facet; promote a real dock entry.

Components to create:

  • apps/web/src/components/os/apps/MarketplaceApp.tsx — rework in place (reuse the file/appId so Desktop wiring + tests survive). Expand from 2 tabs to the §12.13 model: a left category rail (Skills / Agents / Connectors / MCPs / Models / Templates / All) + a top filter bar (trust, popularity, source, risk dropdowns + search) + a catalog grid. Reuse the existing scanBadge, 403→UpgradeModal routing (:108-117), and install/uninstall handlers verbatim.
  • apps/web/src/components/os/apps/extend/ExtensionCard.tsx (new) — one card type rendering the §14.7 lifecycle badge (Available/Installed/Update available/Installing/Failed/Risk-approval/Disabled), trust chip, risk chip, source, popularity. Props: { ext: Extension; onInstall; onUpdate; onRevoke; onOpenDetail }.
  • apps/web/src/components/os/apps/extend/ExtensionDetail.tsx (new, or generalize CapabilitiesApp's PackDetail :249-334) — detail drawer with permissions/scope, changelog (for "update available", J25), risk approval CTA, install-audit trail for this item (via getAuditInstalls filtered client-side, or a new ?capability= query — see §5).
  • apps/web/src/components/os/apps/extend/InstallAuditPanel.tsx (new, or lift CapabilitiesApp's AuditTab :15-58) — shared Extend governance trail; reused by S21 + Connector/MCP hubs.
  • apps/web/src/lib/extension-catalog.ts (new pure module + co-located .test.ts) — client-side federation/normalization: merge marketplace packages (searchMarketplace), connectors (getConnectors), templates (getWorkspaceTemplates), personas/groups (getPersonas/getAgentGroups), models (getModels/getProviders) into one Extension[] with a unified { type: ExtensionType, trust, risk, installed, updateAvailable, source, popularity } shape; derive lifecycle state. Keeps the fat federation logic out of the component (file-org rule: many small files).

Reuse targets: MarketplaceApp install/uninstall + 403 handler; CapabilitiesApp PackCard/PackDetail/AuditTab + lib/skill-pack-display.ts (describeTrust, summariseSkills) + lib/dedupe-packs.ts; components/ui/* (card/badge/tabs/input/select/tooltip); HintTooltip.

Adapter methods/hooks: existing — searchMarketplace, getMarketplacePacks, getMarketplaceInstalled, installMarketplacePackage, uninstallMarketplacePackage, getAuditInstalls, getConnectors, getWorkspaceTemplates, getPersonas, getModels, getProviders (all in lib/adapter.ts, per frontend.md §c). New adapter methods to add for the §16.9 gaps: getMcps, installMcp, testMcp, revokeMcp, syncConnector, revokeConnector (see §5). New hook hooks/useExtensions.ts wrapping extension-catalog.ts (catalog + filters + install/update/revoke actions + react-query caching).

IA cleanup (frontend.md §f): decide one home for marketplace. Recommended: keep MarketplaceApp as the dedicated S21 Extend surface, add a real dock entry under the extend zone-parent (dock-tiers.ts:62-71), and have CapabilitiesApp's marketplace tab deep-link into it (or drop that tab) to remove the double representation.

5. Backend work (PRD §16.9 endpoint-by-endpoint)

PRD §16.9 endpoint Status EXTEND vs NET-NEW + substrate / .mind
GET /api/connectors EXISTS connectors.ts:6. No change.
POST /api/connectors/:id/connect EXISTS connectors.ts:55. No change.
POST /api/connectors/:id/sync MISSING NET-NEW thin action in connectors.ts (re-init/re-fetch from connected service). Touches connector credential resolution (vault) + connector definition; record an audit entry (auditStore.record, type connector). No .mind migration.
POST /api/connectors/:id/revoke PARTIAL EXTEND — alias to existing POST /api/connectors/:id/disconnect (connectors.ts:107, removes vault creds + sub-keys). Same intent, add /revoke route delegating to disconnect + write a revoked audit entry.
GET /api/mcps PARTIAL EXTEND/NET-NEW — no /api/mcps. MCP catalog lives in @waggle/shared mcp-catalog.ts; installed MCPs surface inside GET /api/capabilities/status (mcpServers[]) and the marketplace catalog (waggle_install_type:'mcp', marketplace/src/mcp-registry.ts). Add a dedicated GET /api/mcps route that joins catalog + installed-state + .mcp.json. Reads existing substrate; no migration.
POST /api/mcps/install PARTIAL EXTEND — route through the existing marketplace installer (POST /api/marketplace/install, which already handles installType:'mcp' → writes .mcp.json, installer.ts:580). Add an MCP-specific endpoint that resolves an MCP id → marketplace package → install, OR keep marketplace install as the single path and have the FE call it. Audit already recorded.
POST /api/mcps/:id/test MISSING NET-NEW — no MCP health/test route. Closest analog is GET /api/connectors/:id/health (connectors.ts:16). Add an MCP test route (spawn/handshake the MCP server, report ok/error). No .mind.
POST /api/mcps/:id/revoke MISSING NET-NEW — no MCP revoke-by-id. Closest: DELETE /api/plugins/:name (skills.ts). Add an MCP revoke route (remove from .mcp.json + record revoked audit).
GET /api/marketplace PARTIAL EXTEND — bare path = alias of GET /api/marketplace/search with default params (marketplace.ts:56). Add alias or accept both.
POST /api/marketplace/install EXISTS marketplace.ts:180 (PRO + SecurityGate). No change.

Plus implied additions for §12.13 (not in §16.9's list but required by FRs):

  • Six-category catalog. The §12.13 categories (Agents/Models/Templates) have no marketplace representation (InstallationType is skill|plugin|mcp only, types.ts:185). Two options: (a) widen InstallationType + catalog schema to add agent|model|template (a marketplace-DB change in packages/marketplace, NOT a .mind migration — marketplace.db is a separate SQLite store) and seed registries; or (b) federate at read time (preferred for incremental scope) — the FE extension-catalog.ts merges marketplace (skill/plugin/mcp) + /api/connectors + /api/workspace-templates + /api/personas+/api/agent-groups + /api/litellm/models. Option (b) needs no backend change beyond §16.9; option (a) is a later phase if a true unified catalog with publish/install for all 6 kinds is wanted.
  • Install-audit read for Extend governance. EXISTS — GET /api/audit/installs (skills.ts:685). (Substrate inventory §d#1 said missing; it is present.) Optional EXTEND: add a ?capability= / ?type= filter param so the detail drawer can show per-item history without client-side filtering (currently only ?limit=). The store already has getByCapability() (install-audit.ts:125) — just expose it.
  • Workspace/team scoping of installs (§12.13). No per-workspace install scoping exists today (installs are global to ~/.waggle/). This is net-new product surface; recommend deferring to Phase 4 polish (PRD §22 risk register flags "marketplace scope creep" — PRD:1435 — "Start with catalog + install audit, postpone billing/public marketplace").

.mind migration flag: none required for S21. The audit table (install_audit) already exists in both DDL sites (install-audit.ts:54 + schema.ts:119). Latent bug to flag (substrate-types §d#2): AuditRiskLevel TS includes 'critical' (install-audit.ts:16) but both DDL CHECKs only allow ('low','medium','high') (install-audit.ts:65, schema.ts:130) — a record({riskLevel:'critical'}) throws. The marketplace route sidesteps it (maps CRITICAL→high+blocked, marketplace.ts:228). If S21 surfaces a true "critical risk approval" state that writes critical, this CHECK must be widened first (idempotent ADD/relax via the migration runner's ADD-COLUMN pattern, mind/db.ts:116).

6. Shared types needed (PRD §15 vs lib/types.ts)

  • ExtensionType (PRD §15.2, line 953: 'skill' | 'connector' | 'mcp' | 'model' | 'template' | 'external_tool') — MISSING from apps/web/src/lib/types.ts (substrate-types §e). Add. Note PRD union has external_tool but §12.13 categories say "Agents" not "external_tool" — reconcile: §12.13 needs an 'agent' category; PRD §15.2 ExtensionType lacks agent and adds external_tool. Flag as open question; likely the union should be skill|agent|connector|mcp|model|template (+ optional external_tool).
  • Extension interface (new) — { id, name, description, type: ExtensionType, trust: AuditTrustSource-ish, risk: AuditRiskLevel, source, popularity?, version?, installed, updateAvailable, lifecycle: 'available'|'installed'|'update_available'|'installing'|'failed'|'risk_approval'|'disabled', scope?: Scope }. No PRD §15 interface spells this out; derive from §12.13 + §14.7.
  • Scope (PRD §15.2: 'personal'|'workspace'|'team'|'organization') — MISSING, needed for install scoping; add (shared with other Extend/Memory cards).
  • Existing FE MarketplacePackage (local interface in MarketplaceApp.tsx:8-18) and SkillPack (types.ts) are narrower than Extension; extension-catalog.ts normalizes both into Extension.
  • Reuse the canonical audit enums (AuditTrustSource, AuditRiskLevel, AuditCapabilityType from packages/core/src/install-audit.ts:16-22) rather than re-declaring trust/risk literals in the FE — export them or mirror minimally.

7. Dependencies (screens / phases first)

  • PRD phasing: Extend layer = Phase 4 / Sprint 7 (PRD:241-245,1353-1357). Comes after Work (Home/Workspace), Intelligence (Agents/Skills), so S21 should land in that phase.
  • Depends on / cross-links:
    • Connector Hub (§12.7) and MCP Hub (§12.8) screens — S21 federates their catalogs and shares the InstallAuditPanel; their /sync//revoke//test backend routes (§5) are shared deliverables.
    • Skills Hub (§12.6) — S21's Skills facet reuses CapabilitiesApp's pack flow.
    • Agent Center (§12.9), Models (Settings→Models), Templates (workspace-templates) — needed as catalog sources for the 6-category federation.
    • Dock IA refactor (dock-tiers.ts Extend zone) — S21 needs a real dock entry (currently none).
    • UpgradeModal / tier gating (waggle:tier-insufficient event) — already wired; reuse.
  • Blocks: nothing downstream; it is a leaf surface.

8. Effort

L. The frontend rework is moderate (consolidate 2 surfaces → 1 faceted catalog + extract shared ExtensionCard/InstallAuditPanel + new extension-catalog.ts federation). What pushes it to L: the 6-category federation spans five different backend domains (marketplace, connectors, templates, personas/agents, models) each with a different shape, plus 6 net-new/extend §16.9 routes (connector sync/revoke, mcps list/install/test/revoke) and the FE/BE type reconciliation. Not XL because no .mind migration, no new data store (federate-at-read), audit read route already exists, and install/uninstall + tier-gating + audit-trail plumbing are all already live.

9. Open questions

  1. ExtensionType reconciliation. PRD §12.13 categories = Skills/Agents/Connectors/MCPs/Models/Templates (6, incl. Agents). PRD §15.2 ExtensionType = skill/connector/mcp/model/template/external_tool (6, incl. external_tool, no agent). Which is canonical — add agent, keep external_tool, or both (7)?
  2. Federate-at-read vs unified catalog table. Ship S21 by federating existing domains client-side (no marketplace-DB change, fast), or invest in widening InstallationType + marketplace.db to natively catalog all 6 kinds (enables publish/install/version-tracking parity for agents/models/templates)? Recommend federate-first; revisit per §22 "postpone public marketplace".
  3. Install scoping (§12.13 "workspace/team scoping"). Installs are global to ~/.waggle/ today. Is per-workspace/per-team install scoping in scope for Phase 4, or deferred (it implies new persisted scope state on every installed capability)?
  4. "Update available" + changelog (J25). No version/changelog tracking exists for installed packages beyond a version column on the marketplace package. Where does the "update available" signal come from — catalog sync diffing installed version vs latest, or a new field?
  5. MCP test semantics. POST /api/mcps/:id/test — does "test" mean spawn-and-handshake the MCP server (live), or static manifest/permission validation? Affects whether it can run without launching a process.
  6. Should MarketplaceApp or CapabilitiesApp be the canonical S21 home (IA double-representation, frontend.md §f)? Recommend MarketplaceApp reworked + real dock entry; CapabilitiesApp keeps Skills/Tools/Audit and drops its marketplace tab.
Reference in New Issue View Git Blame Copy Permalink