waggle-os/docs/HARVEST-EXPORT-MANUAL.md

Harvest Export Manual — All Marko's AI Accounts

Purpose: Step-by-step guide to export conversation data from every AI platform for harvest into Waggle. Time estimate: ~30-45 minutes total across all platforms. Output: One folder per platform in D:\Projects\waggle-os\harvest-imports\

---

Prep: Create the import folder

mkdir D:\Projects\waggle-os\harvest-imports
mkdir D:\Projects\waggle-os\harvest-imports\chatgpt
mkdir D:\Projects\waggle-os\harvest-imports\claude-web
mkdir D:\Projects\waggle-os\harvest-imports\claude-desktop
mkdir D:\Projects\waggle-os\harvest-imports\claude-code
mkdir D:\Projects\waggle-os\harvest-imports\gemini
mkdir D:\Projects\waggle-os\harvest-imports\perplexity
mkdir D:\Projects\waggle-os\harvest-imports\cursor

---

1. ChatGPT (chatgpt.com)

Adapter: chatgpt-adapter.ts (shipped, production-tested) Format: JSON (conversations + memories + custom instructions)

Steps:

1. Go to https://chatgpt.com
2. Click your profile icon (bottom-left) → Settings
3. Click Data controls
4. Click Export data → Export
5. You'll get an email (usually within 5-30 minutes) with a download link
6. Download the ZIP file
7. Extract it — you'll get a folder with:
   • conversations.json (this is the main file)
   • user.json (account info)
   • model_comparisons.json (optional)
   • message_feedback.json (optional)
   • chat.html (visual backup, not needed)
8. Copy conversations.json to D:\Projects\waggle-os\harvest-imports\chatgpt\

What gets harvested: All conversations, custom instructions, memories, message content with timestamps.

---

2. Claude Web (claude.ai) — marolinik@gmail.com account

Adapter: claude-adapter.ts (shipped, production-tested) Format: JSON

Steps:

1. Go to https://claude.ai
2. Log in with marolinik@gmail.com
3. Click your profile icon (bottom-left) → Settings
4. Scroll to Account section
5. Click Export Data
6. Confirm the export
7. You'll get an email with a download link (usually 5-15 minutes)
8. Download the ZIP
9. Extract — look for conversations.json or similar JSON files
10. Copy all JSON files to D:\Projects\waggle-os\harvest-imports\claude-web\gmail\

Repeat for marko.markovic@egzakta.com account:

1. Log out of claude.ai
2. Log in with marko.markovic@egzakta.com
3. Same steps 3-9 above
4. Copy to D:\Projects\waggle-os\harvest-imports\claude-web\egzakta\

---

3. Claude Desktop App

Adapter: claude-adapter.ts (same as web — uses same export format) Location: Desktop app stores conversations locally

Steps:

1. Open Claude Desktop app
2. Menu → File → Export conversations (or Settings → Export)
3. If no export button: the desktop app syncs with claude.ai — your web export (step 2) already includes desktop conversations
4. If there's a separate local database:
   • Check %APPDATA%\Claude\ on Windows
   • Look for .db or .json files
   • Copy any conversation data to D:\Projects\waggle-os\harvest-imports\claude-desktop\

Note: Claude Desktop and claude.ai share the same conversation history. If you already exported from claude.ai, you likely have the desktop conversations too. Check for any offline-only conversations.

---

4. Claude Code — ALL sessions, ALL projects, BOTH accounts

Adapter: claude-code-adapter.ts (shipped, 156 frames already harvested) Format: JSONL session transcripts

Where Claude Code stores sessions:

Sessions are stored per-project in:

C:\Users\MarkoMarkovic\.claude\projects\<project-dir-encoded>\*.jsonl

Steps:

A. Gather ALL session files across ALL projects:

1. Open a terminal and run:

# List all projects with session files
find "C:/Users/MarkoMarkovic/.claude/projects" -name "*.jsonl" -type f > D:/Projects/waggle-os/harvest-imports/claude-code/session-list.txt

# Count total sessions
wc -l D:/Projects/waggle-os/harvest-imports/claude-code/session-list.txt

2. Copy all JSONL files (organized by project):

# This copies every session transcript, preserving project structure
cd "C:/Users/MarkoMarkovic/.claude/projects"
for dir in */; do
  if ls "$dir"*.jsonl 1>/dev/null 2>&1; then
    mkdir -p "D:/Projects/waggle-os/harvest-imports/claude-code/$dir"
    cp "$dir"*.jsonl "D:/Projects/waggle-os/harvest-imports/claude-code/$dir"
  fi
done

B. Key projects to verify are included:

Project dir	What it is
D--Projects-waggle-os	Waggle OS (main project — dozens of sessions)
D--Projects-SocialPresence	Social presence work
D--Projects-HiveMind	HiveMind project
D--Projects-MS-Claw*	MS Claw projects
D--Projects-eF	eF project
D--Projects-ReFarm	ReFarm project
D--Projects-TCG	TCG project
D--Projects-Dubai*	Dubai offering
D--Projects-Egzakta*	Egzakta investor pitch
C--Users-MarkoMarkovic*	Various personal projects

C. Also grab the memory files (per-project learned context):

# Copy all memory directories too
cd "C:/Users/MarkoMarkovic/.claude/projects"
for dir in */; do
  if [ -d "${dir}memory" ]; then
    mkdir -p "D:/Projects/waggle-os/harvest-imports/claude-code/${dir}memory"
    cp -r "${dir}memory/"* "D:/Projects/waggle-os/harvest-imports/claude-code/${dir}memory/"
  fi
done

D. Both accounts:

Claude Code sessions are stored locally regardless of which account you're logged in with. All sessions from both marolinik@gmail.com and marko.markovic@egzakta.com are in the same .claude/projects/ directory. The copy above captures both.

---

5. Gemini (gemini.google.com)

Adapter: gemini-adapter.ts (shipped, needs real-data verification) Format: JSON (via Google Takeout)

Steps:

1. Go to https://takeout.google.com
2. Click Deselect all (top of page)
3. Scroll down and check ONLY "Gemini Apps" (formerly Bard)
4. Click Next step
5. Choose:
   • Delivery: Send download link via email
   • Frequency: Export once
   • File type: ZIP
   • File size: 2 GB (default is fine)
6. Click Create export
7. Wait for email (can take minutes to hours depending on volume)
8. Download the ZIP
9. Extract — navigate to Takeout/Gemini Apps/
10. You'll find conversation JSON files
11. Copy all files to D:\Projects\waggle-os\harvest-imports\gemini\

---

6. Perplexity (perplexity.ai)

Adapter: perplexity-adapter.ts (shipped in S3, production-tested) Format: JSON (threads with citations)

Steps:

1. Go to https://perplexity.ai
2. Click your profile icon → Settings
3. Scroll to Account section
4. Look for Export data or Download your data
5. If no export button available:
   • Go to https://perplexity.ai/settings/account
   • Look for a data export option
   • If still not available: Perplexity may not have a bulk export yet
6. Alternative: Use the Perplexity API to fetch your thread history:
   • Check if you have API access at https://perplexity.ai/settings/api
   • Threads can be fetched programmatically
7. Copy any exported JSON to D:\Projects\waggle-os\harvest-imports\perplexity\

Note: If Perplexity doesn't offer bulk export, we can build a browser-based scraper or use the API. Let me know and I'll build it.

---

7. Cursor (cursor.sh)

Adapter: NOT BUILT YET (I'll build it when you're ready) Format: SQLite database + workspace logs

Steps:

1. Cursor stores conversations locally in:
   • Windows: %APPDATA%\Cursor\User\
   • Look for: workspaceStorage/, globalStorage/, or state.vscdb
2. Navigate to C:\Users\MarkoMarkovic\AppData\Roaming\Cursor\User\
3. Look for:
   • Any .sqlite or .db files
   • workspaceStorage\*\state.vscdb (per-workspace state)
   • globalStorage\*\ directories with conversation data
4. Copy the entire relevant folder:

mkdir -p "D:/Projects/waggle-os/harvest-imports/cursor"
cp -r "C:/Users/MarkoMarkovic/AppData/Roaming/Cursor/User/workspaceStorage" "D:/Projects/waggle-os/harvest-imports/cursor/"
cp -r "C:/Users/MarkoMarkovic/AppData/Roaming/Cursor/User/globalStorage" "D:/Projects/waggle-os/harvest-imports/cursor/"

Note: I'll reverse-engineer the format and build the adapter once you've copied the data.

---

8. Microsoft Graph (email + calendar + files) — FUTURE

Connector: NOT BUILT YET (OAuth2 + REST API needed) What it covers: Outlook email, Calendar events, OneDrive/SharePoint files

Prep (for when I build the connector):

1. Go to https://portal.azure.com
2. Navigate to App registrations → New registration
3. Name: "Waggle OS Local"
4. Redirect URI: http://localhost:3333/api/oauth/callback
5. Supported account types: "Accounts in this organizational directory only"
6. After creation, note:
   • Application (client) ID
   • Directory (tenant) ID
7. Go to Certificates & secrets → New client secret → copy the value
8. Go to API permissions → Add:
   • Mail.Read
   • Calendars.Read
   • Files.Read.All
   • User.Read
9. Admin consent if required by your Egzakta tenant

I'll build the full OAuth flow + Graph API connector. Just prep the app registration.

---

Checklist

#	Platform	Export method	Where to put it	Done?
1	ChatGPT	Settings → Export	harvest-imports/chatgpt/	[ ]
2	Claude Web (gmail)	Settings → Export	harvest-imports/claude-web/gmail/	[ ]
3	Claude Web (egzakta)	Settings → Export	harvest-imports/claude-web/egzakta/	[ ]
4	Claude Desktop	Check if separate from web	harvest-imports/claude-desktop/	[ ]
5	Claude Code (ALL)	Copy ~/.claude/projects/*.jsonl	harvest-imports/claude-code/	[ ]
6	Gemini	Google Takeout → Gemini Apps	harvest-imports/gemini/	[ ]
7	Perplexity	Settings → Export	harvest-imports/perplexity/	[ ]
8	Cursor	Copy AppData/Cursor/	harvest-imports/cursor/	[ ]
9	MS Graph	Azure App Registration	(prep only — connector not built)	[ ]

Keys to add to Waggle vault

Key	For	How to get
OPENAI_API_KEY	GPT-5 judge	https://platform.openai.com/api-keys
GOOGLE_API_KEY	Gemini 2.5 Pro judge	https://aistudio.google.com/apikey

---

When you're done

Drop me a message with:

1. "Exports ready" — I'll start the harvest pipeline on everything
2. Which API keys you've added to vault
3. Budget confirmation for the full $2-3K test

I'll continue building while you prep:

• Wire persona denylist (Phase 0)
• Build Cursor adapter
• Fix remaining review majors
• Prep the harvest pipeline for bulk ingest