This commit is contained in:
338
.agents/skills/ax-agent-optimize/SKILL.md
Normal file
338
.agents/skills/ax-agent-optimize/SKILL.md
Normal file
@@ -0,0 +1,338 @@
|
||||
---
|
||||
name: ax-agent-optimize
|
||||
description: This skill helps an LLM generate correct AxAgent tuning and evaluation code using @ax-llm/ax. Use when the user asks about agent.optimize(...), judgeOptions, eval datasets, optimization targets, saved optimizedProgram artifacts, or recursive optimization guidance.
|
||||
version: "19.0.33"
|
||||
---
|
||||
|
||||
# AxAgent Optimize Codegen Rules (@ax-llm/ax)
|
||||
|
||||
Use this skill for `agent.optimize(...)` workflows. Prefer short, modern, copyable patterns. Do not repeat general agent-authoring guidance unless the user needs it.
|
||||
|
||||
Your job is to help the model choose a good optimization setup for the user's actual goal:
|
||||
|
||||
- If the user wants better tool use, prefer action-aware tasks and either a deterministic metric or the built-in judge depending on how objective the scoring is.
|
||||
- If the user wants better wording only, responder optimization may be enough.
|
||||
- If the user wants reusable improvements, include artifact save/load.
|
||||
- If the user wants cost or recursion behavior improved, make the eval tasks expose those tradeoffs explicitly.
|
||||
|
||||
## Use These Defaults
|
||||
|
||||
- Use `agent.optimize(...)` only after the agent is already configured and runnable.
|
||||
- Prefer a deterministic custom `metric` when success is easy to score from the prediction and task record.
|
||||
- Prefer the built-in judge path for open-ended assistant tasks: `judgeAI` plus `judgeOptions`.
|
||||
- Only reach for a plain typed `AxGen` evaluator when the user needs LLM-as-judge behavior outside the built-in `agent.optimize(...)` flow.
|
||||
- Default optimize target is `root.actor`; use `target: 'responder'` or explicit program IDs only when the user clearly asks for that.
|
||||
- Use eval-safe tools or in-memory mocks because optimization replays tasks many times.
|
||||
- Prefer precise tool return schemas such as `f.object(...)` over vague `f.json(...)` whenever the agent must reason about returned fields.
|
||||
- Prefer task wording with canonical entity names like "the Atlas project" instead of ambiguous labels like "Atlas" when ambiguity could trigger pointless clarification.
|
||||
- Save `result.optimizedProgram`, then restore with `new AxOptimizedProgramImpl(...)` and `agent.applyOptimization(...)`.
|
||||
- When recursive behavior matters, keep `mode: 'advanced'` on the agent and tune against realistic `recursionOptions`.
|
||||
|
||||
## Decision Guide
|
||||
|
||||
Pick the optimization shape from the user's need:
|
||||
|
||||
- "Make the agent use tools correctly" -> optimize `root.actor` with `expectedActions` and `forbiddenActions`.
|
||||
- "Make final answers read better" -> consider `target: 'responder'`, but only if the task is not mostly tool-selection or clarification behavior.
|
||||
- "Make the whole agent better" -> use the default actor target first; only broaden target selection when the user clearly wants that extra scope.
|
||||
- "Tune recursive delegation" -> keep `mode: 'advanced'` and use tasks that actually exercise recursion depth, fan-out, and termination choices.
|
||||
- "Compare before and after" -> include a held-out task plus artifact save/load and replay.
|
||||
|
||||
Choose task design carefully:
|
||||
|
||||
- Prefer a small number of realistic tasks over broad but vague datasets.
|
||||
- Prefer concrete criteria over generic "be helpful" language.
|
||||
- Prefer explicit action expectations when correctness depends on tools, recipients, dates, or side effects.
|
||||
- Prefer eval-safe mocks anytime the task touches email, scheduling, external APIs, or persistence.
|
||||
|
||||
## Make Agents Optimizable
|
||||
|
||||
Optimization works much better when the agent and dataset remove avoidable ambiguity:
|
||||
|
||||
- Prefer typed tool outputs over free-form JSON blobs so the actor can rely on exact field names.
|
||||
- Tell the actor the exact tool fields it may use when payload shape matters.
|
||||
- Explicitly ban invented fields if the model has any reason to guess hidden IDs or alternate key names.
|
||||
- If recursive children only see explicit `llmQuery(..., context)` payloads, say that directly in the actor prompt.
|
||||
- For recursive synthesis, tell the agent what the narrowed context should look like before delegation.
|
||||
- Keep `maxSubAgentCalls` small in examples unless the user is explicitly testing broad fan-out behavior.
|
||||
- Use canonical, unambiguous task wording so the model does not burn turns asking for fake clarification.
|
||||
- In JS-runtime agents, require raw runnable JavaScript only. Ban `javascript:` prefixes, mixed prose/code, and multi-snippet turns.
|
||||
|
||||
Good pattern:
|
||||
|
||||
- tool schema says exactly what fields exist
|
||||
- task names the exact entity to look up
|
||||
- actor prompt says which fields to extract before delegation
|
||||
- metric or judge penalizes unnecessary recursion and tool misuse
|
||||
|
||||
Bad pattern:
|
||||
|
||||
- tool returns `json` with an underspecified shape
|
||||
- task uses overloaded names like `Atlas` without clarifying whether that is a project, team, or account
|
||||
- recursive child is expected to infer hidden parent state that was never passed in context
|
||||
- code agent is allowed to mix natural language with JavaScript in the same turn
|
||||
|
||||
## Metric vs Judge
|
||||
|
||||
Choose the scoring path based on how objectively the task can be measured:
|
||||
|
||||
- Use a custom `metric` when you can score success directly from `prediction` and `example`.
|
||||
- Use the built-in agent judge when success depends on a full-run qualitative review across tool choices, clarifications, and final output.
|
||||
- Use `judgeOptions.description` to tell the built-in judge what to value most.
|
||||
- Use helper-based judge code only when the user is not inside `agent.optimize(...)` and still wants LLM judging.
|
||||
|
||||
Quick rules:
|
||||
|
||||
- Tool correctness with exact expected calls or forbidden calls: prefer a deterministic metric first.
|
||||
- Simple extraction or classification with known correct answers: prefer a deterministic metric.
|
||||
- Open-ended assistant quality, nuanced clarification behavior, or broad synthesis quality: prefer the built-in judge.
|
||||
- GEPA or optimizer flows outside agents that still need LLM judging: use a plain typed `AxGen` evaluator.
|
||||
|
||||
Important:
|
||||
|
||||
- A custom `metric` overrides the built-in judge path entirely.
|
||||
- Do not introduce a dedicated judge abstraction in new examples; prefer a plain typed `AxGen`.
|
||||
- Do not add both a custom `metric` and judge guidance unless the user explicitly wants two separate scoring systems and understands only the custom metric drives optimization.
|
||||
- If the user builds a plain `AxGen` judge metric, prefer a numeric `score:number` output over a string tier when possible. It is simpler and less fragile in practice.
|
||||
|
||||
## Canonical Pattern
|
||||
|
||||
```typescript
|
||||
import {
|
||||
AxAIGoogleGeminiModel,
|
||||
AxJSRuntime,
|
||||
AxOptimizedProgramImpl,
|
||||
axDefaultOptimizerLogger,
|
||||
agent,
|
||||
ai,
|
||||
f,
|
||||
fn,
|
||||
} from '@ax-llm/ax';
|
||||
|
||||
const tools = [
|
||||
fn('sendEmail')
|
||||
.namespace('email')
|
||||
.description('Send an email message')
|
||||
.arg('to', f.string('Recipient email address'))
|
||||
.arg('body', f.string('Email body text'))
|
||||
.returns(
|
||||
f.object({
|
||||
sent: f.boolean('Whether the email was sent'),
|
||||
to: f.string('Recipient email address'),
|
||||
})
|
||||
)
|
||||
.handler(async ({ to }) => ({ sent: true, to }))
|
||||
.build(),
|
||||
];
|
||||
|
||||
const studentAI = ai({
|
||||
name: 'google-gemini',
|
||||
apiKey: process.env.GOOGLE_APIKEY!,
|
||||
config: { model: AxAIGoogleGeminiModel.Gemini25FlashLite, temperature: 0.2 },
|
||||
});
|
||||
|
||||
const judgeAI = ai({
|
||||
name: 'google-gemini',
|
||||
apiKey: process.env.GOOGLE_APIKEY!,
|
||||
config: { model: AxAIGoogleGeminiModel.Gemini3Pro, temperature: 1.0 },
|
||||
});
|
||||
|
||||
const assistant = agent('query:string -> answer:string', {
|
||||
ai: studentAI,
|
||||
judgeAI,
|
||||
contextFields: [],
|
||||
runtime: new AxJSRuntime(),
|
||||
functions: { local: tools },
|
||||
contextPolicy: { preset: 'checkpointed', budget: 'balanced' },
|
||||
judgeOptions: {
|
||||
description: 'Prefer correct tool use over polished wording.',
|
||||
model: 'judge-model',
|
||||
},
|
||||
});
|
||||
|
||||
const tasks = [
|
||||
{
|
||||
input: { query: 'Send an email to Jim saying good morning.' },
|
||||
criteria: 'Use the email tool and send the message to Jim.',
|
||||
expectedActions: ['email.sendEmail'],
|
||||
},
|
||||
];
|
||||
|
||||
const result = await assistant.optimize(tasks, {
|
||||
target: 'actor',
|
||||
maxMetricCalls: 12,
|
||||
verbose: true,
|
||||
optimizerLogger: axDefaultOptimizerLogger,
|
||||
onProgress: (progress) => {
|
||||
console.log(
|
||||
`round ${progress.round}/${progress.totalRounds} current=${progress.currentScore} best=${progress.bestScore}`
|
||||
);
|
||||
},
|
||||
});
|
||||
|
||||
const saved = JSON.stringify(result.optimizedProgram, null, 2);
|
||||
const restored = new AxOptimizedProgramImpl(JSON.parse(saved));
|
||||
assistant.applyOptimization(restored);
|
||||
```
|
||||
|
||||
## Deterministic Metric Pattern
|
||||
|
||||
Use this when the task has crisp correctness and cost/behavior tradeoffs:
|
||||
|
||||
```typescript
|
||||
const result = await assistant.optimize(tasks, {
|
||||
target: 'actor',
|
||||
metric: ({ prediction, example }) => {
|
||||
if (prediction.completionType !== 'final' || !prediction.output) {
|
||||
return 0;
|
||||
}
|
||||
|
||||
let score = 0;
|
||||
|
||||
if (prediction.output.answer.includes('Jim')) score += 0.4;
|
||||
|
||||
if (
|
||||
prediction.functionCalls.some(
|
||||
(call) => call.qualifiedName === 'email.sendEmail'
|
||||
)
|
||||
) {
|
||||
score += 0.4;
|
||||
}
|
||||
|
||||
if ((prediction.recursiveStats?.recursiveCallCount ?? 0) === 0) {
|
||||
score += 0.2;
|
||||
}
|
||||
|
||||
return score;
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
Use this pattern when:
|
||||
|
||||
- the task has a known correct answer or exact action pattern
|
||||
- recursion cost or tool count must be measured explicitly
|
||||
- you want repeatable, low-variance optimization runs
|
||||
|
||||
## Built-In Judge Pattern
|
||||
|
||||
Use this when the agent behavior needs holistic review:
|
||||
|
||||
```typescript
|
||||
const result = await assistant.optimize(tasks, {
|
||||
judgeAI,
|
||||
judgeOptions: {
|
||||
model: AxAIGoogleGeminiModel.Gemini3Pro,
|
||||
description:
|
||||
'Be strict about unnecessary delegation, weak clarifications, and incorrect tool choices.',
|
||||
},
|
||||
maxMetricCalls: 12,
|
||||
});
|
||||
```
|
||||
|
||||
Use this pattern when:
|
||||
|
||||
- task quality is open-ended or hard to score exactly
|
||||
- the final answer quality matters together with the action trace
|
||||
- the user wants a judge to consider clarifications, tool errors, and overall completion quality
|
||||
|
||||
## Plain `AxGen` Judge Pattern
|
||||
|
||||
Use this only when the user needs LLM judging outside the built-in `agent.optimize(...)` path:
|
||||
|
||||
```typescript
|
||||
import { AxGen, s } from '@ax-llm/ax';
|
||||
|
||||
const judgeGen = new AxGen(
|
||||
s(`
|
||||
taskInput:json "Task input",
|
||||
candidateOutput:json "Candidate output",
|
||||
expectedOutput?:json "Optional reference output"
|
||||
->
|
||||
score:number "Normalized score from 0 to 1"
|
||||
`)
|
||||
);
|
||||
judgeGen.setInstruction(
|
||||
'Score the candidate output from 0 to 1. Reward correctness and task completion. Return only the score field.'
|
||||
);
|
||||
|
||||
const metric = async ({ prediction, example }) => {
|
||||
const result = await judgeGen.forward(judgeAI, {
|
||||
taskInput: example,
|
||||
candidateOutput: prediction,
|
||||
expectedOutput: example.expectedOutput,
|
||||
});
|
||||
|
||||
return Math.max(0, Math.min(1, result.score));
|
||||
};
|
||||
|
||||
const result = await optimizer.compile(program, train, metric, {
|
||||
validationExamples: validation,
|
||||
});
|
||||
```
|
||||
|
||||
Use this pattern when:
|
||||
|
||||
- the user is optimizing an `AxGen`, flow, or another program directly
|
||||
- the user wants LLM judging without the higher-level `agent.optimize(...)` wrapper
|
||||
- the user wants to inspect judge results directly, not just a numeric score
|
||||
|
||||
## Dataset And Judge Rules
|
||||
|
||||
- Pass already-loaded tasks. Do not invent a benchmark loader unless the user asks for one.
|
||||
- Use `expectedActions` and `forbiddenActions` when tool correctness matters.
|
||||
- `judgeOptions` mirrors normal forward options and supports extra judge guidance through `description`.
|
||||
- The built-in judge scores from the full agent run, not just the final reply. It can see completion type, clarification payload, final output, action log, normalized function calls, tool errors, and turn count.
|
||||
- For recursive advanced-mode evals, the built-in judge can also see `recursiveTrace` and `recursiveStats`.
|
||||
- If the user provides a custom `metric`, that overrides the built-in judge path.
|
||||
- If the user provides an LLM-based custom metric, keep the output schema as small as possible and prefer a direct numeric score.
|
||||
|
||||
Decision rules:
|
||||
|
||||
- Prefer a custom metric when the user has deterministic business scoring, exact action expectations, or explicit cost tradeoffs.
|
||||
- Prefer the built-in judge when the user wants practical assistant-quality tuning and does not already have a trusted metric.
|
||||
- Prefer a plain typed `AxGen` evaluator when the user is not calling `agent.optimize(...)` but still wants LLM judging.
|
||||
- Prefer `judgeOptions.description` to steer the judge toward the user's real priority, such as tool correctness, brevity, groundedness, or policy compliance.
|
||||
|
||||
## Eval Semantics
|
||||
|
||||
- `agent.optimize(...)` runs each evaluation rollout from a clean continuation state.
|
||||
- Saved runtime state from `getState()` and `setState(...)` is not used during eval rollouts.
|
||||
- During optimize/eval, `askClarification(...)` is treated as a scored evaluation outcome instead of going through the responder.
|
||||
- For clarification outcomes in custom metrics, expect `prediction.completionType === 'askClarification'`, populated `prediction.clarification`, and absent `prediction.output`.
|
||||
- For final outcomes in custom metrics, expect `prediction.completionType === 'final'` and populated `prediction.output`.
|
||||
- `target: 'responder'` still works, but clarification-heavy tasks are usually low-signal for responder optimization.
|
||||
|
||||
## Recursive Optimization Notes
|
||||
|
||||
- Recursive-slot artifacts require an agent configured for recursive advanced mode.
|
||||
- Keep `mode: 'advanced'` top-level; child recursion behavior still follows `recursionOptions`.
|
||||
- When recursive behavior matters, tune against the same `maxDepth` and tool/discovery structure you expect in production.
|
||||
- Use recursive traces and recursive stats when the user wants to diagnose where token or delegation cost is coming from.
|
||||
- For recursion-efficiency tuning, prefer a deterministic metric unless the user specifically needs a qualitative LLM review of decomposition quality.
|
||||
- Tell the actor that recursive children only see passed context, not parent globals or prior tool results.
|
||||
- For synthesis-style recursive tasks, specify the desired delegation pattern explicitly, for example "use at most one focused delegated child analysis after narrowing the tool output in JS."
|
||||
- Penalize over-decomposition directly in the metric or judge prompt.
|
||||
- If one training task keeps collapsing to zero, inspect that task first instead of adding more optimizer rounds. Most failures come from task ambiguity, weak tool schemas, or vague delegation guidance rather than GEPA itself.
|
||||
|
||||
## Artifacts And Replay
|
||||
|
||||
- Save `result.optimizedProgram` if the user wants portable artifacts.
|
||||
- Restore artifacts with `new AxOptimizedProgramImpl(...)`, then call `agent.applyOptimization(...)`.
|
||||
- For demonstrations, use fresh eval-safe tool state for baseline, optimize, and restored replay so side effects do not leak across phases.
|
||||
- If the user wants to show improvement, run a held-out task before optimization, then replay it on a freshly restored optimized agent.
|
||||
|
||||
## Examples
|
||||
|
||||
- [RLM Agent Optimize](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/rlm-agent-optimize.ts) — Gemini office-assistant tuning with save/load
|
||||
- [RLM Agent Recursive Optimize](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/rlm-agent-recursive-optimize.ts) — recursive-slot optimization artifacts
|
||||
|
||||
## Do Not Generate
|
||||
|
||||
- Do not optimize against production tools with real side effects unless the user explicitly wants that.
|
||||
- Do not recommend responder-only optimization by default for clarification-heavy workflows.
|
||||
- Do not omit artifact save/load steps when the user asks for reusable optimized configurations.
|
||||
- Do not introduce a dedicated judge class or helper abstraction in new agent-optimize examples; prefer the built-in judge path or a plain typed `AxGen`.
|
||||
- Do not rely on vague `json` tool returns when the agent must reason about specific fields across recursive steps.
|
||||
- Do not leave recursive child context implicit. If the child needs a fact, pass it explicitly.
|
||||
- Do not let code-generation agents mix prose and JavaScript if the user is optimizing runtime behavior.
|
||||
1090
.agents/skills/ax-agent/SKILL.md
Normal file
1090
.agents/skills/ax-agent/SKILL.md
Normal file
File diff suppressed because it is too large
Load Diff
245
.agents/skills/ax-ai/SKILL.md
Normal file
245
.agents/skills/ax-ai/SKILL.md
Normal file
@@ -0,0 +1,245 @@
|
||||
---
|
||||
name: ax-ai
|
||||
description: This skill helps an LLM generate correct AI provider setup and configuration code using @ax-llm/ax. Use when the user asks about ai(), providers, models, presets, embeddings, extended thinking, context caching, or mentions OpenAI/Anthropic/Google/Azure/Groq/DeepSeek/Mistral/Cohere/Together/Ollama/HuggingFace/Reka/OpenRouter with @ax-llm/ax.
|
||||
version: "19.0.33"
|
||||
---
|
||||
|
||||
# AI Provider Codegen Rules (@ax-llm/ax)
|
||||
|
||||
Use this skill to generate AI provider setup, configuration, and chat code. Prefer short, modern, copyable patterns. Do not write tutorial prose unless the user explicitly asks for explanation.
|
||||
|
||||
## Quick Setup
|
||||
|
||||
```typescript
|
||||
import { ai } from '@ax-llm/ax';
|
||||
|
||||
const openai = ai({ name: 'openai', apiKey: 'sk-...' });
|
||||
const Codex = ai({ name: 'anthropic', apiKey: 'sk-ant-...' });
|
||||
const gemini = ai({ name: 'google-gemini', apiKey: 'AIza...' });
|
||||
const azure = ai({ name: 'azure-openai', apiKey: 'your-key', resourceName: 'your-resource', deploymentName: 'gpt-4' });
|
||||
const groq = ai({ name: 'groq', apiKey: 'gsk_...' });
|
||||
const deepseek = ai({ name: 'deepseek', apiKey: 'sk-...' });
|
||||
const mistral = ai({ name: 'mistral', apiKey: 'your-key' });
|
||||
const cohere = ai({ name: 'cohere', apiKey: 'your-key' });
|
||||
const together = ai({ name: 'together', apiKey: 'your-key' });
|
||||
const openrouter = ai({ name: 'openrouter', apiKey: 'your-key' });
|
||||
const ollama = ai({ name: 'ollama', url: 'http://localhost:11434' });
|
||||
const hf = ai({ name: 'huggingface', apiKey: 'hf_...' });
|
||||
const reka = ai({ name: 'reka', apiKey: 'your-key' });
|
||||
const grok = ai({ name: 'x-grok', apiKey: 'your-key' });
|
||||
```
|
||||
|
||||
## Model Presets
|
||||
|
||||
```typescript
|
||||
import { ai, AxAIGoogleGeminiModel } from '@ax-llm/ax';
|
||||
|
||||
const gemini = ai({
|
||||
name: 'google-gemini',
|
||||
apiKey: process.env.GOOGLE_APIKEY!,
|
||||
config: { model: 'simple' },
|
||||
models: [
|
||||
{ key: 'tiny', model: AxAIGoogleGeminiModel.Gemini20FlashLite, description: 'Fast + cheap', config: { maxTokens: 1024, temperature: 0.3 } },
|
||||
{ key: 'simple', model: AxAIGoogleGeminiModel.Gemini20Flash, description: 'Balanced', config: { temperature: 0.6 } },
|
||||
],
|
||||
});
|
||||
|
||||
await gemini.chat({ model: 'tiny', chatPrompt: [{ role: 'user', content: 'Hi' }] });
|
||||
```
|
||||
|
||||
## Chat
|
||||
|
||||
```typescript
|
||||
const res = await llm.chat({
|
||||
chatPrompt: [
|
||||
{ role: 'system', content: 'You are concise.' },
|
||||
{ role: 'user', content: 'Write a haiku about the ocean.' },
|
||||
],
|
||||
});
|
||||
console.log(res.results[0]?.content);
|
||||
```
|
||||
|
||||
## Common Options
|
||||
|
||||
- `stream` (boolean): enable SSE; true by default
|
||||
- `thinkingTokenBudget`: `'minimal'` | `'low'` | `'medium'` | `'high'` | `'highest'` | `'none'`
|
||||
- `showThoughts`: include thoughts in output
|
||||
- `functionCallMode`: `'auto'` | `'native'` | `'prompt'`
|
||||
- `debug`, `logger`, `tracer`, `rateLimiter`, `timeout`
|
||||
|
||||
## Extended Thinking
|
||||
|
||||
```typescript
|
||||
import { ai, AxAIAnthropicModel } from '@ax-llm/ax';
|
||||
|
||||
const Codex = ai({
|
||||
name: 'anthropic',
|
||||
apiKey: process.env.ANTHROPIC_APIKEY!,
|
||||
config: { model: AxAIAnthropicModel.Claude46Opus },
|
||||
});
|
||||
|
||||
const res = await Codex.chat(
|
||||
{ chatPrompt: [{ role: 'user', content: 'Solve step by step...' }] },
|
||||
{ thinkingTokenBudget: 'medium', showThoughts: true },
|
||||
);
|
||||
console.log(res.results[0]?.thought);
|
||||
console.log(res.results[0]?.content);
|
||||
```
|
||||
|
||||
### Budget Levels
|
||||
|
||||
| Level | Anthropic (tokens) | Gemini (tokens) |
|
||||
|---|---|---|
|
||||
| `'none'` | disabled | minimal |
|
||||
| `'minimal'` | 1,024 | 200 |
|
||||
| `'low'` | 5,000 | 800 |
|
||||
| `'medium'` | 10,000 | 5,000 |
|
||||
| `'high'` | 20,000 | 10,000 |
|
||||
| `'highest'` | 32,000 | 24,500 |
|
||||
|
||||
### Anthropic Model-Specific Behavior
|
||||
|
||||
- Opus 4.6: adaptive thinking, effort levels
|
||||
- Opus 4.5: budget_tokens + effort levels (capped at `'high'`)
|
||||
- Other thinking models: budget tokens only
|
||||
|
||||
### Custom Thinking Levels
|
||||
|
||||
```typescript
|
||||
const Codex = ai({
|
||||
name: 'anthropic',
|
||||
apiKey: '...',
|
||||
config: {
|
||||
model: AxAIAnthropicModel.Claude46Opus,
|
||||
thinkingTokenBudgetLevels: {
|
||||
minimal: 2048,
|
||||
low: 8000,
|
||||
medium: 16000,
|
||||
high: 25000,
|
||||
highest: 40000,
|
||||
},
|
||||
effortLevelMapping: {
|
||||
minimal: 'low',
|
||||
low: 'medium',
|
||||
medium: 'high',
|
||||
high: 'high',
|
||||
highest: 'max',
|
||||
},
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
## Embeddings
|
||||
|
||||
```typescript
|
||||
const { embeddings } = await llm.embed({
|
||||
texts: ['hello', 'world'],
|
||||
embedModel: 'text-embedding-005',
|
||||
});
|
||||
```
|
||||
|
||||
## Context Caching
|
||||
|
||||
```typescript
|
||||
const result = await gen.forward(llm, { code, language }, {
|
||||
mem,
|
||||
sessionId: 'code-review-session',
|
||||
contextCache: {
|
||||
ttlSeconds: 3600,
|
||||
cacheBreakpoint: 'after-examples',
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
Breakpoint values: `'system'` | `'after-functions'` | `'after-examples'`
|
||||
|
||||
Provider behavior:
|
||||
|
||||
- Google Gemini: explicit caching with cache resource ID, auto TTL refresh
|
||||
- Anthropic: implicit via `cache_control` markers
|
||||
|
||||
### External Registry (serverless)
|
||||
|
||||
```typescript
|
||||
const registry: AxContextCacheRegistry = {
|
||||
get: async (key) => { /* redis.get */ },
|
||||
set: async (key, entry) => { /* redis.set */ },
|
||||
};
|
||||
```
|
||||
|
||||
## AWS Bedrock
|
||||
|
||||
```typescript
|
||||
import { AxAIBedrock, AxAIBedrockModel } from '@ax-llm/ax-ai-aws-bedrock';
|
||||
|
||||
const bedrock = new AxAIBedrock({
|
||||
region: 'us-east-2',
|
||||
fallbackRegions: ['us-west-2'],
|
||||
config: { model: AxAIBedrockModel.ClaudeSonnet4 },
|
||||
});
|
||||
```
|
||||
|
||||
## Vercel AI SDK Integration
|
||||
|
||||
```typescript
|
||||
import { ai } from '@ax-llm/ax';
|
||||
import { AxAIProvider } from '@ax-llm/ax-ai-sdk-provider';
|
||||
import { generateText } from 'ai';
|
||||
|
||||
const axAI = ai({ name: 'openai', apiKey: process.env.OPENAI_APIKEY! });
|
||||
const model = new AxAIProvider(axAI);
|
||||
const result = await generateText({
|
||||
model,
|
||||
messages: [{ role: 'user', content: 'Hello!' }],
|
||||
});
|
||||
```
|
||||
|
||||
## MCP + AxJSRuntime
|
||||
|
||||
```typescript
|
||||
import { AxMCPClient } from '@ax-llm/ax';
|
||||
import { axCreateMCPStdioTransport } from '@ax-llm/ax-tools';
|
||||
|
||||
const transport = axCreateMCPStdioTransport({
|
||||
command: 'npx',
|
||||
args: ['-y', '@anthropic/mcp-server-filesystem'],
|
||||
});
|
||||
const client = new AxMCPClient(transport);
|
||||
```
|
||||
|
||||
## Critical Rules
|
||||
|
||||
- Use `ai()` factory for all providers.
|
||||
- Provider names: `'openai'`, `'anthropic'`, `'google-gemini'`, `'azure-openai'`, `'mistral'`, `'groq'`, `'cohere'`, `'together'`, `'deepseek'`, `'ollama'`, `'huggingface'`, `'openrouter'`, `'reka'`, `'x-grok'`
|
||||
- Thinking constraints on Anthropic: `temperature` and `topK` are ignored; `topP` only sent if >= 0.95.
|
||||
- Bedrock uses `new AxAIBedrock()`, not `ai()`.
|
||||
- Vercel AI SDK uses `AxAIProvider` wrapper.
|
||||
|
||||
## Examples
|
||||
|
||||
Fetch these for full working code:
|
||||
|
||||
- [Embeddings](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/embed.ts) — embedding generation
|
||||
- [Anthropic Thinking](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/anthropic-thinking-function.ts) — extended thinking with functions
|
||||
- [Anthropic Thinking Separation](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/anthropic-thinking-separation.ts) — thinking separation
|
||||
- [Anthropic Web Search](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/anthropic-web-search.ts) — Anthropic web search
|
||||
- [OpenAI Web Search](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/openai-web-search.ts) — OpenAI web search
|
||||
- [OpenAI Responses](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/openai-responses.ts) — OpenAI responses API
|
||||
- [o3 Reasoning](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/reasoning-o3-example.ts) — o3 reasoning
|
||||
- [Gemini Context Cache](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/gemini-context-cache.ts) — Gemini context caching
|
||||
- [Gemini Files](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/gemini-file-support.ts) — Gemini file handling
|
||||
- [Grok Live Search](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/grok-live-search.ts) — Grok live search
|
||||
- [OpenRouter](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/openrouter.ts) — OpenRouter provider
|
||||
- [Vertex AI Auth](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/vertex-auth-example.ts) — Vertex AI authentication
|
||||
- [MCP Stdio](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/mcp-client-memory.ts) — MCP stdio transport
|
||||
- [MCP HTTP](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/mcp-client-pipedream.ts) — MCP HTTP transport
|
||||
- [Telemetry](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/telemetry.ts) — OpenTelemetry tracing
|
||||
- [Multi-Modal](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/multi-modal.ts) — image handling
|
||||
|
||||
## Do Not Generate
|
||||
|
||||
- Do not use `new AxAIOpenAI(...)` or similar class constructors for standard providers; use `ai()`.
|
||||
- Do not hardcode provider class names when `ai({ name: ... })` covers the provider.
|
||||
- Do not mix `thinkingTokenBudget` with explicit `temperature` on Anthropic thinking models.
|
||||
- Do not use `ai()` for AWS Bedrock; use `new AxAIBedrock()`.
|
||||
- Do not omit `resourceName` and `deploymentName` for Azure OpenAI.
|
||||
402
.agents/skills/ax-flow/SKILL.md
Normal file
402
.agents/skills/ax-flow/SKILL.md
Normal file
@@ -0,0 +1,402 @@
|
||||
---
|
||||
name: ax-flow
|
||||
description: This skill helps an LLM generate correct AxFlow workflow code using @ax-llm/ax. Use when the user asks about flow(), AxFlow, workflow orchestration, parallel execution, DAG workflows, conditional routing, map/reduce patterns, or multi-node AI pipelines.
|
||||
version: "19.0.33"
|
||||
---
|
||||
|
||||
# AxFlow Codegen Rules (@ax-llm/ax)
|
||||
|
||||
Use this skill to generate `AxFlow` workflow code. Prefer short, modern, copyable patterns. Do not write tutorial prose unless the user explicitly asks for explanation.
|
||||
|
||||
## Use These Defaults
|
||||
|
||||
- Use `flow()` factory, not `new AxFlow()`.
|
||||
- Import: `import { ai, flow, f } from '@ax-llm/ax';`
|
||||
- `autoParallel: true` is the default; independent executes run in parallel automatically.
|
||||
- Node results are stored as `${nodeName}Result` in state.
|
||||
- Always define `.node()` before `.execute()` for that node.
|
||||
- Use `.returns()` (or `.r()`) as the last step to lock the output type.
|
||||
- Use descriptive node names: `documentSummarizer`, not `proc1`.
|
||||
- Use descriptive field names: `userInput`, `responseText`, not `text`, `result`.
|
||||
|
||||
## Critical Rules
|
||||
|
||||
- Use `flow()` factory syntax for new code.
|
||||
- Node results in state follow the pattern `state.${nodeName}Result.${fieldName}`.
|
||||
- `.execute()` maps current state to node inputs; `.map()` transforms state without AI calls.
|
||||
- `.returns()` maps final state to the flow output type.
|
||||
- Always define nodes before executing them; reversed order throws at runtime.
|
||||
- Keep state flat; avoid deep nesting in `.map()`.
|
||||
- Ensure loop conditions can change to avoid infinite loops.
|
||||
- Structure independent executes to maximize auto-parallelization.
|
||||
- Use `flow<InputType, OutputType>()` for typed flows.
|
||||
- Aliases: `.n()` = `.node()`, `.nx()` = `.nodeExtended()`, `.m()` = `.map()`, `.r()` = `.returns()`.
|
||||
|
||||
## Canonical Pattern
|
||||
|
||||
```typescript
|
||||
import { ai, flow } from '@ax-llm/ax';
|
||||
|
||||
const llm = ai({ name: 'openai', apiKey: process.env.OPENAI_APIKEY! });
|
||||
|
||||
const wf = flow<{ userInput: string }, { responseText: string }>()
|
||||
.node('testNode', 'userInput:string -> responseText:string')
|
||||
.execute('testNode', (state) => ({ userInput: state.userInput }))
|
||||
.returns((state) => ({ responseText: state.testNodeResult.responseText }));
|
||||
|
||||
const result = await wf.forward(llm, { userInput: 'Hello world' });
|
||||
console.log(result.responseText);
|
||||
```
|
||||
|
||||
## Factory Options
|
||||
|
||||
```typescript
|
||||
// Basic
|
||||
const wf = flow();
|
||||
|
||||
// With options
|
||||
const wf = flow({ autoParallel: false });
|
||||
|
||||
// Typed
|
||||
const wf = flow<InputType, OutputType>();
|
||||
|
||||
// Typed with options
|
||||
const wf = flow<InputType, OutputType>({ autoParallel: true, batchSize: 5 });
|
||||
```
|
||||
|
||||
## State Evolution
|
||||
|
||||
State grows with each executed node. Results are stored as `${nodeName}Result`:
|
||||
|
||||
```typescript
|
||||
// Initial state: { userInput: 'Hello' }
|
||||
flow.execute('processor', (state) => ({ input: state.userInput }));
|
||||
// State: { userInput: 'Hello', processorResult: { output: '...' } }
|
||||
|
||||
flow.execute('analyzer', (state) => ({ text: state.processorResult.output }));
|
||||
// State: { ..., analyzerResult: { sentiment: '...', confidence: 0.8 } }
|
||||
```
|
||||
|
||||
## Node Definition
|
||||
|
||||
```typescript
|
||||
// String signature (creates AxGen automatically)
|
||||
flow.node('processor', 'input:string -> output:string');
|
||||
|
||||
// Multiple outputs
|
||||
flow.node('analyzer', 'text:string -> sentiment:string, confidence:number');
|
||||
|
||||
// Array outputs
|
||||
flow.node('extractor', 'documentText:string -> entities:string[]');
|
||||
|
||||
// Short alias
|
||||
flow.n('processor', 'input:string -> output:string');
|
||||
```
|
||||
|
||||
## Extended Nodes (nx)
|
||||
|
||||
Add fields to a base signature without rewriting it:
|
||||
|
||||
```typescript
|
||||
import { f, flow } from '@ax-llm/ax';
|
||||
|
||||
// Chain-of-thought reasoning
|
||||
flow.nx('reasoner', 'question:string -> answer:string', {
|
||||
prependOutputs: [
|
||||
{ name: 'reasoning', type: f.internal(f.string('Step-by-step reasoning')) },
|
||||
],
|
||||
});
|
||||
|
||||
// Add confidence scoring
|
||||
flow.nx('analyzer', 'input:string -> result:string', {
|
||||
appendOutputs: [{ name: 'confidence', type: f.number('Confidence 0-1') }],
|
||||
});
|
||||
|
||||
// Add optional context input
|
||||
flow.nx('processor', 'query:string -> response:string', {
|
||||
appendInputs: [{ name: 'context', type: f.optional(f.string('Extra context')) }],
|
||||
});
|
||||
```
|
||||
|
||||
Extension options: `prependInputs`, `appendInputs`, `prependOutputs`, `appendOutputs`.
|
||||
|
||||
## Execute With Input Mapping
|
||||
|
||||
```typescript
|
||||
flow.execute('summarizer', (state) => ({ documentText: state.document }));
|
||||
|
||||
// With AI override (use a different model for this node)
|
||||
flow.execute('processor', (state) => ({ input: state.data }), { ai: alternativeAI });
|
||||
```
|
||||
|
||||
## Map (State Transformation)
|
||||
|
||||
Use `map()` for data shaping without AI calls:
|
||||
|
||||
```typescript
|
||||
// Sync
|
||||
flow.map((state) => ({ ...state, upperText: state.rawText.toUpperCase() }));
|
||||
|
||||
// Async
|
||||
flow.map(async (state) => {
|
||||
const data = await fetchFromAPI(state.query);
|
||||
return { ...state, enrichedData: data };
|
||||
});
|
||||
|
||||
// Parallel async transforms
|
||||
flow.map([
|
||||
async (state) => ({ ...state, result1: await api1(state.data) }),
|
||||
async (state) => ({ ...state, result2: await api2(state.data) }),
|
||||
], { parallel: true });
|
||||
```
|
||||
|
||||
## Returns (Final Output)
|
||||
|
||||
```typescript
|
||||
const wf = flow<{ input: string }>()
|
||||
.map((state) => ({ ...state, upper: state.input.toUpperCase(), len: state.input.length }))
|
||||
.returns((state) => ({ upper: state.upper, isLong: state.len > 20 }));
|
||||
|
||||
// Result is typed as { upper: string; isLong: boolean }
|
||||
const result = await wf.forward(llm, { input: 'test' });
|
||||
```
|
||||
|
||||
## Sequential Processing
|
||||
|
||||
```typescript
|
||||
const wf = flow<{ input: string }, { finalResult: string }>()
|
||||
.node('step1', 'input:string -> intermediate:string')
|
||||
.node('step2', 'intermediate:string -> output:string')
|
||||
.execute('step1', (state) => ({ input: state.input }))
|
||||
.execute('step2', (state) => ({ intermediate: state.step1Result.intermediate }))
|
||||
.returns((state) => ({ finalResult: state.step2Result.output }));
|
||||
```
|
||||
|
||||
## Auto-Parallel Execution
|
||||
|
||||
Independent executes run in parallel automatically (`autoParallel: true` by default):
|
||||
|
||||
```typescript
|
||||
const wf = flow<{ text: string }, { combined: string }>()
|
||||
.node('sentimentAnalyzer', 'text:string -> sentiment:string')
|
||||
.node('topicExtractor', 'text:string -> topics:string[]')
|
||||
.node('entityRecognizer', 'text:string -> entities:string[]')
|
||||
// These three run in parallel (all depend only on state.text)
|
||||
.execute('sentimentAnalyzer', (state) => ({ text: state.text }))
|
||||
.execute('topicExtractor', (state) => ({ text: state.text }))
|
||||
.execute('entityRecognizer', (state) => ({ text: state.text }))
|
||||
// This waits for all three
|
||||
.returns((state) => ({
|
||||
combined: JSON.stringify({
|
||||
sentiment: state.sentimentAnalyzerResult.sentiment,
|
||||
topics: state.topicExtractorResult.topics,
|
||||
entities: state.entityRecognizerResult.entities,
|
||||
}),
|
||||
}));
|
||||
|
||||
// Inspect execution plan
|
||||
const plan = wf.getExecutionPlan();
|
||||
console.log(plan.parallelGroups, plan.maxParallelism);
|
||||
```
|
||||
|
||||
Disable auto-parallel:
|
||||
|
||||
```typescript
|
||||
const wf = flow({ autoParallel: false });
|
||||
// or per execution:
|
||||
await wf.forward(llm, input, { autoParallel: false });
|
||||
```
|
||||
|
||||
## Conditional Branching
|
||||
|
||||
```typescript
|
||||
const wf = flow<{ query: string; expertMode: boolean }, { response: string }>()
|
||||
.node('simple', 'query:string -> response:string')
|
||||
.node('expert', 'query:string -> response:string')
|
||||
.branch((state) => state.expertMode)
|
||||
.when(true)
|
||||
.execute('expert', (state) => ({ query: state.query }))
|
||||
.when(false)
|
||||
.execute('simple', (state) => ({ query: state.query }))
|
||||
.merge()
|
||||
.returns((state) => ({
|
||||
response: state.expertResult?.response ?? state.simpleResult?.response,
|
||||
}));
|
||||
```
|
||||
|
||||
After `.merge()`, only the taken branch's result exists; use optional chaining (`?.`) on untaken branch results.
|
||||
|
||||
## While Loops
|
||||
|
||||
```typescript
|
||||
const wf = flow<{ content: string }, { finalContent: string }>()
|
||||
.node('processor', 'content:string -> processedContent:string')
|
||||
.node('qualityChecker', 'content:string -> qualityScore:number')
|
||||
.map((state) => ({ currentContent: state.content, iteration: 0, qualityScore: 0 }))
|
||||
.while((state) => state.iteration < 3 && state.qualityScore < 0.8)
|
||||
.map((state) => ({ ...state, iteration: state.iteration + 1 }))
|
||||
.execute('processor', (state) => ({ content: state.currentContent }))
|
||||
.execute('qualityChecker', (state) => ({
|
||||
content: state.processorResult.processedContent,
|
||||
}))
|
||||
.map((state) => ({
|
||||
...state,
|
||||
currentContent: state.processorResult.processedContent,
|
||||
qualityScore: state.qualityCheckerResult.qualityScore,
|
||||
}))
|
||||
.endWhile()
|
||||
.returns((state) => ({ finalContent: state.currentContent }));
|
||||
```
|
||||
|
||||
Rules:
|
||||
- Every `.while()` needs a matching `.endWhile()`.
|
||||
- Ensure the loop condition can change to avoid infinite loops.
|
||||
|
||||
## Feedback Loops (label/feedback)
|
||||
|
||||
```typescript
|
||||
const wf = flow<{ prompt: string }, { result: string }>()
|
||||
.node('gen', 'prompt:string -> result:string, quality:number')
|
||||
.map((state) => ({ ...state, tries: 0 }))
|
||||
.label('retry')
|
||||
.map((state) => ({ ...state, tries: state.tries + 1 }))
|
||||
.execute('gen', (state) => ({ prompt: state.prompt }))
|
||||
.feedback((state) => state.genResult.quality < 0.9 && state.tries < 3, 'retry')
|
||||
.returns((state) => ({ result: state.genResult.result }));
|
||||
```
|
||||
|
||||
Rules:
|
||||
- Define the label before referencing it in `.feedback()`.
|
||||
- Always include a max-iteration guard to avoid infinite loops.
|
||||
|
||||
## Explicit Parallel Sub-Flows
|
||||
|
||||
```typescript
|
||||
flow
|
||||
.parallel([
|
||||
(sub) => sub.execute('analyzer1', (state) => ({ text: state.input })),
|
||||
(sub) => sub.execute('analyzer2', (state) => ({ text: state.input })),
|
||||
(sub) => sub.execute('analyzer3', (state) => ({ text: state.input })),
|
||||
])
|
||||
.merge('combinedResults', (r1, r2, r3) => ({
|
||||
a1: r1.analyzer1Result.analysis,
|
||||
a2: r2.analyzer2Result.analysis,
|
||||
a3: r3.analyzer3Result.analysis,
|
||||
}));
|
||||
```
|
||||
|
||||
## Derive (Batch/Array Processing)
|
||||
|
||||
```typescript
|
||||
const wf = flow<{ items: string[] }, { processed: string[] }>({ batchSize: 3 })
|
||||
.derive('processed', 'items', (item, index) => `processed-${item}-${index}`, {
|
||||
batchSize: 2,
|
||||
});
|
||||
```
|
||||
|
||||
## Dynamic AI Context (Multi-Model)
|
||||
|
||||
Route nodes to different AI providers:
|
||||
|
||||
```typescript
|
||||
const fast = ai({ name: 'groq', apiKey: '...' });
|
||||
const smart = ai({ name: 'anthropic', apiKey: '...' });
|
||||
|
||||
const wf = flow<{ text: string }, { out: string }>()
|
||||
.node('draft', 'text:string -> out:string')
|
||||
.node('refine', 'text:string -> out:string')
|
||||
.execute('draft', (state) => ({ text: state.text }), { ai: fast })
|
||||
.execute('refine', (state) => ({ text: state.draftResult.out }), { ai: smart })
|
||||
.returns((state) => ({ out: state.refineResult.out }));
|
||||
```
|
||||
|
||||
## Description and toFunction
|
||||
|
||||
```typescript
|
||||
const wf = flow<{ userQuestion: string }, { responseText: string }>()
|
||||
.node('qa', 'userQuestion:string -> responseText:string')
|
||||
.execute('qa', (state) => ({ userQuestion: state.userQuestion }))
|
||||
.returns((state) => ({ responseText: state.qaResult.responseText }))
|
||||
.description('Question Answerer', 'Answers user questions concisely.');
|
||||
|
||||
const fn = wf.toFunction();
|
||||
// fn.name, fn.parameters (JSON Schema), fn.func
|
||||
```
|
||||
|
||||
## Instrumentation (Tracing)
|
||||
|
||||
```typescript
|
||||
import { ai, flow } from '@ax-llm/ax';
|
||||
import { context, trace } from '@opentelemetry/api';
|
||||
|
||||
const tracer = trace.getTracer('axflow');
|
||||
const llm = ai({ name: 'openai', apiKey: '...' });
|
||||
|
||||
const wf = flow<{ userQuestion: string }>()
|
||||
.node('summarizer', 'documentText:string -> summaryText:string')
|
||||
.execute('summarizer', (s) => ({ documentText: s.userQuestion }))
|
||||
.returns((s) => ({ answer: s.summarizerResult.summaryText }));
|
||||
|
||||
const result = await wf.forward(llm, { userQuestion: 'hi' }, {
|
||||
tracer,
|
||||
traceContext: context.active(),
|
||||
});
|
||||
```
|
||||
|
||||
## Program IDs and Demos
|
||||
|
||||
```typescript
|
||||
const wf = flow<{ input: string }>()
|
||||
.node('summarizer', 'text:string -> summary:string')
|
||||
.node('classifier', 'text:string -> category:string');
|
||||
|
||||
// Discover program IDs
|
||||
console.log(wf.namedPrograms());
|
||||
// [{ id: 'root.summarizer', ... }, { id: 'root.classifier', ... }]
|
||||
|
||||
// Set demos (TypeScript catches typos)
|
||||
wf.setDemos([{ programId: 'root.summarizer', traces: [] }]);
|
||||
|
||||
// Apply optimization
|
||||
wf.applyOptimization(optimizedProgram);
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
```typescript
|
||||
try {
|
||||
const result = await wf.forward(llm, input);
|
||||
} catch (error) {
|
||||
console.error('Flow execution failed:', error);
|
||||
}
|
||||
```
|
||||
|
||||
Common errors:
|
||||
- `"Node 'x' not found"` -- define `.node()` before `.execute()`.
|
||||
- `"endWhile() without matching while()"` -- every `.while()` needs `.endWhile()`.
|
||||
- `"when() without matching branch()"` -- `.when()` must be inside `.branch()`/`.merge()`.
|
||||
- `"merge() without matching branch()"` -- every `.branch()` needs `.merge()`.
|
||||
- `"Label 'x' not found"` -- define `.label()` before `.feedback()` references it.
|
||||
|
||||
## Examples
|
||||
|
||||
Fetch these for full working code:
|
||||
|
||||
- [Flow](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/ax-flow.ts) — complete flow usage
|
||||
- [Auto-Parallel](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/ax-flow-auto-parallel.ts) — auto-parallelization
|
||||
- [Async Map](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/ax-flow-async-map.ts) — async map transforms
|
||||
- [Enhanced Demo](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/ax-flow-enhanced-demo.ts) — instance-based nodes
|
||||
- [Flow as Function](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/ax-flow-to-function.ts) — flow as callable function
|
||||
- [Fluent Builder](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/fluent-flow-example.ts) — fluent builder pattern
|
||||
- [Flow Logging](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/flow-logging-simple.ts) — flow logging
|
||||
- [Load Balancing](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/balancer.ts) — load balancing
|
||||
|
||||
## Do Not Generate
|
||||
|
||||
- Do not use `new AxFlow(...)` for new code.
|
||||
- Do not execute a node before defining it with `.node()`.
|
||||
- Do not use generic field names like `text`, `result`, `data`, `input`, `output`.
|
||||
- Do not create deep-nested state objects in `.map()`.
|
||||
- Do not create loop conditions that can never change.
|
||||
- Do not add unnecessary dependencies between executes (kills auto-parallelism).
|
||||
- Do not forget to use optional chaining on branch results after `.merge()`.
|
||||
323
.agents/skills/ax-gen/SKILL.md
Normal file
323
.agents/skills/ax-gen/SKILL.md
Normal file
@@ -0,0 +1,323 @@
|
||||
---
|
||||
name: ax-gen
|
||||
description: This skill helps an LLM generate correct AxGen code using @ax-llm/ax. Use when the user asks about ax(), AxGen, generators, forward(), streamingForward(), assertions, field processors, step hooks, self-tuning, or structured outputs.
|
||||
version: "19.0.33"
|
||||
---
|
||||
|
||||
# AxGen Codegen Rules (@ax-llm/ax)
|
||||
|
||||
Use this skill to generate `AxGen` code. Prefer short, modern, copyable patterns. Do not write tutorial prose unless the user explicitly asks for explanation.
|
||||
|
||||
## Use These Defaults
|
||||
|
||||
- Use `ax(...)` factory, not `new AxGen(...)`.
|
||||
- Always pass an AI instance from `ai(...)` as the first argument to `forward()`.
|
||||
- Streaming uses `streamingForward()`, not `forward()` with a stream option.
|
||||
- Assertions auto-retry with error feedback on failure.
|
||||
- Step hook mutations are applied at the next step boundary (pending pattern).
|
||||
- `stopFunction` accepts a string or string[] for multiple stop functions.
|
||||
- Multi-step continues until: all outputs filled, stop function called, or `maxSteps` reached.
|
||||
|
||||
## Canonical Pattern
|
||||
|
||||
```typescript
|
||||
import { ai, ax, s } from '@ax-llm/ax';
|
||||
|
||||
const llm = ai({
|
||||
name: 'openai',
|
||||
apiKey: process.env.OPENAI_APIKEY!,
|
||||
});
|
||||
|
||||
// Inline signature
|
||||
const gen = ax('input:string -> output:string, reasoning:string');
|
||||
|
||||
// Reusable signature
|
||||
const sig = s('question:string, context:string[] -> answer:string');
|
||||
const gen2 = ax(sig);
|
||||
|
||||
// With options
|
||||
const gen3 = ax('input -> output', {
|
||||
description: 'A helpful assistant',
|
||||
maxRetries: 3,
|
||||
maxSteps: 10,
|
||||
temperature: 0.7,
|
||||
});
|
||||
|
||||
const result = await gen.forward(llm, { input: 'Hello world' });
|
||||
console.log(result.output);
|
||||
```
|
||||
|
||||
## Running AxGen
|
||||
|
||||
### `forward()`
|
||||
|
||||
```typescript
|
||||
const result = await gen.forward(llm, { input: '...' });
|
||||
|
||||
// With options
|
||||
const result = await gen.forward(llm, { input: '...' }, {
|
||||
maxRetries: 5,
|
||||
model: 'gpt-4.1',
|
||||
modelConfig: { temperature: 0.9, maxTokens: 1000 },
|
||||
debug: true,
|
||||
});
|
||||
```
|
||||
|
||||
### `streamingForward()`
|
||||
|
||||
```typescript
|
||||
const stream = gen.streamingForward(llm, { input: 'Write a long story' });
|
||||
for await (const chunk of stream) {
|
||||
if (chunk.delta.output) process.stdout.write(chunk.delta.output);
|
||||
}
|
||||
```
|
||||
|
||||
## Stopping And Cancellation
|
||||
|
||||
```typescript
|
||||
import { AxAIServiceAbortedError } from '@ax-llm/ax';
|
||||
|
||||
const timer = setTimeout(() => gen.stop(), 3_000);
|
||||
|
||||
try {
|
||||
const result = await gen.forward(llm, { topic: 'Long document' }, {
|
||||
abortSignal: AbortSignal.timeout(10_000),
|
||||
});
|
||||
} catch (err) {
|
||||
if (err instanceof AxAIServiceAbortedError) console.log('Aborted');
|
||||
}
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- `gen.stop()` gracefully stops multi-step execution at the next step boundary.
|
||||
- `abortSignal` cancels the underlying AI service call immediately.
|
||||
- Catch `AxAIServiceAbortedError` when using either mechanism.
|
||||
|
||||
## Assertions And Validation
|
||||
|
||||
```typescript
|
||||
// Standard assertion (checked after forward completes)
|
||||
gen.addAssert(
|
||||
(args) => args.output.length > 50,
|
||||
'Output must be at least 50 characters'
|
||||
);
|
||||
|
||||
// Streaming assertion (checked during streaming)
|
||||
gen.addStreamingAssert(
|
||||
'output',
|
||||
(text) => !text.includes('forbidden'),
|
||||
'Output contains forbidden text'
|
||||
);
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- Failed assertions cause an automatic retry with the error message fed back to the LLM.
|
||||
- `addAssert` receives the full output object.
|
||||
- `addStreamingAssert` targets a specific field and receives the partial text so far.
|
||||
|
||||
## Field Processors
|
||||
|
||||
```typescript
|
||||
// Post-processing after generation
|
||||
gen.addFieldProcessor('summary', (value, context) => value.toUpperCase());
|
||||
|
||||
// Streaming field processor (called on each chunk)
|
||||
gen.addStreamingFieldProcessor('content', (partialValue, context) => {
|
||||
console.log(`Received ${partialValue.length} chars`);
|
||||
return partialValue;
|
||||
});
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- `addFieldProcessor` runs once after the field is fully generated.
|
||||
- `addStreamingFieldProcessor` runs on each streaming chunk for the target field.
|
||||
- Both must return the (possibly transformed) value.
|
||||
|
||||
## Function Calling
|
||||
|
||||
```typescript
|
||||
const result = await gen.forward(llm, { question: '...' }, {
|
||||
functions: tools,
|
||||
functionCallMode: 'auto',
|
||||
stopFunction: 'finalAnswer',
|
||||
});
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- `functionCallMode` can be `'auto'`, `'none'`, or a specific function name to force.
|
||||
- `stopFunction` accepts a string or string[] to halt multi-step on specific function calls.
|
||||
- Multi-step continues until all outputs filled, stop function called, or `maxSteps` reached.
|
||||
|
||||
## Caching
|
||||
|
||||
### Response Caching
|
||||
|
||||
```typescript
|
||||
const gen = ax('question:string -> answer:string', {
|
||||
cachingFunction: async (key, value?) => {
|
||||
if (value !== undefined) {
|
||||
await cache.set(key, value);
|
||||
return;
|
||||
}
|
||||
return await cache.get(key);
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
### Context Caching
|
||||
|
||||
```typescript
|
||||
const result = await gen.forward(llm, { question: '...' }, {
|
||||
contextCache: { cacheBreakpoint: 'after-examples' },
|
||||
});
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- `cachingFunction` acts as a get/set: called with `(key)` to read, `(key, value)` to write.
|
||||
- `contextCache` enables AI provider-level prompt caching for long context.
|
||||
|
||||
## Sampling And Result Picker
|
||||
|
||||
```typescript
|
||||
const result = await gen.forward(llm, { question: '...' }, {
|
||||
sampleCount: 3,
|
||||
resultPicker: async (samples) => {
|
||||
// Evaluate each sample and return the index of the best one
|
||||
return bestIndex;
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- `sampleCount` generates multiple completions in parallel.
|
||||
- `resultPicker` receives all samples and must return the index of the chosen result.
|
||||
|
||||
## Extended Thinking
|
||||
|
||||
```typescript
|
||||
const result = await gen.forward(llm, { question: '...' }, {
|
||||
thinkingTokenBudget: 'medium',
|
||||
showThoughts: true,
|
||||
});
|
||||
console.log(result.thought);
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- `thinkingTokenBudget` can be `'low'`, `'medium'`, `'high'`, or a number.
|
||||
- Set `showThoughts: true` to include the model's reasoning in `result.thought`.
|
||||
|
||||
## Step Hooks
|
||||
|
||||
```typescript
|
||||
const result = await gen.forward(llm, values, {
|
||||
stepHooks: {
|
||||
beforeStep: (ctx) => {
|
||||
if (ctx.functionsExecuted.has('complexanalysis')) {
|
||||
ctx.setModel('smart');
|
||||
ctx.setThinkingBudget('high');
|
||||
}
|
||||
},
|
||||
afterStep: (ctx) => {
|
||||
console.log(`Usage: ${ctx.usage.totalTokens} tokens`);
|
||||
},
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
### AxStepContext Read-Only Properties
|
||||
|
||||
- `stepIndex` - current step number
|
||||
- `maxSteps` - configured maximum steps
|
||||
- `isFirstStep` - whether this is the first step
|
||||
- `functionsExecuted` - `Set<string>` of function names called so far
|
||||
- `lastFunctionCalls` - array of the most recent function call results
|
||||
- `usage` - token usage statistics
|
||||
- `state` - current step state
|
||||
|
||||
### AxStepContext Mutators
|
||||
|
||||
- `setModel(model)` - change the model for the next step
|
||||
- `setThinkingBudget(budget)` - adjust thinking budget
|
||||
- `setTemperature(temp)` - adjust temperature
|
||||
- `setMaxTokens(max)` - adjust max output tokens
|
||||
- `setOptions(opts)` - set arbitrary forward options
|
||||
- `addFunctions(fns)` - add functions for the next step
|
||||
- `removeFunctions(names)` - remove functions by name
|
||||
- `stop()` - stop multi-step execution
|
||||
|
||||
Rules:
|
||||
|
||||
- All mutations are pending and applied at the next step boundary.
|
||||
- `beforeStep` runs before each LLM call; `afterStep` runs after.
|
||||
- Use `afterFunctionExecution` to react to specific function results.
|
||||
|
||||
## Self-Tuning
|
||||
|
||||
```typescript
|
||||
// Simple: enable all self-tuning
|
||||
const result = await gen.forward(llm, values, { selfTuning: true });
|
||||
|
||||
// Granular: pick what to tune
|
||||
const result = await gen.forward(llm, values, {
|
||||
selfTuning: {
|
||||
model: true,
|
||||
thinkingBudget: true,
|
||||
functions: [searchWeb, calculate],
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- `selfTuning: true` enables automatic model and parameter selection.
|
||||
- Granular config allows tuning specific aspects independently.
|
||||
- `selfTuning.functions` provides a pool of functions the tuner may add or remove per step.
|
||||
|
||||
## Error Handling
|
||||
|
||||
```typescript
|
||||
import { AxGenerateError } from '@ax-llm/ax';
|
||||
|
||||
try {
|
||||
const result = await gen.forward(llm, { input: '...' });
|
||||
} catch (error) {
|
||||
if (error instanceof AxGenerateError) {
|
||||
console.log(error.details.model, error.details.signature);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- `AxGenerateError` includes `details` with `model` and `signature` for debugging.
|
||||
- `AxAIServiceAbortedError` is thrown on cancellation via `stop()` or `abortSignal`.
|
||||
|
||||
## Examples
|
||||
|
||||
Fetch these for full working code:
|
||||
|
||||
- [Streaming](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/streaming.ts) — streaming with assertions
|
||||
- [Assertions](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/asserts.ts) — output validation
|
||||
- [Streaming Assertions](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/streaming-asserts.ts) — streaming with assertion checks
|
||||
- [Structured Output](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/structured_output.ts) — fluent API with validation
|
||||
- [Debug Logging](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/debug-logging.ts) — debug mode and step hooks
|
||||
- [Stop Function](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/stop-function.ts) — stop functions
|
||||
- [Fibonacci](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/fibonacci.ts) — streaming with thinking
|
||||
- [Extraction](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/extract.ts) — information extraction
|
||||
- [Multi-Sampling](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/sample-count.ts) — sample count usage
|
||||
|
||||
## Do Not Generate
|
||||
|
||||
- Do not use `new AxGen(...)` for new code unless explicitly required.
|
||||
- Do not pass raw API keys or config objects where an `ai(...)` instance is expected.
|
||||
- Do not use `forward()` for streaming; use `streamingForward()`.
|
||||
- Do not forget that assertions auto-retry; avoid manual retry loops around assertion logic.
|
||||
- Do not mutate step hook context expecting immediate effect; mutations are pending until the next step.
|
||||
- Do not assume multi-step stops after one LLM call; it continues until outputs are filled, a stop function fires, or `maxSteps` is reached.
|
||||
260
.agents/skills/ax-gepa/SKILL.md
Normal file
260
.agents/skills/ax-gepa/SKILL.md
Normal file
@@ -0,0 +1,260 @@
|
||||
---
|
||||
name: ax-gepa
|
||||
description: This skill helps an LLM generate correct AxGEPA optimization code using @ax-llm/ax. Use when the user asks about AxGEPA, GEPA, Pareto optimization, multi-objective prompt tuning, reflective prompt evolution, validationExamples, maxMetricCalls, or optimizing a generator, flow, or agent tree.
|
||||
version: "19.0.33"
|
||||
---
|
||||
|
||||
# AxGEPA Codegen Rules (@ax-llm/ax)
|
||||
|
||||
Use this skill to generate direct `AxGEPA` optimization code. Prefer short, modern, copyable patterns over long explanation.
|
||||
|
||||
## Use These Defaults
|
||||
|
||||
- Use `new AxGEPA({ studentAI, teacherAI, ... })`.
|
||||
- Prefer `ai()`, `ax()`, and `flow()` for new code.
|
||||
- Use a strong `teacherAI` and a cheaper `studentAI`.
|
||||
- Always pass `validationExamples` to `compile()`.
|
||||
- Always set `maxMetricCalls` to bound optimizer cost.
|
||||
- Use scalar metrics for one objective and object metrics for Pareto optimization.
|
||||
- Apply results with `program.applyOptimization(result.optimizedProgram!)`.
|
||||
- For tree-wide runs, expect `optimizedProgram.instructionMap`.
|
||||
|
||||
## Critical Rules
|
||||
|
||||
- `AxGEPA.compile()` works for a single generator and for tree-aware roots such as flows or agents with registered instruction-bearing descendants.
|
||||
- There is no separate flow-only GEPA optimizer. Use `AxGEPA` for flows too.
|
||||
- The metric may return either `number` or `Record<string, number>`.
|
||||
- Keep metrics deterministic and cheap by default.
|
||||
- Avoid extra LLM calls inside the metric unless the user explicitly wants judge-based evaluation.
|
||||
- If the user needs LLM-as-judge scoring for a non-agent GEPA run, prefer a plain typed `AxGen` evaluator instead of writing a custom judge abstraction.
|
||||
- `maxMetricCalls` must be large enough to cover the initial validation pass over `validationExamples`.
|
||||
- GEPA optimizes instructions. If a tree has no instruction-bearing nodes, optimization will fail.
|
||||
- Use held-out validation examples for selection. Do not reuse the training set as `validationExamples`.
|
||||
- `result.optimizedProgram` is the easy-to-apply best candidate. `result.paretoFront` is the full trade-off set for multi-objective runs.
|
||||
|
||||
## Metric Selection
|
||||
|
||||
Choose the evaluation path deliberately:
|
||||
|
||||
- Prefer a deterministic metric when correctness can be read directly from `prediction` and `example`.
|
||||
- Prefer a deterministic metric when cost, latency, recursion depth, or tool count matters.
|
||||
- Use a plain typed `AxGen` evaluator only when the task is genuinely qualitative and hard to score exactly.
|
||||
- For `agent.optimize(...)`, prefer the built-in judge path instead of manually wrapping a judge metric.
|
||||
|
||||
Rule of thumb:
|
||||
|
||||
- `AxGEPA` on `AxGen` or flow: use a metric first, optionally a plain typed `AxGen` evaluator if needed.
|
||||
- `agent.optimize(...)`: use custom `metric` for crisp scoring, otherwise `judgeAI` plus `judgeOptions`.
|
||||
|
||||
## Canonical Scalar Pattern
|
||||
|
||||
```typescript
|
||||
import { ai, ax, AxAIOpenAIModel, AxGEPA } from '@ax-llm/ax';
|
||||
|
||||
const student = ai({
|
||||
name: 'openai',
|
||||
apiKey: process.env.OPENAI_APIKEY!,
|
||||
config: { model: AxAIOpenAIModel.GPT4OMini },
|
||||
});
|
||||
|
||||
const teacher = ai({
|
||||
name: 'openai',
|
||||
apiKey: process.env.OPENAI_APIKEY!,
|
||||
config: { model: AxAIOpenAIModel.GPT4O },
|
||||
});
|
||||
|
||||
const classifier = ax(
|
||||
'emailText:string -> priority:class "high, normal, low", rationale:string'
|
||||
);
|
||||
|
||||
const train = [
|
||||
{ emailText: 'URGENT: Server down!', priority: 'high' },
|
||||
{ emailText: 'Weekly newsletter', priority: 'low' },
|
||||
];
|
||||
|
||||
const validation = [
|
||||
{ emailText: 'Invoice overdue', priority: 'high' },
|
||||
{ emailText: 'Lunch plans?', priority: 'low' },
|
||||
];
|
||||
|
||||
const metric = ({ prediction, example }: { prediction: any; example: any }) =>
|
||||
prediction?.priority === example?.priority ? 1 : 0;
|
||||
|
||||
const optimizer = new AxGEPA({
|
||||
studentAI: student,
|
||||
teacherAI: teacher,
|
||||
numTrials: 12,
|
||||
minibatch: true,
|
||||
minibatchSize: 4,
|
||||
earlyStoppingTrials: 4,
|
||||
sampleCount: 1,
|
||||
});
|
||||
|
||||
const result = await optimizer.compile(classifier, train, metric, {
|
||||
validationExamples: validation,
|
||||
maxMetricCalls: 120,
|
||||
});
|
||||
|
||||
classifier.applyOptimization(result.optimizedProgram!);
|
||||
console.log(result.bestScore);
|
||||
```
|
||||
|
||||
## Canonical Pareto Pattern
|
||||
|
||||
```typescript
|
||||
import { ai, flow, AxAIOpenAIModel, AxGEPA } from '@ax-llm/ax';
|
||||
|
||||
const student = ai({
|
||||
name: 'openai',
|
||||
apiKey: process.env.OPENAI_APIKEY!,
|
||||
config: { model: AxAIOpenAIModel.GPT4OMini },
|
||||
});
|
||||
|
||||
const teacher = ai({
|
||||
name: 'openai',
|
||||
apiKey: process.env.OPENAI_APIKEY!,
|
||||
config: { model: AxAIOpenAIModel.GPT4O },
|
||||
});
|
||||
|
||||
const wf = flow<{ emailText: string }>()
|
||||
.n('classifier', 'emailText:string -> priority:class "high, normal, low"')
|
||||
.n(
|
||||
'rationale',
|
||||
'emailText:string, priority:string -> rationale:string "One concise sentence"'
|
||||
)
|
||||
.e('classifier', (state) => ({ emailText: state.emailText }))
|
||||
.e('rationale', (state) => ({
|
||||
emailText: state.emailText,
|
||||
priority: state.classifierResult.priority,
|
||||
}))
|
||||
.r((state) => ({
|
||||
priority: state.classifierResult.priority,
|
||||
rationale: state.rationaleResult.rationale,
|
||||
}));
|
||||
|
||||
const train = [
|
||||
{ emailText: 'URGENT: Server down!', priority: 'high' },
|
||||
{ emailText: 'Weekly newsletter', priority: 'low' },
|
||||
];
|
||||
|
||||
const validation = [
|
||||
{ emailText: 'Invoice overdue', priority: 'high' },
|
||||
{ emailText: 'Lunch plans?', priority: 'low' },
|
||||
];
|
||||
|
||||
const metric = ({ prediction, example }: { prediction: any; example: any }) => {
|
||||
const accuracy = prediction?.priority === example?.priority ? 1 : 0;
|
||||
const rationale = typeof prediction?.rationale === 'string'
|
||||
? prediction.rationale
|
||||
: '';
|
||||
const brevity = rationale.length <= 40 ? 1 : rationale.length <= 80 ? 0.5 : 0.1;
|
||||
return { accuracy, brevity };
|
||||
};
|
||||
|
||||
const result = await new AxGEPA({
|
||||
studentAI: student,
|
||||
teacherAI: teacher,
|
||||
numTrials: 16,
|
||||
minibatch: true,
|
||||
minibatchSize: 6,
|
||||
earlyStoppingTrials: 5,
|
||||
sampleCount: 1,
|
||||
}).compile(wf, train, metric, {
|
||||
validationExamples: validation,
|
||||
maxMetricCalls: 240,
|
||||
});
|
||||
|
||||
for (const point of result.paretoFront) {
|
||||
console.log(point.scores, point.configuration);
|
||||
}
|
||||
|
||||
wf.applyOptimization(result.optimizedProgram!);
|
||||
console.log(result.optimizedProgram?.instructionMap);
|
||||
```
|
||||
|
||||
## Metric Patterns
|
||||
|
||||
```typescript
|
||||
// Scalar objective
|
||||
const scalarMetric = ({ prediction, example }) =>
|
||||
prediction.answer === example.answer ? 1 : 0;
|
||||
|
||||
// Multi-objective
|
||||
const multiMetric = ({ prediction, example }) => ({
|
||||
accuracy: prediction.answer === example.answer ? 1 : 0,
|
||||
brevity:
|
||||
typeof prediction?.reasoning === 'string' &&
|
||||
prediction.reasoning.length < 120
|
||||
? 1
|
||||
: 0.2,
|
||||
});
|
||||
```
|
||||
|
||||
- Return plain numbers or plain object literals.
|
||||
- Keep objective names stable across calls.
|
||||
- Prefer normalized scores such as `0..1` so trade-offs are easy to reason about.
|
||||
|
||||
## Result Handling
|
||||
|
||||
```typescript
|
||||
const { optimizedProgram, paretoFront } = result;
|
||||
|
||||
program.applyOptimization(optimizedProgram!);
|
||||
|
||||
// Save for later
|
||||
const saved = JSON.stringify(optimizedProgram);
|
||||
|
||||
// Load later and re-apply
|
||||
const loaded = JSON.parse(saved);
|
||||
program.applyOptimization(loaded);
|
||||
```
|
||||
|
||||
- Single-target runs usually populate both `optimizedProgram.instruction` and `optimizedProgram.instructionMap`.
|
||||
- Tree-wide runs rely on `instructionMap`, keyed by full program ID.
|
||||
- Pareto points expose candidate configs under `point.configuration.instructionMap`.
|
||||
|
||||
## Useful Options
|
||||
|
||||
```typescript
|
||||
const optimizer = new AxGEPA({
|
||||
studentAI,
|
||||
teacherAI,
|
||||
numTrials: 20,
|
||||
minibatch: true,
|
||||
minibatchSize: 5,
|
||||
minibatchFullEvalSteps: 5,
|
||||
earlyStoppingTrials: 5,
|
||||
minImprovementThreshold: 0,
|
||||
sampleCount: 1,
|
||||
seed: 42,
|
||||
verbose: true,
|
||||
});
|
||||
```
|
||||
|
||||
- `numTrials`: number of reflection/evolution rounds.
|
||||
- `minibatch`: reduce per-round evaluation cost.
|
||||
- `minibatchSize`: examples per minibatch.
|
||||
- `earlyStoppingTrials`: stop after repeated non-improvement.
|
||||
- `minImprovementThreshold`: reject tiny gains below this threshold.
|
||||
- `seed`: stabilize sampling during demos and tests.
|
||||
|
||||
## Budgeting and Validation
|
||||
|
||||
- Always create distinct `train` and `validationExamples` arrays.
|
||||
- Size `maxMetricCalls` for at least one full validation pass plus several rounds.
|
||||
- If the user wants a strict budget, say so explicitly and set `maxMetricCalls`.
|
||||
- For expensive trees, start with `auto: 'light'` or fewer `numTrials`, then scale up.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- Error about `maxMetricCalls` being too small: increase it until the initial validation pass fits.
|
||||
- Empty or poor Pareto front: verify the metric returns numbers for every example.
|
||||
- No tree optimization effect: ensure child programs are registered under the root and have instructions to mutate.
|
||||
- Saved optimization applies only partly: use `program.applyOptimization(...)`, not just `setInstruction(...)`, so `instructionMap` reaches the full tree.
|
||||
|
||||
## Good Example Targets
|
||||
|
||||
- `/Users/vr/src/ax/src/examples/gepa.ts`
|
||||
- `/Users/vr/src/ax/src/examples/gepa-flow.ts`
|
||||
- `/Users/vr/src/ax/src/examples/gepa-train-inference.ts`
|
||||
- `/Users/vr/src/ax/src/examples/gepa-quality-vs-speed-optimization.ts`
|
||||
268
.agents/skills/ax-learn/SKILL.md
Normal file
268
.agents/skills/ax-learn/SKILL.md
Normal file
@@ -0,0 +1,268 @@
|
||||
---
|
||||
name: ax-learn
|
||||
description: This skill helps an LLM generate correct AxLearn code using @ax-llm/ax. Use when the user asks about self-improving agents, trace-backed learning, feedback-aware updates, or AxLearn modes.
|
||||
version: "19.0.33"
|
||||
---
|
||||
|
||||
# AxLearn Codegen Rules (@ax-llm/ax)
|
||||
|
||||
Use this skill to generate `AxLearn` code that matches the current API.
|
||||
|
||||
## Core Model
|
||||
|
||||
- `AxLearn` wraps an `AxGen`.
|
||||
- `teacher` is for judging, synthesis, and reflection.
|
||||
- `runtimeAI` is the model being improved.
|
||||
- `forward()` and `streamingForward()` are inference-time APIs and auto-log traces when tracing is enabled.
|
||||
- `optimize()` is offline learning.
|
||||
- `applyUpdate()` is a bounded update API for `continuous` and `playbook` modes.
|
||||
- `ready()` should be awaited before assuming checkpoints have been restored.
|
||||
- `improvement` is the score delta from the previous/restored state.
|
||||
|
||||
## Required Inputs
|
||||
|
||||
- Always provide `name`.
|
||||
- Always provide `storage`.
|
||||
- Always provide `teacher`.
|
||||
- Always provide `runtimeAI` if you call `optimize()` or `applyUpdate()`.
|
||||
|
||||
## Modes
|
||||
|
||||
- `batch`: offline prompt learning only.
|
||||
- `continuous`: offline optimization plus bounded feedback-aware `applyUpdate(...)`.
|
||||
- `playbook`: structured context/playbook learning plus `applyUpdate(...)`.
|
||||
|
||||
## Preferred Construction
|
||||
|
||||
```typescript
|
||||
import {
|
||||
AxLearn,
|
||||
ax,
|
||||
ai,
|
||||
type AxCheckpoint,
|
||||
type AxStorage,
|
||||
type AxTrace,
|
||||
} from '@ax-llm/ax';
|
||||
|
||||
const storage: AxStorage = {
|
||||
save: async (_name, _item) => {
|
||||
// persist trace/checkpoint
|
||||
},
|
||||
load: async (_name, _query) => {
|
||||
// return traces/checkpoints
|
||||
return [];
|
||||
},
|
||||
};
|
||||
|
||||
const teacher = ai({
|
||||
name: 'openai',
|
||||
apiKey: process.env.OPENAI_APIKEY!,
|
||||
});
|
||||
|
||||
const runtimeAI = ai({
|
||||
name: 'openai',
|
||||
apiKey: process.env.OPENAI_APIKEY!,
|
||||
});
|
||||
|
||||
const gen = ax(`
|
||||
customerQuery:string "User message" ->
|
||||
supportReply:string "Agent reply"
|
||||
`);
|
||||
|
||||
const agent = new AxLearn(gen, {
|
||||
name: 'support-bot-v1',
|
||||
storage,
|
||||
teacher,
|
||||
runtimeAI,
|
||||
mode: 'continuous',
|
||||
budget: 12,
|
||||
examples: [
|
||||
{
|
||||
customerQuery: 'Where is my order?',
|
||||
supportReply: 'Your order is in transit and should arrive in 2 days.',
|
||||
},
|
||||
{
|
||||
customerQuery: 'I need a refund.',
|
||||
supportReply: 'I can help with that. Please share your order number.',
|
||||
},
|
||||
],
|
||||
generateExamples: false,
|
||||
});
|
||||
|
||||
await agent.ready();
|
||||
```
|
||||
|
||||
## Runtime Pattern
|
||||
|
||||
```typescript
|
||||
const prediction = await agent.forward(runtimeAI, {
|
||||
customerQuery: 'My package is late.',
|
||||
});
|
||||
|
||||
const traces = await agent.getTraces({ limit: 1 });
|
||||
if (traces[0]) {
|
||||
await agent.addFeedback(traces[0].id, {
|
||||
score: 0,
|
||||
label: 'needs-empathy',
|
||||
comment: 'Acknowledge the frustration more directly.',
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
## Offline Optimization
|
||||
|
||||
```typescript
|
||||
const result = await agent.optimize({
|
||||
// Optional overrides
|
||||
budget: 20,
|
||||
});
|
||||
|
||||
console.log(result.mode);
|
||||
console.log(result.score);
|
||||
console.log(result.improvement);
|
||||
console.log(result.checkpointVersion);
|
||||
```
|
||||
|
||||
`result.improvement` is the gain relative to the prior/restored score.
|
||||
|
||||
## Continuous Update
|
||||
|
||||
Use `applyUpdate(...)` only in `continuous` or `playbook` mode.
|
||||
|
||||
- In `continuous` mode, `example` may be input-only.
|
||||
- `prediction` is the observed runtime output being critiqued.
|
||||
- If `example` includes expected output fields, that expected-output row stays eligible for scored optimization.
|
||||
- The observed `prediction` row is feedback/reflection context, not a scored train/validation row by itself.
|
||||
- Feedback-bearing scored examples should stay in the training pool when non-feedback rows can fill validation.
|
||||
- In `playbook` mode, `getInstruction()` returns the active composed prompt.
|
||||
|
||||
```typescript
|
||||
const update = await agent.applyUpdate({
|
||||
example: {
|
||||
customerQuery: 'My package is late.',
|
||||
},
|
||||
prediction,
|
||||
feedback: {
|
||||
score: 0,
|
||||
label: 'needs-empathy',
|
||||
comment: 'Acknowledge the frustration more directly.',
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
## Playbook Mode
|
||||
|
||||
- Use `mode: 'playbook'` when the learned artifact should be structured guidance, not just an instruction tweak.
|
||||
- Playbook checkpoints restore through `ready()`.
|
||||
- `applyUpdate(...)` in playbook mode performs an online structured update.
|
||||
- `getInstruction()` should be treated as the active composed runtime prompt, even before optimization if the base prompt lives in the signature description.
|
||||
- `artifact.playbookSummary` should match the persisted checkpoint `state.artifactSummary`.
|
||||
|
||||
## How Learning Data Is Used
|
||||
|
||||
- `examples` and usable traces become scored optimization rows.
|
||||
- Feedback stored with `addFeedback(...)` becomes reflection feedback for later optimization.
|
||||
- In continuous updates, `example + prediction + feedback` is used as an observed feedback event.
|
||||
- Input-only update examples are useful for reflection, but they are not promoted into scored examples unless expected outputs are present.
|
||||
|
||||
## Important Options
|
||||
|
||||
```typescript
|
||||
const agent = new AxLearn(gen, {
|
||||
name: 'agent-id',
|
||||
storage,
|
||||
teacher,
|
||||
runtimeAI,
|
||||
mode: 'batch', // 'batch' | 'continuous' | 'playbook'
|
||||
budget: 20,
|
||||
metric: async ({ prediction, example }) => {
|
||||
return prediction.supportReply === example.supportReply ? 1 : 0;
|
||||
},
|
||||
criteria: 'accuracy and tone',
|
||||
judgeOptions: {},
|
||||
examples: [],
|
||||
useTraces: true,
|
||||
generateExamples: false,
|
||||
synthCount: 20,
|
||||
validationSplit: 0.2,
|
||||
continuousOptions: {
|
||||
feedbackWindowSize: 25,
|
||||
maxRecentTraces: 100,
|
||||
updateBudget: 4,
|
||||
},
|
||||
playbookOptions: {
|
||||
maxEpochs: 2,
|
||||
},
|
||||
onTrace: (trace) => {
|
||||
console.log(trace.id);
|
||||
},
|
||||
onProgress: (progress) => {
|
||||
console.log(progress.round, progress.score);
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
## Result Shape
|
||||
|
||||
```typescript
|
||||
type AxLearnResult = {
|
||||
mode: 'batch' | 'continuous' | 'playbook';
|
||||
score: number;
|
||||
improvement: number;
|
||||
checkpointVersion: number;
|
||||
stats: {
|
||||
trainingExamples: number;
|
||||
validationExamples: number;
|
||||
feedbackExamples: number;
|
||||
durationMs: number;
|
||||
mode: 'batch' | 'continuous' | 'playbook';
|
||||
};
|
||||
state?: {
|
||||
mode: 'batch' | 'continuous' | 'playbook';
|
||||
instruction?: string;
|
||||
baseInstruction?: string;
|
||||
score?: number;
|
||||
continuous?: {
|
||||
feedbackTraceCount?: number;
|
||||
lastUpdateAt?: string;
|
||||
};
|
||||
playbook?: Record<string, unknown>;
|
||||
artifactSummary?: Record<string, unknown>;
|
||||
};
|
||||
artifact?: {
|
||||
playbook?: Record<string, unknown>;
|
||||
playbookSummary?: {
|
||||
feedbackEvents: number;
|
||||
historyBatches: number;
|
||||
bulletCount: number;
|
||||
updatedAt?: string;
|
||||
};
|
||||
lastUpdateAt?: string;
|
||||
feedbackExamples?: number;
|
||||
};
|
||||
};
|
||||
```
|
||||
|
||||
## Storage Notes
|
||||
|
||||
- `AxStorage.save(name, item)` receives either a trace or checkpoint.
|
||||
- `AxStorage.load(name, query)` should return arrays of traces or checkpoints.
|
||||
- Checkpoints may be returned unsorted. `AxLearn` restores the newest one client-side.
|
||||
|
||||
## Do This
|
||||
|
||||
- Use `runtimeAI` explicitly.
|
||||
- Await `ready()` before relying on restored state.
|
||||
- Run `optimize()` off the hot path.
|
||||
- Use `continuous` mode when you want bounded feedback-aware updates.
|
||||
- Use `playbook` mode when you want persistent structured guidance.
|
||||
- Pass the real observed model output as `prediction` in `applyUpdate(...)`.
|
||||
- Treat `getInstruction()` in playbook mode as the live composed prompt, not just the raw base instruction.
|
||||
|
||||
## Avoid This
|
||||
|
||||
- Do not assume `teacher` is the optimized runtime model.
|
||||
- Do not call `applyUpdate()` in `batch` mode.
|
||||
- Do not claim feedback affects learning unless you are storing it with `addFeedback(...)` or passing it to `applyUpdate(...)`.
|
||||
- Do not assume checkpoints load synchronously in the constructor.
|
||||
- Do not treat `prediction` as the gold answer in continuous updates.
|
||||
192
.agents/skills/ax-signature/SKILL.md
Normal file
192
.agents/skills/ax-signature/SKILL.md
Normal file
@@ -0,0 +1,192 @@
|
||||
---
|
||||
name: ax-signature
|
||||
description: This skill helps an LLM generate correct DSPy signature code using @ax-llm/ax. Use when the user asks about signatures, s(), f(), field types, string syntax, fluent builder API, validation constraints, or type-safe inputs/outputs.
|
||||
version: "19.0.33"
|
||||
---
|
||||
|
||||
# Ax Signature Reference
|
||||
|
||||
## Signature Syntax
|
||||
|
||||
```
|
||||
[description] input1:type, input2:type -> output1:type, output2:type
|
||||
```
|
||||
|
||||
## Field Types
|
||||
|
||||
| Type | Syntax | TypeScript | Example |
|
||||
|------|--------|-----------|---------|
|
||||
| String | `:string` | `string` | `userName:string` |
|
||||
| Number | `:number` | `number` | `score:number` |
|
||||
| Boolean | `:boolean` | `boolean` | `isValid:boolean` |
|
||||
| JSON | `:json` | `any` | `metadata:json` |
|
||||
| Date | `:date` | `Date` | `birthDate:date` |
|
||||
| DateTime | `:datetime` | `Date` | `timestamp:datetime` |
|
||||
| Image | `:image` | `{mimeType, data}` | `photo:image` (input only) |
|
||||
| Audio | `:audio` | `{format?, data}` | `recording:audio` (input only) |
|
||||
| File | `:file` | `{mimeType, data}` | `document:file` (input only) |
|
||||
| URL | `:url` | `string` | `website:url` |
|
||||
| Code | `:code` | `string` | `pythonScript:code` |
|
||||
| Class | `:class "a, b, c"` | `"a" \| "b" \| "c"` | `mood:class "happy, sad"` |
|
||||
|
||||
## Arrays, Optional, and Internal Fields
|
||||
|
||||
```typescript
|
||||
'tags:string[] -> processedTags:string[]' // arrays
|
||||
'query:string, context?:string -> response:string' // optional with ?
|
||||
'problem:string -> reasoning!:string, solution:string' // internal with !
|
||||
```
|
||||
|
||||
## Three Ways to Create Signatures
|
||||
|
||||
### 1. String-Based (Recommended for simple cases)
|
||||
|
||||
```typescript
|
||||
import { ax, s } from '@ax-llm/ax';
|
||||
const gen = ax('input:string -> output:string');
|
||||
const sig = s('query:string -> response:string');
|
||||
```
|
||||
|
||||
### 2. Pure Fluent Builder API
|
||||
|
||||
```typescript
|
||||
import { f } from '@ax-llm/ax';
|
||||
const sig = f()
|
||||
.input('userMessage', f.string('User input'))
|
||||
.input('contextData', f.string('Additional context').optional())
|
||||
.input('tags', f.string('Keywords').array())
|
||||
.output('responseText', f.string('AI response'))
|
||||
.output('confidenceScore', f.number('Confidence 0-1'))
|
||||
.output('debugInfo', f.string('Debug info').internal())
|
||||
.build();
|
||||
```
|
||||
|
||||
### 3. Hybrid
|
||||
|
||||
```typescript
|
||||
import { s, f } from '@ax-llm/ax';
|
||||
const sig = s('base:string -> result:string')
|
||||
.appendInputField('extra', f.json('Metadata').optional())
|
||||
.appendOutputField('score', f.number('Quality score'));
|
||||
```
|
||||
|
||||
## Fluent API Reference
|
||||
|
||||
Type creators:
|
||||
- `f.string(desc)`, `f.number(desc)`, `f.boolean(desc)`, `f.json(desc)`
|
||||
- `f.image(desc)`, `f.audio(desc)`, `f.file(desc)`, `f.url(desc)`
|
||||
- `f.email(desc)`, `f.date(desc)`, `f.datetime(desc)`
|
||||
- `f.class(['a','b','c'], desc)`, `f.code(desc)`
|
||||
- `f.object({ field: f.string() }, desc)`
|
||||
|
||||
Chainable modifiers (method chaining only, no nesting):
|
||||
- `.optional()` - make field optional
|
||||
- `.array()` / `.array('list description')` - make field an array
|
||||
- `.internal()` - output only, hidden from final output
|
||||
- `.cache()` - input only, mark for prompt caching
|
||||
|
||||
```typescript
|
||||
// Correct: pure fluent chaining
|
||||
f.string('description').optional().array()
|
||||
f.string('context').cache().optional()
|
||||
f.object({ field: f.string() }, 'item desc').array('list desc')
|
||||
|
||||
// Wrong: nested function calls (removed)
|
||||
f.array(f.string('description')) // REMOVED
|
||||
f.optional(f.string('description')) // REMOVED
|
||||
f.internal(f.string('description')) // REMOVED
|
||||
```
|
||||
|
||||
## Validation Constraints
|
||||
|
||||
### String Constraints
|
||||
|
||||
```typescript
|
||||
f.string('username').min(3).max(20)
|
||||
f.string('email').email()
|
||||
f.string('website').url()
|
||||
f.string('birthDate').date()
|
||||
f.string('timestamp').datetime()
|
||||
f.string('pattern').regex('^[A-Z0-9]')
|
||||
```
|
||||
|
||||
### Number Constraints
|
||||
|
||||
```typescript
|
||||
f.number('age').min(18).max(120)
|
||||
f.number('score').min(0).max(100)
|
||||
```
|
||||
|
||||
### Complete Validation Example
|
||||
|
||||
```typescript
|
||||
const sig = f()
|
||||
.input('formData', f.string('Raw form data'))
|
||||
.output('user', f.object({
|
||||
username: f.string('Username').min(3).max(20),
|
||||
email: f.string('Email').email(),
|
||||
age: f.number('Age').min(18).max(120),
|
||||
bio: f.string('Bio').max(500).optional(),
|
||||
website: f.string('Website').url().optional(),
|
||||
tags: f.string('Tag').min(2).max(30).array()
|
||||
}, 'User profile'))
|
||||
.build();
|
||||
```
|
||||
|
||||
## Cached Input Fields
|
||||
|
||||
```typescript
|
||||
const sig = f()
|
||||
.input('staticContext', f.string('Context').cache())
|
||||
.input('userQuery', f.string('Dynamic query'))
|
||||
.output('answer', f.string('Response'))
|
||||
.build();
|
||||
```
|
||||
|
||||
## Field Naming Rules
|
||||
|
||||
Good: `userQuestion`, `customerEmail`, `analysisResult`, `confidenceScore`
|
||||
Bad: `text`, `data`, `input`, `output`, `a`, `x`, `val` (too generic), `1field` (starts with number)
|
||||
|
||||
## Media Type Restrictions
|
||||
|
||||
- Media types (image, audio, file) are **top-level input fields only**
|
||||
- Cannot be nested in objects
|
||||
- Cannot be output fields
|
||||
|
||||
## Common Patterns
|
||||
|
||||
```typescript
|
||||
// Chain of Thought
|
||||
'problem:string -> reasoning!:string, solution:string'
|
||||
|
||||
// Classification
|
||||
'email:string -> priority:class "urgent, normal, low"'
|
||||
|
||||
// Multi-modal
|
||||
'imageData:image, question?:string -> description:string, objects:string[]'
|
||||
|
||||
// Data Extraction
|
||||
'invoiceText:string -> invoiceNumber:string, totalAmount:number, lineItems:json[]'
|
||||
|
||||
// With description
|
||||
'"Answer TypeScript questions" question:string -> answer:string, confidence:number'
|
||||
```
|
||||
|
||||
## Critical Rules
|
||||
|
||||
- Use `f()` fluent builder, NOT nested `f.array(f.string())` -- those are removed.
|
||||
- Field names must be descriptive (not generic like `text`, `data`, `input`).
|
||||
- Media types are input-only, top-level only.
|
||||
- `.internal()` is output-only (for chain-of-thought reasoning).
|
||||
- `.cache()` is input-only (for prompt caching).
|
||||
- Validation errors trigger auto-retry with correction feedback.
|
||||
- `f.email()`, `f.url()`, `f.date()`, `f.datetime()` are shorthand for `f.string().email()` etc.
|
||||
|
||||
## Examples
|
||||
|
||||
Fetch these for full working code:
|
||||
|
||||
- [Fluent Signature](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/fluent-signature-example.ts) — fluent f() API
|
||||
- [Structured Output](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/structured_output.ts) — structured output with validation
|
||||
- [Debug Schema](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/debug_schema.ts) — JSON schema validation
|
||||
292
.agents/skills/ax/SKILL.md
Normal file
292
.agents/skills/ax/SKILL.md
Normal file
@@ -0,0 +1,292 @@
|
||||
---
|
||||
name: ax
|
||||
description: This skill helps with using the @ax-llm/ax TypeScript library for building LLM applications. Use when the user asks about ax(), ai(), f(), s(), agent(), flow(), AxGen, AxAgent, AxFlow, signatures, streaming, or mentions @ax-llm/ax.
|
||||
version: "19.0.33"
|
||||
---
|
||||
|
||||
# Ax Library (@ax-llm/ax) Quick Reference
|
||||
|
||||
Ax is a TypeScript library for building LLM-powered applications with type-safe signatures, streaming support, and multi-provider compatibility.
|
||||
|
||||
> **Detailed skills available:** ax-ai (providers), ax-signature (signatures/types), ax-gen (generators), ax-agent (agents/runtime), ax-agent-optimize (agent tuning/eval), ax-flow (workflows), ax-gepa (Pareto optimization), ax-learn (self-improving agents).
|
||||
|
||||
## Imports & Factories
|
||||
|
||||
```typescript
|
||||
// Prefer factory functions: ax(), ai(), agent(), flow() — not new AxGen(), new AxAI(), etc.
|
||||
import { ax, ai, f, s, fn, agent, flow, AxMemory, AxMCPClient, AxLearn } from '@ax-llm/ax';
|
||||
|
||||
// AI provider
|
||||
const llm = ai({ name: 'openai', apiKey: process.env.OPENAI_APIKEY });
|
||||
|
||||
// Generator (from string signature)
|
||||
const gen = ax('question:string -> answer:string');
|
||||
|
||||
// Generator (from fluent signature)
|
||||
const gen = ax(
|
||||
f()
|
||||
.input('question', f.string('User question'))
|
||||
.output('answer', f.string('AI response'))
|
||||
.build()
|
||||
);
|
||||
|
||||
// Reusable signature
|
||||
const sig = s('question:string, context:string[] -> answer:string');
|
||||
|
||||
// Agent
|
||||
const myAgent = agent('userInput:string -> response:string', {
|
||||
name: 'helper',
|
||||
description: 'A helpful assistant',
|
||||
});
|
||||
|
||||
// Flow
|
||||
const wf = flow<{ input: string }, { output: string }>()
|
||||
.node('step1', 'input:string -> output:string')
|
||||
.execute('step1', (state) => ({ input: state.input }))
|
||||
.returns((state) => ({ output: state.step1Result.output }));
|
||||
|
||||
// Function tool
|
||||
const tool = fn('search')
|
||||
.description('Search the web')
|
||||
.arg('query', f.string('Search query'))
|
||||
.returns(f.string('Search results'))
|
||||
.handler(({ query }) => searchWeb(query))
|
||||
.build();
|
||||
```
|
||||
|
||||
## Running
|
||||
|
||||
```typescript
|
||||
// Forward (blocking)
|
||||
const result = await gen.forward(llm, { question: 'What is 2+2?' });
|
||||
|
||||
// Streaming
|
||||
for await (const chunk of gen.streamingForward(llm, { question: 'Tell a story' })) {
|
||||
if (chunk.delta.answer) process.stdout.write(chunk.delta.answer);
|
||||
}
|
||||
```
|
||||
|
||||
## Forward Options Quick Reference
|
||||
|
||||
| Goal | Option | Example |
|
||||
|------|--------|---------|
|
||||
| Model override | `model` | `{ model: 'gpt-4o-mini' }` |
|
||||
| Temperature | `modelConfig.temperature` | `{ modelConfig: { temperature: 0.8 } }` |
|
||||
| Max tokens | `modelConfig.maxTokens` | `{ modelConfig: { maxTokens: 500 } }` |
|
||||
| Retry on failure | `maxRetries` | `{ maxRetries: 3 }` |
|
||||
| Max agent steps | `maxSteps` | `{ maxSteps: 10 }` |
|
||||
| Fail fast | `fastFail` | `{ fastFail: true }` |
|
||||
| Thinking budget | `thinkingTokenBudget` | `{ thinkingTokenBudget: 'medium' }` |
|
||||
| Show thoughts | `showThoughts` | `{ showThoughts: true }` |
|
||||
| Context caching | `contextCache` | `{ contextCache: { cacheBreakpoint: 'after-examples' } }` |
|
||||
| Multi-sampling | `sampleCount` | `{ sampleCount: 5 }` |
|
||||
| Debug logging | `debug` | `{ debug: true }` |
|
||||
| Abort signal | `abortSignal` | `{ abortSignal: controller.signal }` |
|
||||
| Memory | `mem` | `{ mem: new AxMemory() }` |
|
||||
| Stop function | `stopFunction` | `{ stopFunction: 'finalAnswer' }` |
|
||||
| Function mode | `functionCallMode` | `{ functionCallMode: 'auto' }` |
|
||||
|
||||
## Memory and Context
|
||||
|
||||
```typescript
|
||||
import { AxMemory } from '@ax-llm/ax';
|
||||
|
||||
const memory = new AxMemory();
|
||||
|
||||
// Multi-turn conversation
|
||||
await gen.forward(llm, { userMessage: 'My name is Alice' }, { mem: memory });
|
||||
const r = await gen.forward(llm, { userMessage: 'What is my name?' }, { mem: memory });
|
||||
```
|
||||
|
||||
## Few-Shot Examples
|
||||
|
||||
```typescript
|
||||
const classifier = ax('reviewText:string -> sentiment:class "positive, negative, neutral"');
|
||||
|
||||
classifier.setExamples([
|
||||
{ reviewText: 'I love this!', sentiment: 'positive' },
|
||||
{ reviewText: 'Terrible.', sentiment: 'negative' },
|
||||
{ reviewText: 'It works.', sentiment: 'neutral' },
|
||||
]);
|
||||
```
|
||||
|
||||
## Common Patterns
|
||||
|
||||
### Classification
|
||||
|
||||
```typescript
|
||||
const classifier = ax(
|
||||
f()
|
||||
.input('text', f.string())
|
||||
.output('category', f.class(['spam', 'ham', 'uncertain']))
|
||||
.output('confidence', f.number().min(0).max(1))
|
||||
.build()
|
||||
);
|
||||
```
|
||||
|
||||
### Extraction
|
||||
|
||||
```typescript
|
||||
const extractor = ax(
|
||||
f()
|
||||
.input('text', f.string())
|
||||
.output('entities', f.object({
|
||||
people: f.string().array(),
|
||||
organizations: f.string().array(),
|
||||
locations: f.string().array()
|
||||
}))
|
||||
.build()
|
||||
);
|
||||
```
|
||||
|
||||
### Multi-modal (Images)
|
||||
|
||||
```typescript
|
||||
const analyzer = ax(
|
||||
f()
|
||||
.input('image', f.image('Image to analyze'))
|
||||
.input('question', f.string('Question').optional())
|
||||
.output('description', f.string())
|
||||
.output('objects', f.string().array())
|
||||
.build()
|
||||
);
|
||||
|
||||
const result = await analyzer.forward(llm, {
|
||||
image: { mimeType: 'image/jpeg', data: base64Data },
|
||||
question: 'What objects are in this image?'
|
||||
});
|
||||
```
|
||||
|
||||
### Chaining Generators
|
||||
|
||||
```typescript
|
||||
const researcher = ax('topic:string -> research:string, keyFacts:string[]');
|
||||
const writer = ax('research:string, keyFacts:string[] -> article:string');
|
||||
|
||||
const research = await researcher.forward(llm, { topic: 'AGI' });
|
||||
const draft = await writer.forward(llm, { research: research.research, keyFacts: research.keyFacts });
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
```typescript
|
||||
import { AxGenerateError, AxAIServiceError, AxAIServiceAbortedError } from '@ax-llm/ax';
|
||||
|
||||
try {
|
||||
const result = await gen.forward(llm, { input: 'test' });
|
||||
} catch (error) {
|
||||
if (error instanceof AxGenerateError) {
|
||||
console.error('Generation failed:', error.details.model, error.details.signature);
|
||||
} else if (error instanceof AxAIServiceAbortedError) {
|
||||
console.log('Request was aborted');
|
||||
} else if (error instanceof AxAIServiceError) {
|
||||
console.error('AI service error:', error.message);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Debugging
|
||||
|
||||
```typescript
|
||||
import { axCreateDefaultColorLogger } from '@ax-llm/ax';
|
||||
|
||||
const result = await gen.forward(llm, { input: 'test' }, {
|
||||
debug: true,
|
||||
logger: axCreateDefaultColorLogger(),
|
||||
// OpenTelemetry
|
||||
tracer: openTelemetryTracer,
|
||||
meter: openTelemetryMeter,
|
||||
});
|
||||
```
|
||||
|
||||
## MCP Integration
|
||||
|
||||
```typescript
|
||||
import { AxMCPClient } from '@ax-llm/ax';
|
||||
import { AxMCPStdioTransport } from '@ax-llm/ax-tools';
|
||||
|
||||
// Stdio transport (local MCP server)
|
||||
const transport = new AxMCPStdioTransport({
|
||||
command: 'npx',
|
||||
args: ['-y', '@modelcontextprotocol/server-memory'],
|
||||
});
|
||||
|
||||
const mcpClient = new AxMCPClient(transport, { debug: false });
|
||||
await mcpClient.init();
|
||||
|
||||
// Use with agent
|
||||
const myAgent = agent('userMessage:string -> response:string', {
|
||||
name: 'assistant',
|
||||
description: 'An assistant with MCP tools',
|
||||
functions: [mcpClient],
|
||||
});
|
||||
```
|
||||
|
||||
### HTTP Transport (Remote MCP)
|
||||
|
||||
```typescript
|
||||
import { AxMCPStreambleHTTPTransport } from '@ax-llm/ax/mcp/transports/httpStreamTransport.js';
|
||||
|
||||
const transport = new AxMCPStreambleHTTPTransport('https://remote.mcp.pipedream.net', {
|
||||
headers: { 'x-pd-project-id': projectId },
|
||||
authorization: `Bearer ${accessToken}`,
|
||||
});
|
||||
```
|
||||
|
||||
### MCP Capabilities
|
||||
|
||||
| Capability | Prefix | Description |
|
||||
|---|---|---|
|
||||
| Tools | *(none)* | Function calls |
|
||||
| Prompts | `prompt_` | Prompt templates |
|
||||
| Resources | `resource_` | File/data access |
|
||||
|
||||
```typescript
|
||||
const caps = mcpClient.getCapabilities();
|
||||
const functions = mcpClient.toFunction();
|
||||
```
|
||||
|
||||
### Function Overrides
|
||||
|
||||
```typescript
|
||||
const mcpClient = new AxMCPClient(transport, {
|
||||
functionOverrides: [
|
||||
{ name: 'search_documents', updates: { name: 'findDocs', description: 'Search docs' } }
|
||||
]
|
||||
});
|
||||
```
|
||||
|
||||
## Type Reference
|
||||
|
||||
```typescript
|
||||
class AxGen<IN, OUT> {
|
||||
forward(ai: AxAIService, values: IN, options?: AxProgramForwardOptions): Promise<OUT>;
|
||||
streamingForward(ai: AxAIService, values: IN, options?: AxProgramStreamingForwardOptions): AsyncGenerator<{ delta: Partial<OUT> }>;
|
||||
setExamples(examples: Array<Partial<IN & OUT>>): void;
|
||||
addAssert(fn: (output: OUT) => boolean, message?: string): void;
|
||||
addFieldProcessor(field: keyof OUT, fn: (value: any) => any): void;
|
||||
addStreamingFieldProcessor(field: keyof OUT, fn: (chunk: string, ctx: any) => void): void;
|
||||
stop(): void;
|
||||
}
|
||||
|
||||
class AxAgent<IN, OUT> {
|
||||
forward(ai: AxAIService, values: IN, options?: AxAgentOptions): Promise<OUT>;
|
||||
streamingForward(ai: AxAIService, values: IN, options?: AxAgentOptions): AsyncGenerator<{ delta: Partial<OUT> }>;
|
||||
getFunction(): AxFunction;
|
||||
}
|
||||
|
||||
class AxFlow<IN, OUT> {
|
||||
node(name: string, signature: string | AxSignature): AxFlow;
|
||||
execute(name: string, mapper: (state) => any): AxFlow;
|
||||
returns(mapper: (state) => OUT): AxFlow;
|
||||
forward(ai: AxAIService, values: IN): Promise<OUT>;
|
||||
}
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
Fetch these for full working code:
|
||||
|
||||
- [Chat](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/chat.ts) — multi-turn conversation
|
||||
- [Marketing](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/marketing.ts) — product use case
|
||||
- [MCP Integration](https://raw.githubusercontent.com/ax-llm/ax/refs/heads/main/src/examples/mcp-client-memory.ts) — MCP integration
|
||||
Reference in New Issue
Block a user