Skip to main content

Agents and models

An agent node runs an LLM in a loop: it reads the run's state, may call tools, and ends with an answer.

import { createGraph, finalAnswer, model } from "@ailu-ai/graph-sdk";

const app = createGraph({ name: "assistant" })
  .channel("question", { type: "string", default: "" })
  .agentNode("answer", {
    model: model.anthropic("claude-sonnet-4-6"),
    prompt: { system: "Answer in one sentence." }
  })
  .compile();

const out = await app.run({ question: "What is a checkpoint?" });
console.log(finalAnswer(out.channels.agentResult));

What the agent sees​

The agent gets its system prompt and the run's state: every channel, as JSON. Show it less with visibleChannels, which keeps the prompt small and keeps unrelated data away from the model:

builder.agentNode("answer", {
  model: model.anthropic("claude-sonnet-4-6"),
  prompt: { system: "Answer the question." },
  visibleChannels: ["question", "context"]
});

Read the answer​

The agent writes its result to the agentResult channel. Use outputChannel to pick another name, which you need when a graph has several agents.

FieldWhat it holds
reasoningThe agent's steps; the answer follows the last final: marker. Read it with finalAnswer(result).
structuredOutputThe parsed JSON when you asked for structured output.
usageTokens used across the agent's model calls.
approvalRequestsTool calls waiting for a human. See Tools and approval.
todosThe agent's plan, when it uses the todo tool. See Deep agents.

Choose a model​

Pass a model to model:. Everything below is exported as model from @ailu-ai/graph-sdk:

import { model } from "@ailu-ai/graph-sdk";

// A provider and a model id. The key comes from the provider's variable (OPENAI_API_KEY, ...).
const gpt = model.openai("gpt-4o");
const claude = model.anthropic("claude-sonnet-4-6");

// A capability tier: the provider is picked from the keys you have set.
const cheap = model.fast;
const careful = model.frontier;

// A provider and a tier.
const mistralFast = model.mistral.fast;

// The same thing as a string, handy in config files.
const fromConfig = model("openai:gpt-4o");

// Any OpenAI-compatible server: vLLM, LM Studio, a gateway. The key is read only from apiKeyEnv.
const local = model.openaiCompatible({
  baseURL: "http://localhost:8000/v1",
  model: "llama-3.1-8b-instruct",
  apiKeyEnv: "MY_GATEWAY_KEY"
});

With a tier (model.fast, model.balanced, model.frontier), the engine picks the provider from the keys you have set, in this order: Anthropic, OpenAI, Gemini, Mistral, OpenRouter, MiniMax, Hugging Face. The model each tier maps to is listed in Models and providers.

Each provider reads its key from one environment variable (see Install). If the key is missing, the run fails with an error that names it. Set AILU_LLM_MOCK=1 to run offline on the deterministic mock.

Get structured output​

Give the agent a JSON Schema. The provider is asked for JSON that matches it, and the parsed value lands in structuredOutput:

import { createGraph, model } from "@ailu-ai/graph-sdk";

const ticketSchema = {
  type: "object",
  properties: {
    category: { type: "string", enum: ["billing", "bug", "other"] },
    urgent: { type: "boolean" }
  },
  required: ["category", "urgent"],
  additionalProperties: false
};

const app = createGraph({ name: "ticket-triage" })
  .channel("ticket", { type: "string", default: "" })
  .agentNode("classify", {
    model: model.openai("gpt-4o"),
    prompt: { system: "Classify the support ticket." },
    // The agent must answer with JSON that matches the schema.
    middleware: [{ kind: "structuredOutput", params: { schema: ticketSchema, mode: "lenient" } }]
  })
  .compile();

const out = await app.run({ ticket: "I was charged twice this month!" });
const ticket = out.channels.agentResult.structuredOutput as { category: string; urgent: boolean } | undefined;
console.log(ticket); // { category: "billing", urgent: true }

With mode: "required" (the default), the agent retries when the answer doesn't match, then reports an error in its result. With "lenient", it keeps the plain answer and leaves structuredOutput empty.

Control cost and length​

OptionEffect
maxIterationsCaps the agent's loop (model call → tool calls → model call …).
middleware: [{ kind: "terse" }]Asks for short answers. Saves output tokens on prose; don't use it for code.
middleware: [{ kind: "contextBudget", params: { chars: 8000 } }]Trims the state the agent is shown.
middleware: [{ kind: "compress" }]Compresses long prompts through an LLMLingua service at AILU_LLMLINGUA_URL. Does nothing when it is not set.

Call a model directly​

For a single completion you don't need a graph:

import { model } from "@ailu-ai/graph-sdk";

// One call, no graph. The answer text is on `.content`.
const reply = await model.anthropic("claude-sonnet-4-6").invoke("Say hello in French.");
console.log(reply.content, reply.usage);

And for typed JSON, .output():

// Ask for JSON that matches a schema, and get it parsed.
export const classify = async (ticket: string) => {
  const triage = model.openai("gpt-4o").output({
    jsonSchema: {
      type: "object",
      properties: { category: { type: "string", enum: ["billing", "bug", "other"] } },
      required: ["category"]
    },
    parse: (value) => value as { category: "billing" | "bug" | "other" }
  });
  const { parsed } = await triage.invoke(`Classify: ${ticket}`);
  return parsed.category;
};

Next​