RAG and retrieval

Retrieval-augmented generation (RAG) has two steps: find the documents that match the question, then let an agent answer from them. In Ailu each step is a node. The retriever writes its matches to a channel; the agent reads that channel with the rest of the state.

Start with keyword search

BM25 ranks documents by the words they share with the question. It needs no API key and no vector store, and it is hard to beat on short, precise corpora (FAQs, policies, product docs).

import { components, createGraph, finalAnswer, model } from "@ailu-ai/graph-sdk";

const docs = [
  { id: "refunds", content: "Refunds are issued within 5 business days of approval." },
  { id: "shipping", content: "Orders ship within 24 hours on weekdays." },
  { id: "returns", content: "Items can be returned within 30 days." }
];

const app = createGraph({ name: "support-faq" })
  .channel("question", { type: "string", default: "" })
  .channel("context", { type: "json", default: [] as Array<{ id: string; content: string; score: number }> })
  // BM25 keyword search: no API key, no vector store. Top matches land in `context`.
  .component("retrieve", components.bm25Retriever({ query: "question", into: "context", k: 2, docs }))
  // The agent sees the question and the retrieved context.
  .agentNode("answer", {
    model: model.anthropic("claude-sonnet-4-6"),
    prompt: { system: "Answer from the context only. Say 'I don't know' otherwise." }
  })
  .edge("retrieve", "answer")
  .compile();

const out = await app.run({ question: "How long do refunds take?" });
console.log(out.channels.context); // [{ id: "refunds", content: "...", score: ... }, ...]
console.log(finalAnswer(out.channels.agentResult));

Each match is { id, content, score }.

Search by meaning with embeddings

When questions use different words than your documents, embed both and search by similarity. semanticRetriever embeds the documents once, stores the vectors, and returns the closest matches.

import { createEmbeddings, createGraph, createVectorStore, model, semanticRetriever } from "@ailu-ai/graph-sdk";

export const buildHelpdesk = () =>
  createGraph({ name: "helpdesk" })
    .channel("question", { type: "string", default: "" })
    .node(
      "retrieve",
      semanticRetriever({
        queryFrom: "question",
        into: "context",
        k: 4,
        docs: [
          { id: "refunds", content: "Refunds are issued within 5 business days of approval." },
          { id: "shipping", content: "Orders ship within 24 hours on weekdays." }
        ],
        // Embeddings need OPENAI_API_KEY (or provider: "mistral" with MISTRAL_API_KEY).
        embeddings: createEmbeddings({ provider: "openai" }),
        // Vectors are kept in a JSON file, so documents are embedded once.
        store: createVectorStore({ persistPath: "./vectors.json" })
      })
    )
    .agentNode("answer", {
      model: model.openai("gpt-4o"),
      prompt: { system: "Answer from the context only." }
    })
    .edge("retrieve", "answer")
    .compile();

• createEmbeddings({ provider: "openai" | "mistral" }) reads OPENAI_API_KEY or MISTRAL_API_KEY, and throws when the key is missing.
• createVectorStore({ persistPath }) keeps the vectors in a JSON file. Omit persistPath for an in-memory store. For large corpora, implement the VectorStore interface over your database and pass it as store.

Improve the ranking

Component	What it does
components.reranker	Re-scores the matches against the question with a cross-encoder at AILU_RERANK_ENDPOINT. Without it, keeps the order.
components.mergeRanker	Fuses the results of several retrievers (keyword + semantic) into one ranking, with Reciprocal Rank Fusion.
components.documentSplitter	Splits a long text into chunks, by characters or sentences, before you index it.
components.answerBuilder	Assembles the final answer text, with numbered citations if you want them.

All components are listed with their parameters in Components.

Next

• A complete question-answering app: Document Q&A.