Use your own LLM backend (factory mode)
Use BuiltInAgent's factory mode to bring your own AI SDK, TanStack AI, or custom LLM backend.
You have an LLM backend (AI SDK, TanStack AI, or your own HTTP API) and want a
CopilotKit chat on top of it. Factory mode runs your backend inside
BuiltInAgent. You provide a factory function that calls your backend.
CopilotKit turns the stream into AG-UI events, manages the run lifecycle, and
wires the agent into the runtime.
When to use Simple Mode vs Factory Mode
| Simple Mode | Factory Mode | |
|---|---|---|
| Setup | Minimal: pass a model string | You own the LLM call and stream |
| Model resolution | Built-in ("openai/gpt-6-luna") | You set up the model yourself |
| Tools, MCP, state tools | Automatically wired | You wire them in your factory |
| Backend support | Vercel AI SDK only | Any backend: AI SDK, TanStack AI, or custom |
| Best for | Quick setup, standard use cases | Full control, non-standard backends |
If simple mode does what you need, use simple mode. Use factory mode when you need control that simple mode doesn't offer.
To write the whole agent yourself instead of a factory, extend AG-UI's
AbstractAgent. See Write your own AG-UI agent.
Quick Start
Pick your backend:
import {
CopilotRuntime,
createCopilotRuntimeHandler,
InMemoryAgentRunner,
BuiltInAgent,
convertMessagesToVercelAISDKMessages,
} from "@copilotkit/runtime/v2";
import { streamText } from "ai";
import { openai } from "@ai-sdk/openai";
const agent = new BuiltInAgent({
type: "aisdk",
factory: ({ input, abortSignal }) =>
streamText({
model: openai("gpt-6-luna"),
messages: convertMessagesToVercelAISDKMessages(input.messages),
abortSignal,
}),
});
const runtime = new CopilotRuntime({
agents: { default: agent },
runner: new InMemoryAgentRunner(),
});
const handler = createCopilotRuntimeHandler({
runtime,
basePath: "/api/copilotkit",
});
export default handler;The frontend setup is the same as BuiltInAgent: configure the CopilotKit provider and add a chat component.
Open the chat and send a message. The reply streams in from your backend.
How It Works
Factory mode accepts a config with two fields:
type: which backend you're using:"aisdk","tanstack", or"custom"factory: a function that receives the raw request and returns a backend-native stream
The factory receives an BuiltInAgentFactoryContext (from @copilotkit/runtime/v2):
interface BuiltInAgentFactoryContext {
learnedSkills: BuiltInAgentLearnedSkills; // catalog and read-only AI SDK tools, empty when disabled
input: RunAgentInput; // messages, tools, state, context, threadId, runId, forwardedProps
abortController: AbortController; // for TanStack AI (requires AbortController)
abortSignal: AbortSignal; // preferred for AI SDK, fetch, and custom backends
}CopilotKit handles everything else: RUN_STARTED and RUN_FINISHED lifecycle events, stream-to-AG-UI conversion, error handling, and abort/cancellation. Your factory never needs to emit lifecycle events.
Don't call .abort() on the controller you were handed
abortController is passed in for backends like TanStack AI that require a full
AbortController. Hand it to them and let them own it. If you abort it inside the
factory, the stream stops, but CopilotKit does not run the teardown for the run.
To cancel a run, call abortRun() on the agent. For anything that accepts a
signal (the AI SDK, fetch, a custom backend), pass abortSignal instead.
The runner: new InMemoryAgentRunner() in the examples above is the persistence
backend that stores threads in process memory. To understand the runner
abstraction and how to swap in durable persistence, see
AgentRunner & persistence.
Using CopilotKit Intelligence? Replace runner with intelligence
Intelligence stores the threads and runs for you, so remove runner and pass
intelligence (and identifyUser) to CopilotRuntime instead. The custom
agent and its factory do not change. See
An existing runtime that already has a runner.
The factory can be async. Return a Promise if you need to do setup before streaming:
factory: async ({ input, abortSignal }) => {
const apiKey = await getApiKeyFromVault();
return streamText({ model: openai("gpt-6-luna", { apiKey }), ... });
}Examples
With Tools
import {
BuiltInAgent,
convertMessagesToVercelAISDKMessages,
convertToolsToVercelAITools,
} from "@copilotkit/runtime/v2";
import { streamText } from "ai";
import { openai } from "@ai-sdk/openai";
const agent = new BuiltInAgent({
type: "aisdk",
factory: ({ input, abortSignal }) => {
const tools = convertToolsToVercelAITools(input.tools);
return streamText({
model: openai("gpt-6-luna"),
messages: convertMessagesToVercelAISDKMessages(input.messages),
tools,
abortSignal,
});
},
});convertToolsToVercelAITools converts frontend-defined tools into AI SDK's
ToolSet format automatically.
With Reasoning (Thinking Models)
import { BuiltInAgent, convertMessagesToVercelAISDKMessages } from "@copilotkit/runtime/v2";
import { streamText } from "ai";
import { anthropic } from "@ai-sdk/anthropic";
const agent = new BuiltInAgent({
type: "aisdk",
factory: ({ input, abortSignal }) =>
streamText({
model: anthropic("claude-sonnet-4-6"),
providerOptions: {
anthropic: {
thinking: { type: "adaptive" },
// `effort` replaces the fixed token budget used by the older
// `{ type: "enabled", budgetTokens }` form.
effort: "high",
},
},
messages: convertMessagesToVercelAISDKMessages(input.messages),
abortSignal,
}),
});Reasoning events (REASONING_START, REASONING_MESSAGE_CONTENT, REASONING_END) are automatically extracted from the AI SDK stream.
Pick the model with reasoning visibility in mind. Some Anthropic models return no reasoning text by default, so the events above arrive empty. See Anthropic's extended thinking docs for the display setting.
Using BuiltInAgent without a custom factory? The same options go through its providerOptions. See Advanced Configuration.
With System Prompt, Context, and State
import {
BuiltInAgent,
convertMessagesToVercelAISDKMessages,
} from "@copilotkit/runtime/v2";
import { streamText } from "ai";
import { openai } from "@ai-sdk/openai";
const agent = new BuiltInAgent({
type: "aisdk",
factory: ({ input, abortSignal }) => {
const systemParts: string[] = ["You are a helpful assistant."];
// Add context supplied by the frontend
if (input.context?.length) {
for (const ctx of input.context) {
systemParts.push(`${ctx.description}:\n${ctx.value}`);
}
}
// Add shared application state
if (input.state && Object.keys(input.state).length > 0) {
systemParts.push(
`Application State:\n${JSON.stringify(input.state, null, 2)}`,
);
}
const messages = convertMessagesToVercelAISDKMessages(input.messages);
messages.unshift({ role: "system", content: systemParts.join("\n\n") });
return streamText({
model: openai("gpt-6-luna"),
messages,
abortSignal,
});
},
});With forwardedProps
Use forwardedProps only for non-secret, browser-controlled preferences. Validate
each value against backend-owned limits before it affects execution. Never use
these properties for credentials, tenant identity, authorization, or unrestricted
model and provider selection.
import {
BuiltInAgent,
convertMessagesToVercelAISDKMessages,
} from "@copilotkit/runtime/v2";
import { streamText } from "ai";
import { openai } from "@ai-sdk/openai";
const agent = new BuiltInAgent({
type: "aisdk",
factory: ({ input, abortSignal }) => {
const props = (input.forwardedProps ?? {}) as Record<string, unknown>;
const model = props.model === "openai/gpt-6-luna"
? openai("gpt-6-luna")
: openai("gpt-6-luna");
const temperature =
typeof props.temperature === "number" &&
props.temperature >= 0 && props.temperature <= 1
? props.temperature
: 0.7;
return streamText({
model,
temperature,
messages: convertMessagesToVercelAISDKMessages(input.messages),
abortSignal,
});
},
});The examples above accept only explicit model choices and a bounded temperature. Forward those preferences from the frontend provider:
Helper Utilities
These utilities are exported from @copilotkit/runtime/v2 to help convert between CopilotKit's input format and your backend's expected format:
| Utility | Description |
|---|---|
convertInputToTanStackAI(input) | Converts RunAgentInput to { messages, systemPrompts, tools } for TanStack AI's chat(). Handles system/developer messages, context, state, and frontend tools. |
convertMessagesToVercelAISDKMessages(messages) | Converts AG-UI messages to Vercel AI SDK's ModelMessage[] format. |
convertToolsToVercelAITools(tools) | Converts frontend-defined tools (JSON Schema) to AI SDK's ToolSet. |
convertToolDefinitionsToVercelAITools(tools) | Converts defineTool() definitions (Standard Schema) to AI SDK's ToolSet. |
resolveModel(spec) | Resolves "openai/gpt-6-luna" strings to AI SDK LanguageModel instances. |
How is this guide?