AgentRunner and persistence
Control how the runtime starts, reconnects to, and stops agent runs with AgentRunner.
By default, the TypeScript runtime keeps chat history in process memory. A
restart loses that history, and other instances cannot read it. An
AgentRunner decides where history lives. It starts each run, keeps each
thread so a client can reconnect to it, and stops a run on demand. Pick a
built-in runner, or subclass one for your own store.
TypeScript runner APIs
The runner classes and customization examples on this page belong to the TypeScript runtime. Python, Go, Ruby, and C#/.NET runtimes use Intelligence for persistence and do not expose these runner classes. See the Intelligence quickstart for their setup.
The abstraction
AgentRunner is an abstract class with four methods, mirroring the runtime's
HTTP routes:
import type { Observable } from "rxjs";
import type { BaseEvent } from "@ag-ui/client";
abstract class AgentRunner {
// Start a run; returns the stream of AG-UI events.
abstract run(request: AgentRunnerRunRequest): Observable<BaseEvent>;
// Re-attach to an existing thread's stream (reconnect / refresh).
abstract connect(request: AgentRunnerConnectRequest): Observable<BaseEvent>;
// Is a run currently active on this thread?
abstract isRunning(request: AgentRunnerIsRunningRequest): Promise<boolean>;
// Stop the active run on this thread.
abstract stop(request: AgentRunnerStopRequest): Promise<boolean | undefined>;
}run receives the threadId, the cloned agent, the AG-UI RunAgentInput, and
any persistedInputMessages. connect receives the threadId (plus optional
headers and a joinCode). The runner owns whatever storage backs those threads.
The built-in runners
| Runner | Import | Use it for |
|---|---|---|
InMemoryAgentRunner | @copilotkit/runtime/v2 | The default v2 runner. Stores thread runs in process memory. Use it for local development, single-instance deployments, or as a base class to extend. |
SqliteAgentRunner | @copilotkit/sqlite-runner | First-party, file-backed durable runner. Persists thread runs to a SQLite file so history survives restarts on a single instance. Requires the better-sqlite3 peer dependency and a real (non-:memory:) dbPath. |
IntelligenceAgentRunner | @copilotkit/runtime/v2 | Backs CopilotKit Intelligence with threads, cross-instance persistence, and threads/history features. Used automatically on an Intelligence runtime. To run it on your own infrastructure, see Self-host CopilotKit Intelligence. |
If you do not pass a runner, the runtime uses InMemoryAgentRunner. Its
history:
- is lost on restart.
- is bounded while the process runs. See bounding in-memory history.
- is not shared across instances.
To keep history, pick the runner for your deployment:
- One instance that must keep history across restarts:
SqliteAgentRunnerfrom@copilotkit/sqlite-runner. - Several instances: CopilotKit Intelligence's
IntelligenceAgentRunner, or your own runner backed by a shared datastore.
Without identifyUser, a thread id works as a key
Without identifyUser, whoever has a thread id can read, connect to, and stop
that thread through the runtime's
thread routes. GET /threads and
POST /threads/clear return 403. To list and clear threads per user, pass
identifyUser to CopilotRuntime. See
Thread authorization.
A runner serves the thread routes from its local store only when it opts in.
InMemoryAgentRunner does, along with anything extending it such as
AgentCoreRunner. SqliteAgentRunner opts out, so its read routes answer
422. IntelligenceAgentRunner scopes every thread route, including
threads/events, threads/state, and agent/stop, to the user that
identifyUser returns.
Choosing a runner
import { CopilotRuntime, BuiltInAgent, InMemoryAgentRunner } from "@copilotkit/runtime/v2";
const runtime = new CopilotRuntime({
agents: { default: new BuiltInAgent({ model: "openai/gpt-6-luna" }) },
// Explicit, but this is also the default if omitted:
runner: new InMemoryAgentRunner(),
});The runner is configured once on the
CopilotRuntime and applies to every agent it
serves. The same runner handles run, connect, and stop for all
registered agents.
Don't pass a runner on an Intelligence runtime
An Intelligence runtime wires IntelligenceAgentRunner itself, so it accepts no runner of
your own. Passing one is a type error, and the constructor throws:
Intelligence Runtime auto-wires its own `runner`; passing `runner` alongside
`intelligence` is not supported.Choosing a runner is a decision for a self-hosted runtime only. If you are converting an existing runtime, see Connect your runtime to Intelligence.
Bounding in-memory history
If your runtime process keeps growing or runs out of heap, set limits on the in-memory history. If you saw an eviction warning in your logs, see Runtime memory keeps growing.
InMemoryAgentRunner holds every thread's run history in a process-global
store. That store is bounded by default, so a long-lived server evicts old
history instead of growing until the Node.js heap is exhausted. Pass limits to
the constructor when the defaults do not match your workload:
import { CopilotRuntime, BuiltInAgent, InMemoryAgentRunner } from "@copilotkit/runtime/v2";
const runtime = new CopilotRuntime({
agents: { default: new BuiltInAgent({ model: "openai/gpt-6-luna" }) },
runner: new InMemoryAgentRunner({
maxThreads: 200,
maxRunsPerThread: 50,
maxBytes: 128 * 1024 ** 2, // 128 MiB
}),
});| Option | Default | What it bounds |
|---|---|---|
maxThreads | 1000 | Distinct threads retained. Past the cap, the least-recently-used thread is dropped whole. |
maxRunsPerThread | 100 | Runs retained per thread, evicted oldest-first. Infinity or 0 turns the cap off. Do not do this: it is the only per-thread bound, so a single busy thread then grows without a limit. To keep more runs, raise it to a large finite value. |
maxBytes | 536870912 (512 MiB) | Approximate total size of retained history across all threads. Past the ceiling, the runner evicts other least-recently-used threads. It never trims the thread that just finished a run. This is the primary limit. The two counts are secondary limits. |
When a limit is passed, the runner drops the least-recently-used history. Whichever limit is passed first applies. A thread with an active or still-finalizing run is never evicted, even if that means the store passes a limit for a short time.
Eviction takes one of two forms, and they differ in what they remove and what stays visible:
- Whole-thread eviction drops the entire least-recently-used thread: every
run, its events, and the thread's message snapshot. Both the thread-count cap
(
maxThreads) and the byte ceiling (maxBytes) trigger it. A thread dropped this way does not appear inGET /threads, and a laterconnect()has nothing left to replay for it. - Run-cap trimming (
maxRunsPerThread) drops only the oldest runs of a single over-cap thread and keeps the thread itself. The thread stays visible inGET /threadswith its original creation time. Its latest message snapshot and newest run survive. Only the trimmed runs' events are gone, so a laterconnect()replays what remains.
Either form logs the same one-line warning the first time it fires. The warning fires once per store, not once per eviction. A busy thread that trims a run on every append logs one line. Every eviction after that line is silent. Clearing threads does not re-arm the warning. Treat the line as a signal that eviction is happening, not a per-drop audit.
Eviction also weakens message de-duplication on that thread. run() strips
already-seen messages from the next RUN_STARTED input by scanning the runs it
still holds. After a thread passes maxRunsPerThread and its oldest runs are
dropped, the runner cannot see a message that lived only in an evicted run. A
later connect() or run() can send that message again, and the client can
briefly show a historical message it already observed. This is a display
artifact, not corruption. If a thread must never show old messages again, move
to a durable runner. Or raise maxRunsPerThread to a large finite value.
Limits are process-global
Every InMemoryAgentRunner in the process shares the in-memory store, so they
also share these limits. The last runner constructed with limits sets the
limits for all in-memory threads. It replaces the limits that any earlier
runner set. Don't count on a log to catch this: the clobber warning fires only when a
runner passing limits is followed by another runner passing different
limits. The common case (one runner on defaults and a second passing custom
limits) is the first explicit override and logs nothing, as do identical
re-sets and runners that pass only onConcurrentRun. Configure one consistent
set of limits per process.
Bounding prevents the crash. It does not make the runner durable. If losing
history is unacceptable, move to a durable backend. On a single instance, use
the first-party SqliteAgentRunner. It persists thread runs to a SQLite file,
so history survives restarts:
-
Install the runner and its
better-sqlite3peer dependency:npm install @copilotkit/sqlite-runner better-sqlite3 -
Pass a
SqliteAgentRunnerasrunner, with a file path asdbPath. The defaultdbPathis:memory:, which keeps nothing after a restart.app/api/copilotkit/[[...slug]]/route.ts import { CopilotRuntime, BuiltInAgent } from "@copilotkit/runtime/v2"; import { SqliteAgentRunner } from "@copilotkit/sqlite-runner"; const runtime = new CopilotRuntime({ agents: { default: new BuiltInAgent({ model: "openai/gpt-6-luna" }) }, runner: new SqliteAgentRunner({ dbPath: "./data/threads.db" }), });
For durability across horizontally scaled instances, move to CopilotKit
Intelligence's IntelligenceAgentRunner, or supply your own runner
backed by a shared datastore.
Handling a second run on a busy thread
By default, calling run() on a thread that already has a run in flight throws
Thread already running. That is the right behavior when a duplicate request
means a bug. When your UX lets a user send a fast follow-up, or a wedged run
needs to be displaced, opt into superseding instead:
import { InMemoryAgentRunner } from "@copilotkit/runtime/v2";
const runner = new InMemoryAgentRunner({ onConcurrentRun: "supersede" });| Value | Behavior |
|---|---|
"throw" (default) | A concurrent run() on the same thread throws Thread already running. |
"supersede" | The in-flight run is aborted (the same path stop() takes) and the new run starts. The superseded run's partial output is discarded rather than written to history. |
Unlike the memory limits, onConcurrentRun is per-runner. It applies only to
the runner you pass it to.
Extending a runner for a custom backend
The most common customization is subclassing InMemoryAgentRunner to layer
your own persistence (or to reconcile history replayed by an external memory
layer). Override only the methods you need and call super for the rest:
import { InMemoryAgentRunner } from "@copilotkit/runtime/v2";
export class MyRunner extends InMemoryAgentRunner {
override run(request: Parameters<InMemoryAgentRunner["run"]>[0]) {
// persist request.threadId / input here, then delegate
return super.run(request);
}
override connect(request: Parameters<InMemoryAgentRunner["connect"]>[0]) {
// re-hydrate the thread from your store before re-attaching
return super.connect(request);
}
}For a complete production example, see the
AWS AgentCore integration. It extends
InMemoryAgentRunner into an AgentCoreRunner, handles a connect() that
arrives before any run() for a thread, and synthesizes missing tool-call
results from a replayed history.
If connect() can be called for a thread your runner has never seen, such as a
new thread id on first page load, handle that case explicitly. Otherwise the
POST /agent/:id/connect route can return a 404 or an error before the user
sends a message. See the /connect 404 troubleshooting entry.
How is this guide?