AgentRunner and persistence

Control how the runtime starts, reconnects to, and stops agent runs with AgentRunner.

By default, the TypeScript runtime keeps chat history in process memory. A restart loses that history, and other instances cannot read it. An AgentRunner decides where history lives. It starts each run, keeps each thread so a client can reconnect to it, and stops a run on demand. Pick a built-in runner, or subclass one for your own store.

TypeScript runner APIs

The runner classes and customization examples on this page belong to the TypeScript runtime. Python, Go, Ruby, and C#/.NET runtimes use Intelligence for persistence and do not expose these runner classes. See the Intelligence quickstart for their setup.

The abstraction

AgentRunner is an abstract class with four methods, mirroring the runtime's HTTP routes:

import type { Observable } from "rxjs";
import type { BaseEvent } from "@ag-ui/client";

abstract class AgentRunner {
  // Start a run; returns the stream of AG-UI events.
  abstract run(request: AgentRunnerRunRequest): Observable<BaseEvent>;
  // Re-attach to an existing thread's stream (reconnect / refresh).
  abstract connect(request: AgentRunnerConnectRequest): Observable<BaseEvent>;
  // Is a run currently active on this thread?
  abstract isRunning(request: AgentRunnerIsRunningRequest): Promise<boolean>;
  // Stop the active run on this thread.
  abstract stop(request: AgentRunnerStopRequest): Promise<boolean | undefined>;
}

run receives the threadId, the cloned agent, the AG-UI RunAgentInput, and any persistedInputMessages. connect receives the threadId (plus optional headers and a joinCode). The runner owns whatever storage backs those threads.

The built-in runners

RunnerImportUse it for
InMemoryAgentRunner@copilotkit/runtime/v2The default v2 runner. Stores thread runs in process memory. Use it for local development, single-instance deployments, or as a base class to extend.
SqliteAgentRunner@copilotkit/sqlite-runnerFirst-party, file-backed durable runner. Persists thread runs to a SQLite file so history survives restarts on a single instance. Requires the better-sqlite3 peer dependency and a real (non-:memory:) dbPath.
IntelligenceAgentRunner@copilotkit/runtime/v2Backs CopilotKit Intelligence with threads, cross-instance persistence, and threads/history features. Used automatically on an Intelligence runtime. To run it on your own infrastructure, see Self-host CopilotKit Intelligence.

If you do not pass a runner, the runtime uses InMemoryAgentRunner. Its history:

To keep history, pick the runner for your deployment:

  • One instance that must keep history across restarts: SqliteAgentRunner from @copilotkit/sqlite-runner.
  • Several instances: CopilotKit Intelligence's IntelligenceAgentRunner, or your own runner backed by a shared datastore.

Without identifyUser, a thread id works as a key

Without identifyUser, whoever has a thread id can read, connect to, and stop that thread through the runtime's thread routes. GET /threads and POST /threads/clear return 403. To list and clear threads per user, pass identifyUser to CopilotRuntime. See Thread authorization.

A runner serves the thread routes from its local store only when it opts in. InMemoryAgentRunner does, along with anything extending it such as AgentCoreRunner. SqliteAgentRunner opts out, so its read routes answer 422. IntelligenceAgentRunner scopes every thread route, including threads/events, threads/state, and agent/stop, to the user that identifyUser returns.

Choosing a runner

app/api/copilotkit/[[...slug]]/route.ts
import { CopilotRuntime, BuiltInAgent, InMemoryAgentRunner } from "@copilotkit/runtime/v2";

const runtime = new CopilotRuntime({
  agents: { default: new BuiltInAgent({ model: "openai/gpt-6-luna" }) },
  // Explicit, but this is also the default if omitted:
  runner: new InMemoryAgentRunner(),
});

The runner is configured once on the CopilotRuntime and applies to every agent it serves. The same runner handles run, connect, and stop for all registered agents.

Don't pass a runner on an Intelligence runtime

An Intelligence runtime wires IntelligenceAgentRunner itself, so it accepts no runner of your own. Passing one is a type error, and the constructor throws:

Intelligence Runtime auto-wires its own `runner`; passing `runner` alongside
`intelligence` is not supported.

Choosing a runner is a decision for a self-hosted runtime only. If you are converting an existing runtime, see Connect your runtime to Intelligence.

Bounding in-memory history

If your runtime process keeps growing or runs out of heap, set limits on the in-memory history. If you saw an eviction warning in your logs, see Runtime memory keeps growing.

InMemoryAgentRunner holds every thread's run history in a process-global store. That store is bounded by default, so a long-lived server evicts old history instead of growing until the Node.js heap is exhausted. Pass limits to the constructor when the defaults do not match your workload:

app/api/copilotkit/[[...slug]]/route.ts
import { CopilotRuntime, BuiltInAgent, InMemoryAgentRunner } from "@copilotkit/runtime/v2";

const runtime = new CopilotRuntime({
  agents: { default: new BuiltInAgent({ model: "openai/gpt-6-luna" }) },
  runner: new InMemoryAgentRunner({
    maxThreads: 200,
    maxRunsPerThread: 50,
    maxBytes: 128 * 1024 ** 2, // 128 MiB
  }),
});
OptionDefaultWhat it bounds
maxThreads1000Distinct threads retained. Past the cap, the least-recently-used thread is dropped whole.
maxRunsPerThread100Runs retained per thread, evicted oldest-first. Infinity or 0 turns the cap off. Do not do this: it is the only per-thread bound, so a single busy thread then grows without a limit. To keep more runs, raise it to a large finite value.
maxBytes536870912 (512 MiB)Approximate total size of retained history across all threads. Past the ceiling, the runner evicts other least-recently-used threads. It never trims the thread that just finished a run. This is the primary limit. The two counts are secondary limits.

When a limit is passed, the runner drops the least-recently-used history. Whichever limit is passed first applies. A thread with an active or still-finalizing run is never evicted, even if that means the store passes a limit for a short time.

Limits are process-global

Every InMemoryAgentRunner in the process shares the in-memory store, so they also share these limits. The last runner constructed with limits sets the limits for all in-memory threads. It replaces the limits that any earlier runner set. Don't count on a log to catch this: the clobber warning fires only when a runner passing limits is followed by another runner passing different limits. The common case (one runner on defaults and a second passing custom limits) is the first explicit override and logs nothing, as do identical re-sets and runners that pass only onConcurrentRun. Configure one consistent set of limits per process.

Bounding prevents the crash. It does not make the runner durable. If losing history is unacceptable, move to a durable backend. On a single instance, use the first-party SqliteAgentRunner. It persists thread runs to a SQLite file, so history survives restarts:

  1. Install the runner and its better-sqlite3 peer dependency:

    npm install @copilotkit/sqlite-runner better-sqlite3
  2. Pass a SqliteAgentRunner as runner, with a file path as dbPath. The default dbPath is :memory:, which keeps nothing after a restart.

    app/api/copilotkit/[[...slug]]/route.ts
    import { CopilotRuntime, BuiltInAgent } from "@copilotkit/runtime/v2";
    import { SqliteAgentRunner } from "@copilotkit/sqlite-runner";
    
    const runtime = new CopilotRuntime({
      agents: { default: new BuiltInAgent({ model: "openai/gpt-6-luna" }) },
      runner: new SqliteAgentRunner({ dbPath: "./data/threads.db" }),
    });

For durability across horizontally scaled instances, move to CopilotKit Intelligence's IntelligenceAgentRunner, or supply your own runner backed by a shared datastore.

Handling a second run on a busy thread

By default, calling run() on a thread that already has a run in flight throws Thread already running. That is the right behavior when a duplicate request means a bug. When your UX lets a user send a fast follow-up, or a wedged run needs to be displaced, opt into superseding instead:

import { InMemoryAgentRunner } from "@copilotkit/runtime/v2";

const runner = new InMemoryAgentRunner({ onConcurrentRun: "supersede" });
ValueBehavior
"throw" (default)A concurrent run() on the same thread throws Thread already running.
"supersede"The in-flight run is aborted (the same path stop() takes) and the new run starts. The superseded run's partial output is discarded rather than written to history.

Unlike the memory limits, onConcurrentRun is per-runner. It applies only to the runner you pass it to.

Extending a runner for a custom backend

The most common customization is subclassing InMemoryAgentRunner to layer your own persistence (or to reconcile history replayed by an external memory layer). Override only the methods you need and call super for the rest:

import { InMemoryAgentRunner } from "@copilotkit/runtime/v2";

export class MyRunner extends InMemoryAgentRunner {
  override run(request: Parameters<InMemoryAgentRunner["run"]>[0]) {
    // persist request.threadId / input here, then delegate
    return super.run(request);
  }

  override connect(request: Parameters<InMemoryAgentRunner["connect"]>[0]) {
    // re-hydrate the thread from your store before re-attaching
    return super.connect(request);
  }
}

For a complete production example, see the AWS AgentCore integration. It extends InMemoryAgentRunner into an AgentCoreRunner, handles a connect() that arrives before any run() for a thread, and synthesizes missing tool-call results from a replayed history.

If connect() can be called for a thread your runner has never seen, such as a new thread id on first page load, handle that case explicitly. Otherwise the POST /agent/:id/connect route can return a 404 or an error before the user sends a message. See the /connect 404 troubleshooting entry.

How is this guide?