AWS Lambda
Run the CopilotKit runtime on AWS Lambda, including the response-streaming setup a chat UI requires.
The CopilotKit runtime is a Fetch handler, so it runs on Lambda without a server. The one thing that needs care is streaming. Chat responses stream as Server-Sent Events. If something in front of Lambda buffers them, the chat shows a long pause and then the whole reply at once.
Start here: pick the right front door
| Front door | Streams SSE? | Use it when |
|---|---|---|
Lambda Function URL with InvokeMode: RESPONSE_STREAM | Yes | The default. Nothing sits between the browser and the function. |
API Gateway REST API with responseTransferMode: STREAM | Yes | You need REST API features in front of the runtime: WAF, usage plans, a custom authorizer, or an existing REST API to extend. |
| API Gateway HTTP API | No | Response streaming is REST-only. Buffered: 6 MB cap, 29-second timeout. |
| Application Load Balancer | No | ALB has no streaming path for Lambda targets. |
Streaming is opt-in on every path
Both streaming front doors default to buffering: a Function URL to BUFFERED invoke mode,
an API Gateway integration to responseTransferMode: BUFFERED. Deploy either without
flipping the switch and chat appears to hang for the length of the agent run, then dumps
the entire reply in one piece.
Each front door has its own section below. Read only the section for your front door. To run agents on AWS without operating Lambda yourself, see AWS AgentCore.
The handler code below is the same for both streaming paths; only the event shape and the infrastructure config differ.
Function URL with response streaming
Response streaming needs three things:
- A Function URL with its invoke mode set to
RESPONSE_STREAM. - A handler wrapped in
awslambda.streamifyResponse. - A Node.js managed runtime of version 20 or later. AWS supports response streaming from Node.js 18, but
@copilotkit/runtimerequires Node.js 20.
1. The handler
Lambda Function URLs deliver events in payload format 2.0. Convert that into a Request, hand it to the CopilotKit handler, then pipe the Response body into the Lambda response stream.
import { Readable } from "node:stream";
import { pipeline } from "node:stream/promises";
import type { LambdaFunctionURLEvent } from "aws-lambda";
import {
CopilotRuntime,
createCopilotRuntimeHandler,
BuiltInAgent,
} from "@copilotkit/runtime/v2";
// Created once per container, reused across warm invocations.
const runtime = new CopilotRuntime({
agents: {
default: new BuiltInAgent({ model: "openai/gpt-6-luna" }),
},
});
const copilotHandler = createCopilotRuntimeHandler({
runtime,
basePath: "/api/copilotkit",
cors: true,
});
function toFetchRequest(event: LambdaFunctionURLEvent): Request {
const method = event.requestContext.http.method;
const host = event.headers.host ?? "localhost";
const query = event.rawQueryString ? `?${event.rawQueryString}` : "";
const hasBody = method !== "GET" && method !== "HEAD" && event.body != null;
return new Request(`https://${host}${event.rawPath}${query}`, {
method,
headers: new Headers(event.headers as Record<string, string>),
body: hasBody
? event.isBase64Encoded
? Buffer.from(event.body!, "base64")
: event.body
: undefined,
});
}
export const handler = awslambda.streamifyResponse(
async (event: LambdaFunctionURLEvent, responseStream) => {
const response = await copilotHandler(toFetchRequest(event));
// Status and headers must be attached before the first byte is written.
const stream = awslambda.HttpResponseStream.from(responseStream, {
statusCode: response.status,
headers: Object.fromEntries(response.headers),
});
if (!response.body) {
stream.end();
return;
}
// Readable.fromWeb bridges the Fetch ReadableStream to a Node stream, so
// SSE chunks reach the client as the agent produces them.
await pipeline(Readable.fromWeb(response.body as any), stream);
},
);awslambda is a global injected by the managed Node.js runtime, so TypeScript needs to be told it exists:
import type { Writable } from "node:stream";
declare global {
namespace awslambda {
function streamifyResponse<TEvent>(
handler: (event: TEvent, responseStream: Writable, context: unknown) => Promise<void>,
): (event: TEvent, responseStream: Writable, context: unknown) => Promise<void>;
namespace HttpResponseStream {
function from(
stream: Writable,
metadata: { statusCode: number; headers?: Record<string, string> },
): Writable;
}
}
}
export {};2. Set the invoke mode
Streaming is off by default. The Function URL must be created with InvokeMode: RESPONSE_STREAM. Flipping it later requires updating the URL config, not the function.
aws lambda create-function-url-config \
--function-name copilotkit-runtime \
--auth-type NONE \
--invoke-mode RESPONSE_STREAM \
--cors '{"AllowOrigins":["https://myapp.com"],"AllowHeaders":["content-type","authorization"],"AllowMethods":["GET","POST"]}'3. Point the frontend at it
The Function URL is the origin; basePath is appended to it.
Verify streaming actually works
curl -N on the run endpoint should print SSE events progressively. If the whole body
arrives at once, streaming is not active. Check that the Function URL's invoke mode is
RESPONSE_STREAM, and that nothing buffering (an HTTP API, an ALB, a CloudFront
distribution without streaming configured) sits in front of it.
API Gateway REST API with response streaming
Streaming on a REST API is set per integration and is off by default. Set responseTransferMode to STREAM. API Gateway then invokes the function through InvokeWithResponseStream and forwards bytes as they arrive. This also removes the 10 MB response cap and the 29-second integration timeout that apply to buffered integrations.
Use this path when you want REST API features in front of the runtime. If you don't, the Function URL above is less to configure.
1. Adapt the handler to payload format 1.0
The streaming half of the handler is unchanged: awslambda.streamifyResponse and HttpResponseStream.from emit exactly the metadata-plus-delimiter format API Gateway expects. What changes is the event: a REST proxy integration delivers payload format 1.0, not the 2.0 shape a Function URL sends.
import type { APIGatewayProxyEvent } from "aws-lambda";
// REST proxy integrations deliver payload format 1.0: `httpMethod` and `path`
// rather than `requestContext.http.method` and `rawPath`. `path` excludes the
// stage name, which is what we want — it lines up with the runtime's basePath.
function toFetchRequest(event: APIGatewayProxyEvent): Request {
const method = event.httpMethod;
const headers = new Headers();
for (const [name, value] of Object.entries(event.headers)) {
if (value != null) headers.append(name, value);
}
const query = new URLSearchParams();
for (const [name, values] of Object.entries(
event.multiValueQueryStringParameters ?? {},
)) {
for (const value of values ?? []) query.append(name, value);
}
const search = query.toString();
const hasBody = method !== "GET" && method !== "HEAD" && event.body != null;
return new Request(
`https://${headers.get("host") ?? "localhost"}${event.path}${search ? `?${search}` : ""}`,
{
method,
headers,
body: hasBody
? event.isBase64Encoded
? Buffer.from(event.body!, "base64")
: event.body
: undefined,
},
);
}The awslambda.streamifyResponse(...) block from the Function URL section works as-is on top of this.
2. Configure the integration
Three settings matter:
- A Lambda proxy (
AWS_PROXY) integration. Streaming is not supported on non-proxy integration types. - A URI ending in
/response-streaming-invocations. responseTransferMode: STREAM.
On an existing integration, patch the URI and the transfer mode together, then redeploy the stage:
aws apigateway update-integration \
--rest-api-id a1b2c3 \
--resource-id aaa111 \
--http-method ANY \
--patch-operations '[
{"op":"replace","path":"/uri","value":"arn:aws:apigateway:us-east-1:lambda:path/2021-11-15/functions/arn:aws:lambda:us-east-1:111122223333:function:copilotkit-runtime/response-streaming-invocations"},
{"op":"replace","path":"/responseTransferMode","value":"STREAM"},
{"op":"replace","path":"/timeoutInMillis","value":"900000"}
]'
aws apigateway create-deployment --rest-api-id a1b2c3 --stage-name prodNote the API version in the URI path: 2021-11-15/.../response-streaming-invocations, not the 2015-03-31/.../invocations a buffered Lambda proxy integration uses. The permission is unchanged: InvokeWithResponseStream authorizes against lambda:InvokeFunction, so an existing add-permission grant still applies.
3. Plan around the streaming constraints
AWS sets these limits and can change them. Check the current values in the API Gateway guide to response streaming with Lambda.
- Pick a Regional endpoint. Streams are subject to an idle timeout: 5 minutes for Regional and private endpoints, but 30 seconds for edge-optimized ones. An agent that thinks for longer than 30 seconds between SSE events will have its stream cut on an edge-optimized API. Fronting a Regional API with your own CloudFront distribution and raising its response timeout is the way to keep an edge cache.
- Raise the integration timeout. It still defaults to 29 seconds. With
STREAMyou can take it up to 15 minutes on Regional and private APIs. Match it to the Lambda timeout. STREAMdisables buffered-only features on that integration: endpoint caching, content encoding, and VTL response transformation. Compress inside the integration if you need it.- Bandwidth. The first 10 MB of each streamed response is uncapped; beyond that API Gateway throttles to 2 MB/s.
- A client timeout doesn't stop the function. When API Gateway closes the connection, the Lambda keeps running and billing until it finishes or hits its own timeout.
The console test tab always buffers
TestInvokeMethod and the console's Test tab buffer the stream and return it in one
piece, so a correctly configured integration still looks buffered there. Verify against a
deployed stage with curl -i --no-buffer instead.
Buffered fallback: HTTP APIs and ALB
Behind an HTTP API or an Application Load Balancer there is no streaming path, so serverless-http adapting the Express handler is as good as it gets. Chat still functions end to end, but every response arrives in one piece at the end of the run.
npm install express serverless-http @copilotkit/runtimeimport express from "express";
import serverlessHttp from "serverless-http";
import { CopilotRuntime, BuiltInAgent } from "@copilotkit/runtime/v2";
import { createCopilotExpressHandler } from "@copilotkit/runtime/v2/express";
const runtime = new CopilotRuntime({
agents: {
default: new BuiltInAgent({ model: "openai/gpt-6-luna" }),
},
});
const app = express();
app.use(
createCopilotExpressHandler({
runtime,
basePath: "/api/copilotkit",
cors: true,
}),
);
export const handler = serverlessHttp(app);Two constraints to plan around:
- 6 MB response cap. Lambda caps a buffered response at 6 MB. A long agent run with large tool results can exceed it, and the request fails rather than truncating.
- 29-second integration timeout. Agent runs longer than that return a 504 even though the Lambda keeps executing. On an HTTP API this ceiling is fixed; both limits are what
responseTransferMode: STREAMlifts on a REST API.
Single-route mode pairs well here, since it collapses the runtime to one POST and avoids configuring a catch-all proxy resource:
createCopilotExpressHandler({
runtime,
basePath: "/api/copilotkit",
mode: "single-route",
cors: true,
});Authentication
Use the runtime's onRequest hook to reject unauthenticated calls before any routing happens. It works identically on every front door above.
const copilotHandler = createCopilotRuntimeHandler({
runtime,
basePath: "/api/copilotkit",
cors: true,
hooks: {
onRequest: async ({ request }) => {
const token = request.headers.get("authorization");
if (!token || !(await verifyToken(token))) {
// Throwing a Response short-circuits the handler.
throw new Response("Unauthorized", { status: 401 });
}
},
},
});For AWS-native auth instead, set the Function URL's AuthType to AWS_IAM and have callers sign requests with SigV4. That moves the check into IAM, but then the browser cannot call the URL directly. You need a signing proxy in between.
See Authentication for forwarding user identity through to your agent.
Operational notes
- Bundle your dependencies.
@copilotkit/runtimeis not in any AWS-managed layer. UseNodejsFunction(CDK),BuildMethod: esbuild(SAM), or your own bundler. - Construct the runtime outside the handler. Putting
new CopilotRuntime(...)at module scope means warm invocations reuse it rather than rebuilding it per request. - Set the timeout to cover a full agent run. Lambda's maximum is 15 minutes; the default of 3 seconds will cut off almost anything.
- Cold starts are visible in chat. Provisioned concurrency is the usual fix if first-token latency matters.
- Streaming has its own limits. A streamed response can reach 200 MB against the 6 MB buffered ceiling, but Lambda throttles to 2 MB/s past the first 6 MB (API Gateway, past the first 10 MB). Streaming also incurs cost, and a streamed run is billed for the full function duration even if the client disconnects. See Lambda response streaming for the current limits.
How is this guide?