Self Hosting (Copilot Runtime)

The Copilot Runtime is the back-end component of CopilotKit, handling the communication with LLM, message history, state and more.

You may choose to self-host the Copilot Runtime, or use Copilot Cloud (recommended).

Service Adapters

Service Adapters are responsible for executing the request with the LLM and standardizing the request/response format in a way that the Copilot Runtime can understand.

Currently, we support the following service providers/adapters:

Integration

Step 1: Create an Endpoint

Copilot Runtime is designed to run as a handler for HTTP endpoints (typically /copilotkit or /api/copilotkit). We provide plug and play support for many common frameworks. For more information, check out the Self Hosting documentation.

Create a new route to handle the /api/copilotkit endpoint.

app/api/copilotkit/route.ts

import {
  CopilotRuntime,
  OpenAIAdapter,
  copilotRuntimeNextJSAppRouterEndpoint,
} from "@copilotkit/runtime";
import OpenAI from "openai";
import { NextRequest } from "next/server";
 
const openai = new OpenAI();
const serviceAdapter = new OpenAIAdapter({ openai });
 
const runtime = new CopilotRuntime();
 
export const POST = async (req: NextRequest) => {
  const { handleRequest } = copilotRuntimeNextJSAppRouterEndpoint({
    runtime,
    serviceAdapter,
    endpoint: "/api/copilotkit",
  });
 
  return handleRequest(req);
};

Your Copilot Runtime endpoint should be available at http://localhost:{port}/api/copilotkit.

Create a new route to handle the /api/copilotkit endpoint:

pages/api/copilotkit.ts

import { NextApiRequest, NextApiResponse } from "next";
import {
  CopilotRuntime,
  OpenAIAdapter,
  copilotRuntimeNextJSPagesRouterEndpoint,
} from "@copilotkit/runtime";
import OpenAI from "openai";
 
const openai = new OpenAI();
 
const handler = async (req: NextApiRequest, res: NextApiResponse) => {
  const serviceAdapter = new OpenAIAdapter({ openai });
  const runtime = new CopilotRuntime();
 
  const handleRequest = copilotRuntimeNextJSPagesRouterEndpoint({
    endpoint: "/api/copilotkit",
    runtime,
    serviceAdapter,
  });
 
  return await handleRequest(req, res);
};

Your Copilot Runtime endpoint should be available at http://localhost:{port}/api/copilotkit.

Create a new Express.js app and set up the Copilot Runtime handler:

server.ts

import express from "express";
import { CopilotRuntime, OpenAIAdapter, copilotRuntimeNodeHttpEndpoint } from "@copilotkit/runtime";
import OpenAI from "openai";
 
const app = express();
const openai = new OpenAI();
 
app.use("/copilotkit", (req, res, next) => {
  const serviceAdapter = new OpenAIAdapter({ openai });
  const runtime = new CopilotRuntime();
  const handler = copilotRuntimeNodeHttpEndpoint({
    endpoint: "/copilotkit",
    runtime,
    serviceAdapter,
  });
 
  return handler(req, res, next);
});
 
app.listen(4000, () => {
  console.log("Listening at http://localhost:4000/copilotkit");
});

Your Copilot Runtime endpoint should be available at http://localhost:4000/copilotkit.

Set up a simple Node.js HTTP server and use the Copilot Runtime to handle requests:

server.ts

import { createServer } from "node:http";
import { CopilotRuntime, OpenAIAdapter, copilotRuntimeNodeHttpEndpoint } from "@copilotkit/runtime";
import OpenAI from "openai";
 
const openai = new OpenAI();
 
const server = createServer((req, res) => {
  const serviceAdapter = new OpenAIAdapter({ openai });
  const runtime = new CopilotRuntime();
  const handler = copilotRuntimeNodeHttpEndpoint({
    endpoint: "/copilotkit",
    runtime,
    serviceAdapter,
  });
 
  return handler(req, res);
});
 
server.listen(4000, () => {
  console.log("Listening at http://localhost:4000/copilotkit");
});

Your Copilot Runtime endpoint should be available at http://localhost:4000/copilotkit.

copilotkit.controller.ts

import { All, Controller, Req, Res } from "@nestjs/common";
import { AppService } from "./app.service";
import { CopilotRuntime, OpenAIAdapter } from "@copilotkit/backend";
 
@Controller()
export class AppController {
constructor(private readonly appService: AppService) {}
 
  @All("/copilotkit")
  copilotkit(@Req() req: Request, @Res() res: Response) {
    const copilotKit = new CopilotRuntime();
    const openaiAdapter = new OpenAIAdapter();
    copilotKit.streamHttpServerResponse(req, res, openaiAdapter);
  }
 
}

Your Copilot Runtime endpoint should be available at http://localhost:{port}/copilotkit.

Step 2: Configure the `<CopilotKit>` Provider

Now you can configure the <CopilotKit> provider to point to your Copilot Runtime URL. For example, if your runtime endpoint is served at http://localhost:3000/api/runtime, you can configure the provider like this:

layout.tsx

"use client";
 
import { CopilotKit } from "@copilotkit/react-core";
import { CopilotSidebar } from "@copilotkit/react-ui";
 
export default function RootLayout({children}) {
  return (
    <CopilotKit runtimeUrl="<copilot-runtime-url>">
      <CopilotSidebar>
        {children}
      </CopilotSidebar>
    </CopilotKit>
  );
}

Self Hosting (Copilot Runtime)

Service Adapters

Integration

Step 1: Create an Endpoint

Step 2: Configure the <CopilotKit> Provider

Further Reading

Step 2: Configure the `<CopilotKit>` Provider