Self Hosting (Copilot Runtime)
The Copilot Runtime is the back-end component of CopilotKit, handling the communication with LLM, message history, state and more.
You may choose to self-host the Copilot Runtime, or use Copilot Cloud (recommended).
Service Adapters
Service Adapters are responsible for executing the request with the LLM and standardizing the request/response format in a way that the Copilot Runtime can understand.
Currently, we support the following service providers/adapters:
- OpenAIAdapter
- OpenAIAssistantAdapter
- LangChainAdapter
- GroqAdapter
- GoogleGenerativeAIAdapter
- AnthropicAdapter
Integration
Step 1: Create an Endpoint
Copilot Runtime is designed to run as a handler for HTTP endpoints (typically /copilotkit
or /api/copilotkit
). We provide plug and play support for many common frameworks. For more information, check out the Self Hosting documentation.
Create a new route to handle the /api/copilotkit
endpoint.
import {
CopilotRuntime,
OpenAIAdapter,
copilotRuntimeNextJSAppRouterEndpoint,
} from "@copilotkit/runtime";
import OpenAI from "openai";
import { NextRequest } from "next/server";
const openai = new OpenAI();
const serviceAdapter = new OpenAIAdapter({ openai });
const runtime = new CopilotRuntime();
export const POST = async (req: NextRequest) => {
const { handleRequest } = copilotRuntimeNextJSAppRouterEndpoint({
runtime,
serviceAdapter,
endpoint: "/api/copilotkit",
});
return handleRequest(req);
};
Your Copilot Runtime endpoint should be available at http://localhost:{port}/api/copilotkit
.
Step 2: Configure the <CopilotKit>
Provider
Now you can configure the <CopilotKit>
provider to point to your Copilot Runtime URL. For example, if your runtime endpoint is served at http://localhost:3000/api/runtime
, you can configure the provider like this:
"use client";
import { CopilotKit } from "@copilotkit/react-core";
import { CopilotSidebar } from "@copilotkit/react-ui";
export default function RootLayout({children}) {
return (
<CopilotKit runtimeUrl="<copilot-runtime-url>">
<CopilotSidebar>
{children}
</CopilotSidebar>
</CopilotKit>
);
}