import {
  CopilotRuntime,
  OpenAIAdapter
} from "@copilotkit/backend";

export async function POST(req: Request) {
  const copilotKit = new CopilotRuntime();
  return copilotKit.response(req, new OpenAIAdapter());
}

import {
  CopilotRuntime,
  OpenAIAdapter
} from "@copilotkit/backend";

export async function POST(req: Request) {
  const copilotKit = new CopilotRuntime();
  return copilotKit.response(req, new OpenAIAdapter());
}

This class is the main entry point for the runtime. It handles requests from the frontend, provides function calling and various LLM backends.

For example, to use OpenAI as a backend (check the OpenAI Adapter docs for more info):

const copilotKit = new CopilotRuntime();
return copilotKit.response(req, new OpenAIAdapter());

Currently we support:

Server Side Actions

CopilotKit supports actions that can be executed on the server side. You can define server side actions by passing the actions parameter:

const copilotKit = new CopilotRuntime({
  actions: [
    {
      name: "sayHello",
      description: "Says hello to someone.",
      argumentAnnotations: [
        {
          name: "arg",
          type: "string",
          description: "The name of the person to say hello to.",
          required: true,
        },
      ],
      implementation: async (arg) => {
        console.log("Hello from the server", arg, "!");
      },
    },
  ],
});

Server side actions can also return a result which becomes part of the message history.

This is useful because it gives the LLM context about what happened on the server side. In addition, it can be used to look up information from a vector or relational database and other sources.

In addition to that, server side actions can also come from LangChain, including support for streaming responses.

Returned results can be of the following type:

  • anything serializable to JSON
  • string
  • LangChain types:
    • IterableReadableStream
    • BaseMessageChunk
    • AIMessage

LangServe

The backend also supports LangServe, enabling you to connect to existing chains, for example python based chains. Use the langserve parameter to specify URLs for LangServe.

const copilotKit = new CopilotRuntime({
  langserve: [
    {
      chainUrl: "http://my-langserve.chain",
      name: "performResearch",
      description: "Performs research on a given topic.",
    },
  ],
});

When left out, arguments are automatically inferred from the schema provided by LangServe.

Constructor

actions
Action<T>[]

A list of server side actions that can be executed.

langserve
RemoteChain[]

An array of LangServer URLs.

debug
boolean
copilotCloud
CopilotCloud

response(req: Request, serviceAdapter: CopilotKitServiceAdapter)

Returns a Response object for streaming back the result to the client

req
Request
required

The HTTP request

serviceAdapter
CopilotKitServiceAdapter
required

The adapter to use for the response.

streamHttpServerResponse(req: any, res: any, serviceAdapter: CopilotKitServiceAdapter, headers?: Record<string, string>)

Streams messages back to the client using the HTTP response object. Use with express,

req
any
required

The HTTP request

res
any
required

The HTTP response

serviceAdapter
CopilotKitServiceAdapter
required

The adapter to use for the response.

headers
Record<string, string>

Additional headers to send with the response.