Frontend Tools

Run and connect your agent#

You'll need to run your agent and connect it to CopilotKit before proceeding. If you haven't done so already, you can follow the instructions in the Getting Started guide.

If you don't already have an agent, you can use the coagent starter as a starting point as this guide uses it as a starting point.

Create a frontend tool#

First, you'll need to create a frontend tool using the useFrontendTool hook. Here's a simple one to get you started that says hello to the user.

import { z } from "zod";
import { useFrontendTool } from "@copilotkit/react-core/v2"

export function Page() {
  // ...

  useFrontendTool({
    name: "sayHello",
    description: "Say hello to the user",
    parameters: z.object({
      name: z.string().describe("The name of the user to say hello to"),
    }),
    handler: async ({ name }) => {
      alert(`Hello, ${name}!`);
      return `Said hello to ${name}!`;
    },
  });

  // ...
}

Install the CopilotKit SDK#

Now, we'll need to modify the agent to access these frontend tools. In your terminal, navigate to your agent's folder and continue from there!

Any LangGraph agent can be used with CopilotKit. However, creating deep agentic experiences with CopilotKit requires our LangGraph SDK.

uv add copilotkit

poetry add copilotkit

pip install copilotkit --extra-index-url https://copilotkit.gateway.scarf.sh/simple/

conda install copilotkit -c copilotkit-channel

Inheriting from CopilotKitState#

To access the frontend tools provided by CopilotKit, you can inherit from CopilotKitState in your agent's state definition:

from copilotkit import CopilotKitState 

class YourAgentState(CopilotKitState): 
    your_additional_properties: str

import { StateSchema } from "@langchain/langgraph";
import { CopilotKitStateSchema } from "@copilotkit/sdk-js/langgraph"; 
import { z } from "zod";

export const YourAgentStateSchema = new StateSchema({
    yourAdditionalProperty: z.string(),
    ...CopilotKitStateSchema.fields, 
});
export type YourAgentState = typeof YourAgentStateSchema.State;

By doing this, your agent's state will include the copilotkit property, which contains the frontend tools that can be accessed and invoked.

Accessing Frontend Tools#

Once your agent's state includes the copilotkit property, you can access the frontend tools and utilize them within your agent's logic.

Here's how you can call a frontend tool from your agent:

What is this?#

Frontend tools enable you to define client-side functions that your agent can invoke, with execution happening entirely in the user's browser. When your agent calls a frontend tool, the logic runs on the client side, giving you direct access to the frontend environment.

This can be utilized to let your agent control the UI, for generative UI, or for Human-in-the-loop interactions. In this guide, we cover the use of frontend tools driving and interacting with the UI.

When should I use this?#

Use frontend tools when you need your agent to interact with client-side primitives such as:

• Reading or modifying React component state
• Accessing browser APIs like localStorage, sessionStorage, or cookies
• Triggering UI updates or animations
• Interacting with third-party frontend libraries
• Performing actions that require the user's immediate browser context

Create a frontend tool#

Use the useFrontendTool hook to create a tool that your agent can call from the client side:

import { z } from "zod";
import { useFrontendTool } from "@copilotkit/react-core/v2"; 

export function Page() {
  // ...

  useFrontendTool({
    name: "sayHello",
    description: "Say hello to the user",
    parameters: z.object({
      name: z.string().describe("The name of the user to say hello to"),
    }),
    handler: async ({ name }) => {
      alert(`Hello, ${name}!`);
      return `Said hello to ${name}!`;
    },
  });

  // ...
}

These tools are automatically populated by CopilotKit and are compatible with LangChain's tool call definitions, making it straightforward to integrate them into your agent's workflow.

Give it a try!#

You've now given your agent the ability to directly call any frontend tools you've defined. These tools will be available to the agent where they can be used as needed.