Frontend Tools
Let your agent interact with and update your React Native app's UI.
What is this?
A frontend tool is a function that you register in your app. Your agent can call it, and the function runs on the device. Because the handler runs in your app, it has direct access to component state, native modules, and any library the app already uses. That's how an agent can "reach into" the app: update React state, switch a screen, open a sheet, or read local storage.
This page covers the "agent drives the UI" shape of frontend tools. To render a component for a tool call, see Components as Tools. To ask the user before a tool runs, see Human-in-the-loop.
When should I use this?
Use frontend tools when your agent needs to:
- Read or modify React component state
- Call device APIs through native modules, such as haptics, the clipboard, or storage
- Navigate between screens
- Show alerts, toasts, or bottom sheets
- Interact with third-party libraries in your app
How it works in code
Install the CopilotKit LangGraph SDK
npm install @copilotkit/sdk-jsWire CopilotKit state + tools into your graph
Frontend tools registered with useFrontendTool arrive on the agent's
state at state.copilotkit.actions. Use CopilotKitStateAnnotation to
expose that channel on your graph, then call
convertActionsToDynamicStructuredTools(...) inside your chat node to
bind the LLM to those actions.
import type { RunnableConfig } from "@langchain/core/runnables";
import { SystemMessage } from "@langchain/core/messages";
import { MemorySaver, START, StateGraph } from "@langchain/langgraph";
import { ChatOpenAI } from "@langchain/openai";
import {
convertActionsToDynamicStructuredTools,
CopilotKitStateAnnotation,
} from "@copilotkit/sdk-js/langgraph";
// CopilotKit forwards frontend tools to the agent via
// `state.copilotkit.actions`. `CopilotKitStateAnnotation` adds that
// channel to your graph's state; `convertActionsToDynamicStructuredTools`
// turns the forwarded action schemas into LangChain tools you can bind
// at model-invocation time.
const AgentStateAnnotation = CopilotKitStateAnnotation;
export type AgentState = typeof AgentStateAnnotation.State;
const SYSTEM_PROMPT = "You are a helpful, concise assistant.";
async function runChatNode(
state: AgentState,
config: RunnableConfig,
model: ChatOpenAI,
) {
const modelWithTools = model.bindTools!([
...convertActionsToDynamicStructuredTools(state.copilotkit?.actions ?? []),
]);
const response = await modelWithTools.invoke(
[new SystemMessage({ content: SYSTEM_PROMPT }), ...state.messages],
config,
);
return { messages: response };
}
async function chatNode(state: AgentState, config: RunnableConfig) {
return runChatNode(
state,
config,
new ChatOpenAI({
modelKwargs: { reasoning_effort: "none" },
temperature: 0,
model: "gpt-6-luna",
}),
);
}
function compileGraph(node: typeof chatNode) {
return new StateGraph(AgentStateAnnotation)
.addNode("chat_node", node)
.addEdge(START, "chat_node")
.addEdge("chat_node", "__end__")
.compile({ checkpointer: new MemorySaver() });
}
export const graph = compileGraph(chatNode);Call a frontend tool from a node with no model
A node does not need a model to call a frontend tool. Return an
AIMessage with a tool_calls entry. This example calls the
change_background tool that the frontend above registers. When the run
ends, CopilotKit runs the matching useFrontendTool handler in the
browser. Then it starts a follow-up run with the result as a
ToolMessage. Route that run to a node that reads the result, so the
graph does not call the tool again:
import { randomUUID } from "node:crypto";
import { AIMessage, ToolMessage } from "@langchain/core/messages";
import { END, START, StateGraph } from "@langchain/langgraph";
import { CopilotKitStateAnnotation } from "@copilotkit/sdk-js/langgraph";
type AgentState = typeof CopilotKitStateAnnotation.State;
// Ask the frontend to run `change_background`. No model call.
async function requestBackground(state: AgentState) {
return {
messages: [
new AIMessage({
content: "",
tool_calls: [
{
id: randomUUID(),
name: "change_background",
args: { background: "linear-gradient(135deg, #1e3a8a, #9333ea)" },
type: "tool_call",
},
],
}),
],
};
}
// Runs on the follow-up run, after the frontend returns the tool result.
async function confirm(state: AgentState) {
const result = state.messages.at(-1);
return {
messages: [new AIMessage(`The frontend returned: ${result?.content}`)],
};
}
export const graph = new StateGraph(CopilotKitStateAnnotation)
.addNode("request_background", requestBackground)
.addNode("confirm", confirm)
.addConditionalEdges(START, (state: AgentState) => {
const last = state.messages.at(-1);
return last && ToolMessage.isInstance(last)
? "confirm"
: "request_background";
})
.addEdge("request_background", END)
.addEdge("confirm", END)
.compile();The tool call must be in the graph state. copilotkitEmitToolCall alone
only streams the call. At the end of the run, the messages from the graph
state replace the streamed ones, so the handler never runs.
Register a frontend tool with useFrontendTool. Give it a name, a schema for its parameters, and a handler. The agent can then call it like any other tool, and your app runs it on the device.
parameters takes any Standard Schema, such as a zod schema. zod is an optional peer dependency of @copilotkit/react-native, so add it to your app:
npm install zodimport { useState } from "react";
import { Text, View } from "react-native";
import { useFrontendTool } from "@copilotkit/react-native/headless";
import { z } from "zod";
export function TaskScreen() {
const [filter, setFilter] = useState<"all" | "open" | "done">("all");
useFrontendTool({
name: "filter_tasks",
description: "Filter the visible task list by status.",
parameters: z.object({
status: z.enum(["all", "open", "done"]),
}),
handler: async ({ status }) => {
setFilter(status);
return { applied: true, status };
},
});
return (
<View style={{ padding: 16 }}>
<Text>Current filter: {filter}</Text>
</View>
);
}The handler receives the parsed, type-safe parameters and can do anything your app can do: update state, call an API, or call a native module. Its return value is sent back to the agent as the tool result, so the model can reason about what happened.
Mount the component under CopilotKitProvider, next to your chat. This example uses the prebuilt chat from @copilotkit/react-native/components. A chat you build yourself, like the one in the quickstart, works the same way.
import { View } from "react-native";
import { CopilotKitProvider } from "@copilotkit/react-native/headless";
import { CopilotChat } from "@copilotkit/react-native/components";
import { TaskScreen } from "./src/TaskScreen";
// See the quickstart for emulator and device hosts.
const runtimeUrl = "http://localhost:8200/api/copilotkit";
export default function App() {
return (
<CopilotKitProvider runtimeUrl={runtimeUrl}>
<View style={{ flex: 1 }}>
<TaskScreen />
<CopilotChat />
</View>
</CopilotKitProvider>
);
}Peer dependencies of the prebuilt chat
@copilotkit/react-native/components needs @gorhom/bottom-sheet, react-native-gesture-handler, react-native-reanimated, and react-native-streamdown. If you only use the hooks, import from /headless and skip them. See Import surfaces.
Ask the agent to show only open tasks. "Current filter" changes to open, and the agent replies after it reads the { applied: true } result.
Handlers read the values from registration
useFrontendTool registers the tool once and keeps that handler. If the handler reads a value from props or state, it sees the value from the last registration. Pass those values as the second argument, so the tool registers again when they change:
useFrontendTool(
{
name: "add_task",
description: "Add a task to the list that is open on screen.",
parameters: z.object({ title: z.string() }),
handler: async ({ title }) => {
await saveTask(listId, title); // your own storage call
return { added: title, listId };
},
},
[listId],
);A state setter from useState never changes, so the first example needs no second argument.
Register one tool per call
useFrontendTool registers one tool per call. The React package also has useFrontendTools for a list of tools that changes between renders. @copilotkit/react-native does not export it, so call useFrontendTool once per tool, with the same number of calls on every render.
To hide a tool from the agent without removing it, set available: false. To offer a tool to one agent only, set agentId.
Draw the call in the chat
Add render to the same useFrontendTool call to show UI for the call in the chat. The prebuilt chat draws it, and so does a chat you build with useRenderToolCall. render must return a React Native element or null. A bare string type-checks, then throws inside the chat's FlatList. See Tool Call Rendering for the full pattern.
Before you ship
Your agent discovers the name, description, and parameter schema through the runtime connection. Before you ship a tool:
- Keep the description narrow.
- Validate the inputs.
- Ask the user before an action that cannot be undone. Use
useHumanInTheLoopfor that. - Keep privileged changes, secret API calls, and authorization checks on your backend.
For every option, see the useFrontendTool reference.
How is this guide?