Frontend Tools
Create frontend tools and use them within your LlamaIndex agent.
What is this?
Frontend tools enable you to define client-side functions that your LlamaIndex agent can invoke, with execution happening entirely in the user's browser. When your agent calls a frontend tool, the logic runs on the client side, giving you direct access to the frontend environment.
This can be utilized for to let your agent control the UI, generative UI, or for Human-in-the-loop interactions.
In this guide, we cover the use of frontend tools driving and interacting with the UI.
When should I use this?
Use frontend tools when you need your agent to interact with client-side primitives such as:
- Reading or modifying React component state
- Accessing browser APIs like localStorage, sessionStorage, or cookies
- Triggering UI updates or animations
- Interacting with third-party frontend libraries
- Performing actions that require the user's immediate browser context
Implementation
Run and connect your agent
You'll need to run your agent and connect it to CopilotKit before proceeding.
If you don't already have CopilotKit and your agent connected, choose one of the following options:
Add a tool to your agent
Add a tool to your agent that will be used to say hello to the user. Since the tool will be used in the frontend,
we'll add it to the frontend_tools list.
from fastapi import FastAPI
from llama_index.llms.openai import OpenAI
from llama_index.protocols.ag_ui.router import get_ag_ui_workflow_router
def sayHello(name: str) -> str:
"""Says hello to the user and takes the name as an argument."""
return f"Hello, {name}!"
# Initialize the LLM
llm = OpenAI(model="gpt-4o")
# Create the AG-UI workflow router
agentic_chat_router = get_ag_ui_workflow_router(
llm=llm,
frontend_tools=[sayHello],
system_prompt="You are a helpful AI assistant with access to various tools and capabilities.",
)
# Create FastAPI app
app = FastAPI(
title="LlamaIndex Agent",
description="A LlamaIndex agent integrated with CopilotKit",
version="1.0.0"
)
# Include the router
app.include_router(agentic_chat_router)
# Health check endpoint
@app.get("/health")
async def health_check():
return {"status": "healthy", "agent": "llamaindex"}
if __name__ == "__main__":
import uvicorn
uvicorn.run(app, host="localhost", port=8000)Create a frontend tool
First, you'll need to create a frontend tool using the useFrontendTool hook. Here's a simple one to get you started that says hello to the user.
import { useFrontendTool } from "@copilotkit/react-core"
export function Page() {
// ...
useFrontendTool({
name: "sayHello",
description: "Say hello to the user",
parameters: [
{
name: "name",
type: "string",
description: "The name of the user to say hello to",
required: true,
},
],
handler: async ({ name }) => {
alert(`Hello, ${name}!`);
},
});
// ...
}That's it! Your agent will automatically have access to all frontend tools you've defined using useFrontendTool in your React components.
Frontend tools registered with useFrontendTool are automatically forwarded to your agent through the AG-UI protocol. The agent can invoke them, and execution happens on the frontend.
Give it a try!
You've now given your agent the ability to directly call any frontend tools you've defined. These tools will be available to the agent where they can be used as needed.
