Open Generative UI

Let agents generate fully interactive HTML/CSS/JS UIs that stream live into the chat.

Not available for agent-spec yet

The code on this page comes from the showcase's open-gen-ui demo, and agent-spec doesn't have it yet. See this page for CopilotKit's Built-in Agent, LangGraph (Python), LangGraph (TypeScript).

What is this?

Open Generative UI lets the agent generate complete, sandboxed UI on the fly (HTML, CSS, and JavaScript) and stream it live into the chat. The user sees the interface build in real time: styles apply first, then HTML streams in progressively, and finally JavaScript expressions execute one by one.

Key benefits:

  • No predefined components: the agent creates any UI it needs, on demand
  • Live streaming: HTML streams into a preview as it's generated
  • CDN libraries: the generated UI can load Chart.js, D3, Three.js, and more via <script> tags
  • Secure sandboxing: content runs in an isolated iframe without same-origin access
  • Sandbox functions: optionally let the generated UI call sandbox functions in your app

Minimal setup

Turning on Open Generative UI takes one flag in the runtime plus a plain <CopilotChat /> on the frontend; the built-in activity renderer is auto-registered by CopilotKit, so no extra frontend wiring is needed. Your agent must still give the run's frontend tools to its model (see the warning below).

Enable it in the runtime

Set openGenerativeUI on your CopilotRuntime. List the agents that can generate UI:

The runtime then adds OpenGenerativeUIMiddleware to those agents. It converts the agent's streamed generateSandboxedUi tool call into open-generative-ui activity events, which the built-in OpenGenerativeUIActivityRenderer mounts inside a sandboxed iframe.

Your agent must pass frontend tools to its model

generateSandboxedUi reaches the agent as a frontend tool in the run input. The agent calls it only if it gives the run's frontend tools to its model. If it binds only its own tools, the model never sees generateSandboxedUi, so nothing renders and no error appears.

  • The built-in agent passes frontend tools already.
  • A LangGraph agent built with create_agent (Python) or createAgent (TypeScript) adds CopilotKitMiddleware() (Python, from copilotkit) or copilotkitMiddleware (TypeScript, from @copilotkit/sdk-js/langgraph) to its middleware.
  • A hand-built LangGraph graph binds the frontend tools with its own tools:
    • Python: use CopilotKitState. Bind state["copilotkit"]["actions"] with your own tools.
    • TypeScript: spread CopilotKitStateAnnotation.spec into the state. Convert state.copilotkit.actions with convertActionsToDynamicStructuredTools from @copilotkit/sdk-js/langgraph, then bind them.
  • Other agents forward the frontend tools from the run input. See Frontend Tools.

Drop <CopilotChat /> into the page

Wrap your app in CopilotKit and render <CopilotChat />:

The demo also passes openGenerativeUI.designSkill to change the visual style of the generated UI. This prop is optional. The demo's Chat component renders <CopilotChat agentId="open-gen-ui" /> and adds suggestion pills.

That's it. Ask the agent "build me a simple greeting card" to see HTML stream into a sandboxed preview live.

Let the generated UI call your app

Sandbox functions let the generated UI call functions in your app. The runtime setup stays the same.

Register sandbox functions on the provider

Define an array of sandbox functions. Each function has a name, a description, a parameters schema, and a handler:

sandbox-functions.ts
import { z } from "zod";

export const openGenUiSandboxFunctions = [
  {
    name: "notifyHost",
    description: "Send a short notification message from the sandboxed UI to the host page.",
    parameters: z.object({
      message: z.string().describe("A short status message."),
    }),
    handler: async ({ message }: { message: string }) => {
      console.log("notifyHost:", message);
      return { ok: true, message };
    },
  },
];

Pass the array to openGenerativeUI.sandboxFunctions on CopilotKit:

Not available for agent-spec yet

This sample comes from the showcase's open-gen-ui-advanced demo, and agent-spec doesn't have it yet.

The generated iframe calls a sandbox function with Websandbox.connection.remote.<name>(args). parameters describes the arguments to the agent. CopilotKit does not validate the arguments, so check them in your handler. The handler runs in the host page. CopilotKit adds the name, description, and parameters of each sandbox function to the agent's context, so the agent knows which sandbox functions it can call in the generated HTML and JS.

How the sandbox calls you back

Inside the generated UI, the agent writes JS that calls await Websandbox.connection.remote.notifyHost({ message: "hi" }). The call is proxied back to the host page, where your handler runs with the arguments from the iframe.

Common use cases

  • Theme toggling: generated UI controls your app's appearance
  • Cart / state management: product cards push items into host state
  • Navigation: generated UI triggers route changes in the host app
  • Data fetching: sandbox asks the host to fetch data the iframe can't reach directly

How streaming works

The agent generates the tool call's parameters in an order optimized for the user experience:

  1. placeholderMessages: shown immediately while generating
  2. css: all styles first; the preview starts once CSS is complete
  3. html: streams live into the preview as it's generated
  4. jsFunctions: reusable helpers injected before expressions
  5. jsExpressions: executed one by one; the user sees each take effect

The middleware parses the tool-call arguments incrementally and emits activity events as each parameter completes, so the preview updates progressively.

Using CDN libraries

The sandboxed iframe can load external libraries from CDNs; just include <script> or <link> tags in the generated HTML <head>. Chart.js, D3, Three.js, and any other CDN-hosted library work with no extra setup.

<head>
  <script src="https://cdn.jsdelivr.net/npm/chart.js"></script>
</head>
<body>
  <canvas id="myChart"></canvas>
</body>

Free course: See this pattern built end-to-end in Build Interactive Agents with Generative UI. This free DeepLearning.AI short course is taught by CopilotKit's CEO. It covers the full Generative UI spectrum (Controlled, Declarative, and Open-Ended).

How is this guide?