CopilotKit

Open Generative UI

Let agents generate fully interactive HTML/CSS/JS UIs that stream live into the chat.


"use client";/** * Open-Ended Generative UI — minimal setup. * ----------------------------------------- * The simplest possible example. Enabling `openGenerativeUI` in the * runtime (see `src/app/api/copilotkit-ogui/route.ts`) is all that's * needed — the runtime middleware streams agent-authored HTML + CSS to * the built-in `OpenGenerativeUIActivityRenderer`, which mounts it * inside a sandboxed iframe. No custom sandbox functions, no custom * tools — just chat. * * This page customises the LLM's visual-authoring prompt via * `openGenerativeUI.designSkill` on the provider (see * `VISUALIZATION_DESIGN_SKILL` below) so the cell showcases rich * educational visualisations (3D axes, neural nets, algorithms). * * Reference: https://docs.copilotkit.ai/generative-ui/open-generative-ui */import React from "react";import {  CopilotKit,  CopilotChat,  useConfigureSuggestions,} from "@copilotkit/react-core/v2";// Replaces the default shadcn-flavoured design skill with guidance tuned// for intricate, educational, self-running visualisations. Injected as// agent context for the `generateSandboxedUi` tool (see CopilotKitProvider// `openGenerativeUI.designSkill`).const VISUALIZATION_DESIGN_SKILL = `When generating UI with generateSandboxedUi, your goal is to produce a polished, intricate, EDUCATIONAL visualisation that teaches the concept the user asked about. Treat the output like a figure from a well-designed textbook or explorable-explanation — not a bare-bones demo.Geometry + rendering:- Use inline SVG (preferred) or <canvas> for geometric content — NEVER stack dozens of <div>s to draw shapes. SVG gives you crisp lines, labelled groups, and easy transforms.- Fit content within a ~600x400 content area with ~16-24px of edge padding. Use viewBox + preserveAspectRatio so it scales cleanly.- For 3D-ish scenes, either use SVG with manual perspective math OR CSS 3D transforms (transform-style: preserve-3d, perspective on the parent). Keep vanishing lines consistent.Animation:- Prefer CSS @keyframes + transitions over JS setInterval. Use animation-timing-function ease-in-out or cubic-bezier; 300-900ms per cycle; loop with animation-iteration-count: infinite where the concept is cyclical (rotations, wave cycles, network pulses).- When JS timing IS needed, use requestAnimationFrame, not setInterval.- Stagger related elements with animation-delay so motion reads as layered, not monolithic.Labels + legend + annotations:- EVERY axis gets a label (e.g. "Pitch (X)", "Yaw (Y)", "Roll (Z)").- EVERY colour-coded series gets a legend swatch with a short caption.- Add short text callouts (e.g. "Input layer", "Hidden layer 1", "Forward pass", "Pivot", "Weight = 0.42") that explain what the viewer is watching. Position with SVG <text> or positioned HTML.- Include a 1-line title + 1-line subtitle at the top describing the concept.Palette (use these semantic colours consistently):- Accent / primary motion: indigo #6366f1- Success / stable / correct: emerald #10b981- Warning / attention / active: amber #f59e0b- Error / destructive / contrast: rose #ef4444- Neutral axes, gridlines, secondary text: slate #64748b- Surfaces: white #ffffff; subtle container bg #f8fafc; text #0f172a- Use ONE accent as the motion/highlight colour per scene; reserve other semantic colours for specific series roles.Typography:- system-ui, -apple-system, "Segoe UI", sans-serif.- Title 16-18px / 600, subtitle 12-13px / 500 slate, axis + legend labels 11-12px, callouts 11-13px.- Use tabular-nums for any numeric readouts.Containers:- Outer card: white background, 1px solid #e2e8f0 border, 10-12px border-radius, 20-24px padding.- Group related visuals inside the card — do NOT let the scene bleed to the viewport edges.Motion principles:- Motion must teach. Every animated element should correspond to a step of the concept (e.g. activation flowing forward through layers, a pivot partitioning an array, a vector rotating about an axis).- No decorative spinners or jitter for its own sake.Interactivity:- This minimal cell has NO host-side sandbox functions — the visualisation is self-running. Do NOT attempt fetch, XHR, localStorage, cookies, or Websandbox.connection.remote calls. The scene must loop or auto-advance on its own.Output contract (in order):- Emit initialHeight first (typically 480-560 for these visualisations).- placeholderMessages: 2-3 short lines like ["Sketching the scene…", "Labelling axes…", "Wiring up the animation…"].- css: complete and self-contained.- html: ONE root container with the title + subtitle + SVG/canvas + legend. Include any <script src="https://cdn.jsdelivr.net/…"> tags you need INSIDE the html head/body.Accessibility:- Text contrast >= 4.5:1 against its background.- Do not rely on colour alone to distinguish series — pair colour with shape, dash pattern, or a label.`;const minimalSuggestions = [  {    title: "3D axis visualization (model airplane)",    message:      "Visualize pitch, yaw, and roll using a 3D model airplane. Render a simple airplane silhouette (SVG or CSS-3D) at the origin, with three labelled axes (X=pitch, Y=yaw, Z=roll). Animate the airplane cycling through each rotation in turn — rotate about X, pause, rotate about Y, pause, rotate about Z, pause — with a legend showing which axis is active. Label each axis and add a short caption explaining each degree of freedom.",  },  {    title: "How a neural network works",    message:      'Animate how a simple feed-forward neural network processes an input. Show 3 layers (input 4 nodes, hidden 5 nodes, output 2 nodes) with connections whose thickness encodes weight magnitude. Animate activations pulsing forward from input -> hidden -> output in a loop, brightening each node as it fires. Label each layer and add a short caption ("Forward pass"). Use indigo for active signal, slate for quiescent.',  },  {    title: "Quicksort visualization",    message:      'Visualize quicksort on an array of ~10 bars of varying heights. At each step highlight the pivot in amber, elements being compared in indigo, and swapped elements in emerald; fade sorted elements to slate. Auto-advance through the sort in a loop (~600ms per step) with a caption showing the current operation ("Partition around pivot = 47", "Swap", "Recurse left"). Show the array as SVG rects so heights read cleanly.',  },  {    title: "Fourier: square wave from sines",    message:      'Visualize how a square wave is built from the sum of odd-harmonic sine waves. Show 3 rotating circles on the left (epicycles at frequencies 1, 3, 5 with amplitudes 1, 1/3, 1/5), the running sum traced as a point, and the resulting waveform scrolling to the right over time. Label each harmonic with its frequency and amplitude; add a legend and a title "Fourier series: square wave". Loop continuously.',  },];export default function OpenGenUiDemo() {  // Minimal Open Generative UI frontend: the built-in activity renderer is  // registered by CopilotKitProvider, so a plain <CopilotChat /> is enough —  // no custom tool renderers, no activity-renderer registration.  // We DO pass `openGenerativeUI.designSkill` to swap in visualisation-tuned  // guidance in place of the default shadcn design skill.  return (    <CopilotKit      runtimeUrl="/api/copilotkit-ogui"      agent="open-gen-ui"      openGenerativeUI={{ designSkill: VISUALIZATION_DESIGN_SKILL }}    >      <div className="flex justify-center items-center h-screen w-full">        <div className="h-full w-full max-w-4xl flex flex-col p-3">          <Chat />        </div>      </div>    </CopilotKit>  );}function Chat() {  useConfigureSuggestions({    suggestions: minimalSuggestions,    available: "always",  });  return <CopilotChat agentId="open-gen-ui" className="flex-1 rounded-2xl" />;}

What is this?#

Open Generative UI lets the agent generate complete, sandboxed UI on the fly (HTML, CSS, and JavaScript) and stream it live into the chat. The user sees the interface build in real time: styles apply first, then HTML streams in progressively, and finally JavaScript expressions execute one by one.

Free course: See this pattern built end-to-end in Build Interactive Agents with Generative UI — a free DeepLearning.AI short course taught by CopilotKit's CEO covering the full Generative UI spectrum (Controlled, Declarative, and Open-Ended).

Key benefits:

  • No predefined components — the agent creates any UI it needs, on demand
  • Live streaming — HTML streams into a preview as it's generated
  • CDN libraries — the generated UI can load Chart.js, D3, Three.js, etc. via <script> tags
  • Secure sandboxing — content runs in an isolated iframe without same-origin access
  • Sandbox functions — optionally expose host functions to the generated UI for two-way communication

Minimal setup#

Turning on Open Generative UI takes one flag in the runtime plus a plain <CopilotChat /> on the frontend; the built-in activity renderer is auto-registered by CopilotKit, so no extra wiring is needed.

Enable it in the runtime#

Add OpenGenerativeUIMiddleware to your runtime configuration:

route.ts
// Server-side config is identical for the minimal and advanced cells —// the advanced behaviour (sandbox -> host function calls) is wired// entirely on the frontend via `openGenerativeUI.sandboxFunctions` on// the provider. The single `openGenerativeUI` flag below turns on// Open Generative UI for the listed agent(s); the runtime middleware// converts each agent's streamed `generateSandboxedUi` tool call into// `open-generative-ui` activity events.const runtime = new CopilotRuntime({  // @ts-ignore -- see main route.ts; published CopilotRuntime's `agents`  // type wraps Record in MaybePromise<NonEmptyRecord<...>> which rejects  // plain Records. Fixed in source, pending release.  agents: {    "open-gen-ui": openGenUiAgent,    "open-gen-ui-advanced": openGenUiAdvancedAgent,  },  openGenerativeUI: {    agents: ["open-gen-ui", "open-gen-ui-advanced"],  },});

The OpenGenerativeUIMiddleware then converts the agent's streamed generateSandboxedUi tool call into open-generative-ui activity events, which the built-in OpenGenerativeUIActivityRenderer mounts inside a sandboxed iframe.

Drop <CopilotChat /> into the page#

Wrap your app in CopilotKit and render <CopilotChat> — no extra props needed:

page.tsx
  // Minimal Open Generative UI frontend: the built-in activity renderer is  // registered by CopilotKitProvider, so a plain <CopilotChat /> is enough —  // no custom tool renderers, no activity-renderer registration.  // We DO pass `openGenerativeUI.designSkill` to swap in visualisation-tuned  // guidance in place of the default shadcn design skill.  return (    <CopilotKit      runtimeUrl="/api/copilotkit-ogui"      agent="open-gen-ui"      openGenerativeUI={{ designSkill: VISUALIZATION_DESIGN_SKILL }}    >      <div className="flex justify-center items-center h-screen w-full">        <div className="h-full w-full max-w-4xl flex flex-col p-3">          <Chat />        </div>      </div>    </CopilotKit>  );

That's it. Ask the agent "build me a simple greeting card" to see HTML stream into a sandboxed preview live.

Advanced: With app tool calling#

Sandbox functions let the generated UI call back into your host application — a generated settings panel can toggle your app's theme, a product card can push items into your cart, or a data view can ask the host to fetch data the iframe can't reach directly.

"use client";/** * Open-Ended Generative UI * ------------------------ * The agent streams ONE `generateSandboxedUi` tool call; the runtime's * `OpenGenerativeUIMiddleware` (enabled by `openGenerativeUI: { agents: [...] }` * in `api/copilotkit-ogui/route.ts`) converts that stream into * `open-generative-ui` activity events. Passing `openGenerativeUI` to * CopilotKit here activates the built-in `OpenGenerativeUIActivityRenderer`, * which mounts the agent-authored HTML + CSS inside a sandboxed iframe. * * Reference: https://docs.copilotkit.ai/generative-ui/open-generative-ui */import React from "react";import {  CopilotKit,  CopilotChat,  useConfigureSuggestions,} from "@copilotkit/react-core/v2";import { openGenUiSandboxFunctions } from "./sandbox-functions";import { openGenUiSuggestions } from "./suggestions";export default function OpenGenUiAdvancedDemo() {  return (    // Pass the sandbox-function array on the `openGenerativeUI` provider prop.    // The built-in `OpenGenerativeUIActivityRenderer` wires these as callable    // remotes inside the agent-authored iframe.    <CopilotKit      runtimeUrl="/api/copilotkit-ogui"      agent="open-gen-ui-advanced"      openGenerativeUI={{ sandboxFunctions: openGenUiSandboxFunctions }}    >      <div className="flex justify-center items-center h-screen w-full">        <div className="h-full w-full max-w-4xl">          <Chat />        </div>      </div>    </CopilotKit>  );}function Chat() {  useConfigureSuggestions({    suggestions: openGenUiSuggestions,    available: "always",  });  return (    <div className="flex h-full w-full flex-col p-3">      <CopilotChat        agentId="open-gen-ui-advanced"        className="flex-1 rounded-2xl"      />    </div>  );}

Runtime is unchanged#

The server-side flag is identical to the minimal cell; the advanced behaviour is a pure frontend addition.

route.ts
// Server-side config is identical for the minimal and advanced cells —// the advanced behaviour (sandbox -> host function calls) is wired// entirely on the frontend via `openGenerativeUI.sandboxFunctions` on// the provider. The single `openGenerativeUI` flag below turns on// Open Generative UI for the listed agent(s); the runtime middleware// converts each agent's streamed `generateSandboxedUi` tool call into// `open-generative-ui` activity events.const runtime = new CopilotRuntime({  // @ts-ignore -- see main route.ts; published CopilotRuntime's `agents`  // type wraps Record in MaybePromise<NonEmptyRecord<...>> which rejects  // plain Records. Fixed in source, pending release.  agents: {    "open-gen-ui": openGenUiAgent,    "open-gen-ui-advanced": openGenUiAdvancedAgent,  },  openGenerativeUI: {    agents: ["open-gen-ui", "open-gen-ui-advanced"],  },});

Register sandbox functions on the provider#

Each sandbox function is a Zod-validated, host-side bridge the agent can invoke from inside the generated iframe via Websandbox.connection.remote.<name>(args). The handler runs in the host page and its description is appended to the agent's context, so the agent knows which bridges are available when generating HTML/JS.

page.tsx
import React from "react";import {  CopilotKit,  CopilotChat,  useConfigureSuggestions,} from "@copilotkit/react-core/v2";import { openGenUiSandboxFunctions } from "./sandbox-functions";import { openGenUiSuggestions } from "./suggestions";export default function OpenGenUiAdvancedDemo() {  return (    // Pass the sandbox-function array on the `openGenerativeUI` provider prop.    // The built-in `OpenGenerativeUIActivityRenderer` wires these as callable    // remotes inside the agent-authored iframe.    <CopilotKit      runtimeUrl="/api/copilotkit-ogui"      agent="open-gen-ui-advanced"      openGenerativeUI={{ sandboxFunctions: openGenUiSandboxFunctions }}    >      <div className="flex justify-center items-center h-screen w-full">        <div className="h-full w-full max-w-4xl">          <Chat />        </div>      </div>    </CopilotKit>import { z } from "zod";/** * Host-side functions that agent-authored, sandboxed UIs can invoke from * inside the iframe via `Websandbox.connection.remote.<name>(args)`. * * The names, descriptions, and Zod-derived JSON schemas below are injected * into the agent's context so the LLM knows which bridges exist when it * generates HTML/JS. Each handler runs on the HOST page and its return * value is awaited by the in-iframe caller. * * Keep the surface small and obvious — these are the demo's "app-side * tools" that the sandbox-generated UI can call. */export const openGenUiSandboxFunctions = [  {    name: "evaluateExpression",    description:      "Safely evaluate a basic arithmetic expression on the host page and return the numeric result. " +      "Supports +, -, *, /, parentheses, and decimal numbers. " +      "Use this from inside a calculator or spreadsheet UI.",    parameters: z.object({      expression: z        .string()        .describe("An arithmetic expression, e.g. '12 * (3 + 4.5)'"),    }),    handler: async ({ expression }: { expression: string }) => {      // Evaluate only arithmetic-safe expressions. Reject anything with      // identifiers or suspicious characters so we never exec arbitrary JS.      if (!/^[\d+\-*/().\s]+$/.test(expression)) {        return { ok: false, error: "Unsupported characters in expression." };      }      try {        // eslint-disable-next-line no-new-func        const value = Function(`"use strict"; return (${expression});`)();        if (typeof value !== "number" || !Number.isFinite(value)) {          return { ok: false, error: "Not a finite number." };        }        // eslint-disable-next-line no-console        console.log(          "[open-gen-ui/advanced] evaluateExpression",          expression,          "=",          value,        );        return { ok: true, value };      } catch (err) {        return {          ok: false,          error: err instanceof Error ? err.message : String(err),        };      }    },  },  {    name: "notifyHost",    description:      "Send a short notification message from the sandboxed UI to the host page. " +      "The host logs the message and returns a confirmation object.",    parameters: z.object({      message: z.string().describe("A short status message."),    }),    handler: async ({ message }: { message: string }) => {      // eslint-disable-next-line no-console      console.log("[open-gen-ui/advanced] notifyHost:", message);      return { ok: true, receivedAt: new Date().toISOString(), message };    },  },];

How the sandbox calls you back

Inside the generated UI, the agent writes JS that calls await Websandbox.connection.remote.notifyHost({ message: "hi" }). The call is proxied back to the host page, where your handler runs with the validated args.

Common use cases#

  • Theme toggling — generated UI controls your app's appearance
  • Cart / state management — product cards push items into host state
  • Navigation — generated UI triggers route changes in the host app
  • Data fetching — sandbox asks the host to fetch data the iframe can't reach directly

How streaming works#

The agent generates the tool call's parameters in an order optimized for the user experience:

  1. placeholderMessages — shown immediately while generating
  2. css — all styles first; the preview starts once CSS is complete
  3. html — streams live into the preview as it's generated
  4. jsFunctions — reusable helpers injected before expressions
  5. jsExpressions — executed one by one; the user sees each take effect

The middleware parses the tool-call arguments incrementally and emits activity events as each parameter completes, so the preview updates progressively.

Using CDN libraries#

The sandboxed iframe can load external libraries from CDNs; just include <script> or <link> tags in the generated HTML <head>. Chart.js, D3, Three.js, and any other CDN-hosted library work out of the box.

<head>
  <script src="https://cdn.jsdelivr.net/npm/chart.js"></script>
</head>
<body>
  <canvas id="myChart"></canvas>
</body>