Voice
Real-time speech-to-text in the chat composer. The user speaks, the runtime transcribes, the agent runs the resulting prompt.
Not available for agent-spec yet
voice demo, and agent-spec doesn't have it yet. See this page for CopilotKit's Built-in Agent, LangGraph (Python), LangGraph (TypeScript).You have a working chat surface and you want users to be able to speak instead of type. By the end of this guide, the chat composer will sprout a mic button, recorded audio will be transcribed by the runtime, and the transcript will appear in the composer, ready to send.
When to use this
- Hands-free or accessibility flows where typing isn't the right input modality.
- Mobile or kiosk surfaces where a long voice query is faster than thumb-typing.
- Demo and test loops where you want canned audio to drive the chat without a microphone.
If you only need file uploads (audio, images, video, documents), use Multimodal Attachments instead. Voice is specifically about live transcription of recorded speech into chat input.
Frontend
Render <CopilotChat /> from @copilotkit/react-core/v2. It shows the mic button when the runtime advertises audioFileTranscriptionEnabled: true on its /info endpoint. You turn that on in the Backend section.
If the snippet renders <VoiceChat />, that is a demo wrapper around <CopilotChat /> and a sample-audio button. In your app, render <CopilotChat /> in its place.
The runtimeUrl="/api/copilotkit-voice" points the browser to your Next.js API route. When the user clicks the mic, the chat captures audio, POSTs it to that runtime route's /transcribe endpoint, and puts the resulting transcript into the composer. The user reviews the text and sends it.
Backend
Next.js API route
- Create the route file
app/api/copilotkit-voice/[[...slug]]/route.ts. The[[...slug]]catch-all lets the runtime serve/info,/agent/:id/run, and/transcribeunder the/api/copilotkit-voicebase path. - In that file, create a
CopilotRuntimewith atranscriptionService. Serve it withcreateCopilotRuntimeHandlerfrom@copilotkit/runtime/v2. The V1 wrapper drops thetranscriptionServiceoption. - Set
basePathto/api/copilotkit-voice, the directory path of the route.
Imports from local paths that start with @/ are demo helpers. In your app, replace them with your own agent and transcription service. The guarded service in Custom transcription backends is one example.
With transcriptionService set, the runtime advertises audioFileTranscriptionEnabled: true on /info (which is what tells the chat to render the mic button) and routes POST /transcribe to the service.
Without a service, /transcribe answers 503
A runtime with no transcriptionService still serves the route, and answers every request
503 with { "error": "service_not_configured" }. The mic button never appears, and no server
error shows. If the mic button does not appear, open /info and make sure that
audioFileTranscriptionEnabled is true.
Custom transcription backends
TranscriptionService from @copilotkit/runtime/v2 is an abstract class. Subclass it to plug in any transcription provider: Whisper, AssemblyAI, Deepgram, or your own model. @copilotkit/voice ships TranscriptionServiceOpenAI, which uses OpenAI Whisper.
TranscriptionServiceOpenAI takes an openai 5.x client
@copilotkit/voice depends on openai ^5.9.0, and the openai option of
TranscriptionServiceOpenAI is typed against the copy of openai that
@copilotkit/voice resolves. Pass an OpenAI instance created from that same
installed copy. The OpenAI class has private (#private) members, so an
instance from another copy or another major version fails type-checking, even
when it looks the same.
If your app uses an openai major version other than 5, either:
- align your app on
openai5.x, so your app and@copilotkit/voiceshare one copy, or - subclass
TranscriptionServiceand call your own client insidetranscribeFile.
Return a string, and let provider errors through
transcribeFile returns the transcript as a string, so returning a richer object
is a type error. The handler sends it to the client as { text, size, type }.
Let the provider's own errors propagate unchanged. The runtime classifies failures by reading
the error text for markers like rate, 429, auth and too long, so a provider message
such as OpenAI returned 429 rate limited maps to the right error code on its own. Replacing
it with your own wording bypasses that and everything lands as a generic provider error.
Create the service only when its API key is set. If the key is missing, omit transcriptionService. The runtime then reports the capability as disabled and hides the mic. A direct call to /transcribe returns the 503 described above:
Open the chat. Click the mic, say "What is the weather in Tokyo?", and click the mic again. The text appears in the composer, ready to send.
Testing and calling /transcribe directly
Driving the demo without a mic
For Playwright runs, screenshots, or any flow where prompting for mic permissions is awkward, ship a button that emits a canned sample phrase through an onTranscribed callback, bypassing the transcription endpoint entirely:
The parent chat component can then drop that text into the composer's textarea (matched via data-testid="copilot-chat-textarea") using the native value setter and a synthetic input event so React's managed state updates correctly.
Calling /transcribe yourself
The chat handles this for you; these are the rules if you post to the route directly. As
multipart, the audio field must be named audio. Any other name reads as absent and the
route answers invalid_request. As JSON, mimeType is required alongside the base64
audio, and a payload without it is rejected the same way.
How is this guide?