# Voice Mode

> Voice conversations with an AI assistant. Handles listening, interruptions and captions, in a full-screen session or an in-chat voice pill.

Canonical: https://hextaui.com/blocks/voice-mode
Last updated: 2026-10-29

## Overview

Voice Mode is the visual and the controls for talking to an assistant. Pick one of eight shader styles (Sky, Dither, ASCII, CRT, Ferrofluid, Halftone, Ring or Aura) or four plain dots. Each listens, thinks and answers while moving with the sound, and monochrome styles follow your light or dark theme.

Audio levels are smoothed and measured against a learned noise floor, so a quiet room reads as quiet and the motion stays soft. Rendering pauses when the visual is off screen or the tab is hidden, and reduced motion holds every style still.

VoiceSession lays it out full screen at phone width, with captions, a status line, and mute and end within reach of a thumb. Tap the visual, press Space or just start talking to interrupt. VoiceComposer is a compact version that replaces the composer inside a chat.

useBrowserVoice runs a complete loop on the browser’s own speech recognition and speech: it waits for a pause before answering, can stay silent and show replies as captions, and ignores its own words when they come back through the speakers. For realtime voice APIs, pass the microphone and reply streams to useAudioLevels and set the state from the session’s events.

## Installation

### Step 1: Get HextaUI Pro

One payment unlocks every block, including future ones. [See pricing](https://hextaui.com/pricing).

### Step 2: Add the Pro registry to components.json

```json title="components.json"
{
  "registries": {
    "@hextaui-pro": {
      "url": "https://hextaui.com/r/pro/{name}.json",
      "headers": {
        "Authorization": "Bearer ${HEXTAUI_PRO_TOKEN}"
      }
    }
  }
}
```

### Step 3: Add your token

Create a token on your [account page](https://hextaui.com/account) and put it in `.env.local` as `HEXTAUI_PRO_TOKEN`.

### Step 4: Add the block

```bash
npx shadcn@latest add @hextaui-pro/voice-mode
```

## Usage

### With the browser’s voice

useBrowserVoice listens with the microphone and the browser’s speech recognition, waits for a pause, calls respond with what you said, and talks back with the browser’s speech. Return your model’s reply from respond.

```tsx
"use client"

import { useBrowserVoice } from "@/components/blocks/voice-mode/use-browser-voice"
import { VoiceSession } from "@/components/blocks/voice-mode/voice-session"

export function VoiceChat({ onClose }: { onClose: () => void }) {
  const voice = useBrowserVoice({
    pauseMs: 900,
    respond: async (text, turns, signal) => {
      const response = await fetch("/api/voice", {
        method: "POST",
        body: JSON.stringify({ text, turns }),
        signal,
      })
      const { reply } = await response.json()
      return reply
    },
  })

  return (
    <VoiceSession
      state={voice.state}
      levels={voice.levels}
      muted={voice.muted}
      onMutedChange={voice.setMuted}
      onInterrupt={voice.interrupt}
      onEnd={() => {
        voice.end()
        onClose()
      }}
      captions={voice.captions}
      showCaptions
      error={voice.error}
    />
  )
}
```

### With a realtime voice API

For speech-to-speech models over WebRTC, drive the state from the session’s events and feed both audio streams to useAudioLevels, so the orb follows you while you talk and the model while it answers.

```tsx
"use client"

import * as React from "react"

import { useAudioLevels } from "@/components/blocks/voice-mode/audio"
import type { VoiceState } from "@/components/blocks/voice-mode/voice-orb"
import { VoiceSession } from "@/components/blocks/voice-mode/voice-session"

export function RealtimeVoice({ token }: { token: string }) {
  const [state, setState] = React.useState<VoiceState>("connecting")
  const [mic, setMic] = React.useState<MediaStream | null>(null)
  const [reply, setReply] = React.useState<MediaStream | null>(null)
  const [muted, setMuted] = React.useState(false)
  const peer = React.useRef<RTCPeerConnection | null>(null)
  const listening = useAudioLevels(mic)
  const speaking = useAudioLevels(reply)

  React.useEffect(() => {
    const connection = new RTCPeerConnection()
    peer.current = connection
    connection.ontrack = (event) => setReply(event.streams[0])
    const events = connection.createDataChannel("oai-events")
    events.onmessage = (message) => {
      const event = JSON.parse(message.data)
      if (event.type === "input_audio_buffer.speech_started") setState("listening")
      if (event.type === "input_audio_buffer.committed") setState("thinking")
      if (event.type === "output_audio_buffer.started") setState("speaking")
      if (event.type === "output_audio_buffer.stopped") setState("listening")
    }
    navigator.mediaDevices.getUserMedia({ audio: true }).then(async (stream) => {
      setMic(stream)
      stream.getTracks().forEach((track) => connection.addTrack(track, stream))
      const offer = await connection.createOffer()
      await connection.setLocalDescription(offer)
      const answer = await fetch("https://api.openai.com/v1/realtime/calls", {
        method: "POST",
        body: offer.sdp,
        headers: { Authorization: `Bearer ${token}`, "Content-Type": "application/sdp" },
      })
      await connection.setRemoteDescription({ type: "answer", sdp: await answer.text() })
      setState("listening")
    })
    return () => connection.close()
  }, [token])

  React.useEffect(() => {
    mic?.getAudioTracks().forEach((track) => (track.enabled = !muted))
  }, [mic, muted])

  return (
    <VoiceSession
      state={state}
      levels={state === "speaking" ? speaking : listening}
      muted={muted}
      onMutedChange={setMuted}
      onEnd={() => peer.current?.close()}
    />
  )
}
```

### Inside a chat

VoiceComposer replaces the composer while a voice session runs, with a small orb, the status, a timer, mute and End. Your words and the replies can stream into the thread as text.

```tsx
"use client"

import { useBrowserVoice } from "@/components/blocks/voice-mode/use-browser-voice"
import { VoiceComposer } from "@/components/blocks/voice-mode/voice-session"

export function ChatVoice({ respond }: { respond: (text: string) => Promise<string> }) {
  const voice = useBrowserVoice({ respond: (text) => respond(text) })

  if (voice.state === "idle") {
    return <button onClick={() => void voice.start()}>Start voice</button>
  }

  return (
    <VoiceComposer
      state={voice.state}
      levels={voice.levels}
      muted={voice.muted}
      onMutedChange={voice.setMuted}
      onInterrupt={voice.interrupt}
      onEnd={voice.end}
      startedAt={voice.startedAt}
    />
  )
}
```

## Anatomy

The parts you compose, from the outside in.

| Part | Description |
| --- | --- |
| `VoiceVisual` | Any style by kind: sky, dither, ascii, crt, ferrofluid, halftone, ring, aura or bars. |
| `VoiceOrb` | The shader orb. Takes the state, mute and a levels ref. |
| `VoiceBars` | The four-dot visual, with the same props. |
| `VoiceShader` | The shared WebGL engine. Pass your own fragment shader to build a new style with the same inputs: state, smoothed level, four bands and three colors. |
| `VoiceSession` | The full-screen layout: actions, visual, captions, status, mute and end. |
| `VoiceComposer` | The compact voice pill for inside a chat. |
| `VoicePicker` | A popover of voices with tinted orbs. |
| `useBrowserVoice` | A complete voice loop on browser APIs: listen, detect the end of a turn, respond, speak, interrupt. |
| `useAudioLevels` | Reads a level and four bands from any MediaStream or media element into a ref, without re-rendering. |

## API reference

### VoiceVisual

Also accepts every div prop. Size it with a width class; it stays square.

| Prop | Type | Default | Description |
| --- | --- | --- | --- |
| `kind` | `"sky" \| "dither" \| "ascii" \| "crt" \| "ferrofluid" \| "halftone" \| "ring" \| "aura" \| "bars"` | `"sky"` | Which style to render. |
| `colors` | `[string, string, string]` | – | Three colors for the shader. Hex, any CSS color, or currentColor. Each style ships its own. |
| `palette` | `{ deep, sky, cloud }` | – | Sky colors, as used by VoicePicker voices. |
| `state` | `VoiceState` | – | Drives listening, thinking and speaking motion. |
| `levels` | `RefObject<VoiceLevels>` | – | Live audio. |
| `muted` | `boolean` | `false` | Desaturates and stops reacting. Ring turns dashed. |

### VoiceOrb

Also accepts every div prop. Size it with a width class; it stays square.

| Prop | Type | Default | Description |
| --- | --- | --- | --- |
| `state` | `"idle" \| "connecting" \| "listening" \| "thinking" \| "speaking" \| "error"` | `"idle"` | Drives churn, whirl and ripples. |
| `levels` | `RefObject<VoiceLevels>` | – | Live audio, read every frame. Usually from useAudioLevels or useBrowserVoice. |
| `muted` | `boolean` | `false` | Desaturates and stops reacting. |
| `palette` | `{ deep, sky, cloud }` | `blue sky` | Three hex colors. |

### VoiceSession

| Prop | Type | Default | Description |
| --- | --- | --- | --- |
| `state` | `VoiceState` | – | The conversation state. |
| `levels` | `RefObject<VoiceLevels>` | – | Passed to the visual. |
| `variant` | `VoiceVisualKind` | `"sky"` | Which style to show. |
| `colors` | `[string, string, string]` | – | Override the style’s colors. |
| `muted` | `boolean` | `false` | Mute state. |
| `onMutedChange` | `(muted: boolean) => void` | – | Shows the mute button and the M shortcut. |
| `onInterrupt` | `() => void` | – | Makes the visual a button while thinking or speaking, plus Space. |
| `onEnd` | `() => void` | – | Shows the end button and Escape. |
| `captions` | `{ user, assistant }` | – | Latest words from each side. |
| `showCaptions` | `boolean` | `false` | Show captions under the visual. |
| `onShowCaptionsChange` | `(show: boolean) => void` | – | Shows the captions toggle and C. |
| `status` | `ReactNode` | – | Replaces the state label. |
| `footnote` | `ReactNode` | – | Between the buttons, such as a timer or minutes left. |
| `error` | `string \| null` | – | Shown in place of the status. |
| `actions` | `ReactNode` | – | Top-right controls, such as VoicePicker. |
| `palette` | `VoicePalette` | – | Orb colors. |

### VoiceComposer

| Prop | Type | Default | Description |
| --- | --- | --- | --- |
| `state` | `VoiceState` | – | The conversation state. |
| `levels` | `RefObject<VoiceLevels>` | – | Passed to the small orb. |
| `onEnd` | `() => void` | – | Ends voice and brings back the composer. |
| `muted / onMutedChange` | `boolean / (muted) => void` | – | Mute button. |
| `onInterrupt` | `() => void` | – | Tap the orb to interrupt. |
| `startedAt` | `number \| null` | – | Start time in ms, for the timer. |

### useBrowserVoice(options)

Returns { state, muted, setMuted, start, end, interrupt, levels, turns, captions, error, startedAt }.

| Prop | Type | Default | Description |
| --- | --- | --- | --- |
| `respond` | `(text, turns, signal) => Promise<string>` | – | Called when you finish a sentence. Return the reply; the signal aborts on interrupt. |
| `pauseMs` | `number` | `900` | Silence before your turn ends. |
| `bargeIn` | `boolean` | `true` | Let speaking interrupt the reply. |
| `voiceOutput` | `boolean` | `true` | Speak replies aloud. When false, replies appear as captions with the orb still animating. |
| `voice` | `{ names?, pitch?, rate? }` | – | Preferred browser voices and delivery. |
| `greeting` | `string` | – | Said when the session starts. |
| `lang` | `string` | `"en-US"` | Recognition and speech language. |

### useAudioLevels(source, target?)

Returns a ref of { level, bands } updated every frame.

| Prop | Type | Default | Description |
| --- | --- | --- | --- |
| `source` | `MediaStream \| HTMLMediaElement \| null` | – | Microphone, remote WebRTC stream or an audio element. |
| `target` | `RefObject<VoiceLevels>` | – | Write into an existing ref instead of a new one. |

## Keyboard

| Key | Action |
| --- | --- |
| `Space` | Interrupts while it’s thinking or speaking. |
| `M` | Mutes or unmutes your microphone. |
| `C` | Shows or hides captions. |
| `Esc` | Ends the voice session. |
| `Tab` | Moves through the captions toggle, voice picker, the orb when interruptible, mute and end. |

## Accessibility

- The session is a labelled region, and the status line is a polite live region, so state changes like “Listening” and “Thinking” are announced.
- The visual is a real button labelled “Interrupt” while it can be interrupted, and disabled otherwise.
- Mute uses aria-pressed, and every control has a tooltip naming its shortcut.
- Captions give a text alternative to everything said, and replies can be shown without audio.
- Shortcuts are ignored while typing in a field, except Escape.
- With reduced motion, the orb and dots hold still and only change with the state.
- Without WebGL, every shader style falls back to a still gradient, and the dots work everywhere.

## Built with

The free HextaUI components Voice Mode is made from. Each one installs on its own.

- [Button](https://hextaui.com/docs/button) - Buttons in every variant and size, with a built-in loading, success and error flow that skips the spinner for fast requests.
- [Popover](https://hextaui.com/docs/popover) - A floating panel anchored to a trigger that resizes smoothly with its content and follows the trigger’s direction.
- [Tooltip](https://hextaui.com/docs/tooltip) - A short hint on hover or keyboard focus that opens after a brief rest, switches instantly between neighbours and shows shortcuts.

## Code

10 files, added to components/blocks/voice-mode.
