Streaming TTS

Overview

TTS streaming allows you to play audio as it’s being synthesized, providing faster time-to-first-audio for long text. This is especially useful for reading long articles or generating AI responses.

Basic Usage

import { RunAnywhere } from '@runanywhere/core'

// Stream synthesis with callback
await RunAnywhere.synthesizeStream(
  'This is a longer piece of text that will be synthesized and streamed in chunks for faster playback.',
  { rate: 1.0 },
  (chunk) => {
    // Each chunk contains partial audio
    console.log('Got chunk:', chunk.numSamples, 'samples')
    playChunk(chunk.audio, chunk.sampleRate)
  }
)

API Reference

`synthesizeStream`

Synthesize text with streaming audio chunks.

await RunAnywhere.synthesizeStream(
  text: string,
  options?: TTSConfiguration,
  callback?: (chunk: TTSOutput) => void
): Promise<TTSResult>

Parameters:

Parameter	Type	Description
`text`	`string`	Text to synthesize
`options`	`TTSConfiguration`	Voice configuration
`callback`	`(chunk: TTSOutput) => void`	Called for each audio chunk

TTSOutput (Chunk)

interface TTSOutput {
  /** Base64-encoded audio chunk (float32 PCM) */
  audio: string

  /** Sample rate in Hz */
  sampleRate: number

  /** Number of samples in this chunk */
  numSamples: number

  /** Duration of this chunk in seconds */
  duration: number

  /** Whether this is the final chunk */
  isFinal: boolean
}

Examples

Streaming Playback

StreamingTTS.tsx

import React, { useState, useCallback, useRef } from 'react'
import { View, Button, TextInput, Text, ActivityIndicator } from 'react-native'
import { RunAnywhere, TTSOutput } from '@runanywhere/core'

export function StreamingTTS() {
  const [text, setText] = useState('')
  const [isStreaming, setIsStreaming] = useState(false)
  const [progress, setProgress] = useState(0)
  const audioQueueRef = useRef<TTSOutput[]>([])

  const handleStream = useCallback(async () => {
    if (!text.trim()) return

    setIsStreaming(true)
    setProgress(0)
    audioQueueRef.current = []

    let totalDuration = 0
    let processedDuration = 0

    try {
      const result = await RunAnywhere.synthesizeStream(text, { rate: 1.0 }, (chunk) => {
        // Queue chunk for playback
        audioQueueRef.current.push(chunk)

        // Update progress
        processedDuration += chunk.duration
        if (totalDuration > 0) {
          setProgress(processedDuration / totalDuration)
        }

        // Play chunk immediately
        playAudioChunk(chunk)
      })

      totalDuration = result.duration
      console.log('Total duration:', result.duration, 'seconds')
    } catch (error) {
      console.error('Streaming failed:', error)
    } finally {
      setIsStreaming(false)
    }
  }, [text])

  return (
    <View style={{ padding: 16 }}>
      <TextInput
        value={text}
        onChangeText={setText}
        placeholder="Enter text to stream..."
        multiline
        numberOfLines={4}
        style={{ borderWidth: 1, padding: 12 }}
      />

      <Button
        title={isStreaming ? 'Streaming...' : 'Stream'}
        onPress={handleStream}
        disabled={isStreaming || !text.trim()}
      />

      {isStreaming && (
        <View style={{ marginTop: 16, flexDirection: 'row', alignItems: 'center' }}>
          <ActivityIndicator size="small" />
          <Text style={{ marginLeft: 8 }}>Progress: {(progress * 100).toFixed(0)}%</Text>
        </View>
      )}
    </View>
  )
}

// Helper to play audio chunk
function playAudioChunk(chunk: TTSOutput) {
  // Implementation depends on your audio library
  console.log('Playing chunk:', chunk.duration, 'seconds')
}

Audio Queue Manager

For smooth playback, manage an audio queue using the SDK’s Audio utility:

import { RunAnywhere, TTSOutput } from '@runanywhere/core'

class AudioQueuePlayer {
  private queue: TTSOutput[] = []
  private isPlaying = false

  enqueue(chunk: TTSOutput) {
    this.queue.push(chunk)
    if (!this.isPlaying) {
      this.playNext()
    }
  }

  private async playNext() {
    if (this.queue.length === 0) {
      this.isPlaying = false
      return
    }

    this.isPlaying = true
    const chunk = this.queue.shift()!

    // Convert float32 PCM to WAV and play
    const wavBase64 = RunAnywhere.Audio.createWavFromPCMFloat32(chunk.audio, chunk.sampleRate)
    await RunAnywhere.Audio.playAudio(wavBase64)

    // Play next chunk
    this.playNext()
  }

  async stop() {
    this.queue = []
    this.isPlaying = false
    await RunAnywhere.Audio.stopPlayback()
  }
}

// Usage
const player = new AudioQueuePlayer()

await RunAnywhere.synthesizeStream(longText, { rate: 1.0 }, (chunk) => player.enqueue(chunk))

AudioContext / Web Audio API does not work in React Native. Always use RunAnywhere.Audio.createWavFromPCMFloat32() to convert TTS output before playback.

Custom Hook

useTTSStream.ts

import { useState, useCallback, useRef } from 'react'
import { RunAnywhere, TTSConfiguration, TTSOutput, TTSResult } from '@runanywhere/core'

interface StreamState {
  isStreaming: boolean
  chunks: TTSOutput[]
  result: TTSResult | null
  error: Error | null
}

export function useTTSStream() {
  const [state, setState] = useState<StreamState>({
    isStreaming: false,
    chunks: [],
    result: null,
    error: null,
  })

  const stream = useCallback(
    async (text: string, options?: TTSConfiguration, onChunk?: (chunk: TTSOutput) => void) => {
      setState({ isStreaming: true, chunks: [], result: null, error: null })

      try {
        const result = await RunAnywhere.synthesizeStream(text, options, (chunk) => {
          setState((s) => ({ ...s, chunks: [...s.chunks, chunk] }))
          onChunk?.(chunk)
        })

        setState((s) => ({ ...s, isStreaming: false, result }))
        return result
      } catch (error) {
        const e = error instanceof Error ? error : new Error('Stream failed')
        setState((s) => ({ ...s, isStreaming: false, error: e }))
        throw e
      }
    },
    []
  )

  const reset = useCallback(() => {
    setState({ isStreaming: false, chunks: [], result: null, error: null })
  }, [])

  return { ...state, stream, reset }
}

With Progress Tracking

async function synthesizeWithProgress(
  text: string,
  onProgress: (percent: number) => void
): Promise<TTSResult> {
  let totalChunks = 0
  let processedChunks = 0

  // Estimate chunks based on text length
  const estimatedChunks = Math.ceil(text.length / 100)

  return RunAnywhere.synthesizeStream(text, { rate: 1.0 }, (chunk) => {
    processedChunks++
    if (chunk.isFinal) {
      totalChunks = processedChunks
    }
    onProgress(Math.min(processedChunks / estimatedChunks, 1) * 100)
  })
}

// Usage
const result = await synthesizeWithProgress(longText, (percent) =>
  console.log(`Progress: ${percent.toFixed(0)}%`)
)

Performance Considerations

Streaming provides faster time-to-first-audio (TTFA) for long text. For short phrases (< 100 characters), the overhead may not be worth it — use synthesize() instead.

When to Use Streaming

Scenario	Recommended Method
Short phrases (< 50 chars)	`synthesize()`
Medium text (50-500 chars)	Either works
Long text (> 500 chars)	`synthesizeStream()`
Real-time AI responses	`synthesizeStream()`
Pre-generated content	`synthesize()`

Buffer Size

Chunks are typically 0.5-2 seconds of audio. This provides a good balance between:

Latency: Smaller chunks = faster first audio
Efficiency: Larger chunks = less overhead

Error Handling

try {
  await RunAnywhere.synthesizeStream(text, { rate: 1.0 }, (chunk) => {
    try {
      playChunk(chunk)
    } catch (playError) {
      console.error('Playback error:', playError)
      // Continue streaming, don't fail the whole operation
    }
  })
} catch (error) {
  if (isSDKError(error) && error.code === SDKErrorCode.ttsFailed) {
    console.error('TTS streaming failed:', error.message)
  }
}

Synthesize

Basic synthesis

TTS Voices

Available voices

Voice Agent

Full voice pipeline

Getting Started

Swift SDK

Kotlin SDK

React Native SDK

Flutter SDK

Web SDK

Vibe Coding

Overview

Basic Usage

API Reference

`synthesizeStream`

TTSOutput (Chunk)

Examples

Streaming Playback

Audio Queue Manager

Custom Hook

With Progress Tracking

Performance Considerations

When to Use Streaming

Buffer Size

Error Handling

Synthesize

TTS Voices

Voice Agent

​Overview

​Basic Usage

​API Reference

​synthesizeStream

​TTSOutput (Chunk)

​Examples

​Streaming Playback

​Audio Queue Manager

​Custom Hook

​With Progress Tracking

​Performance Considerations

​When to Use Streaming

​Buffer Size

​Error Handling

​Related

Synthesize

TTS Voices

Voice Agent

Overview

Basic Usage

API Reference

`synthesizeStream`

TTSOutput (Chunk)

Examples

Streaming Playback

Audio Queue Manager

Custom Hook

With Progress Tracking

Performance Considerations

When to Use Streaming

Buffer Size

Error Handling

Related