generateStream()

generateStream returns an AsyncIterable<LLMStreamEvent> synchronously. It is not a promise and it does not resolve to a { stream, result } pair.

generateStream(prompt: string, options?: LLMGenerationOptions): AsyncIterable<LLMStreamEvent>
generateStream(request: LLMGenerateRequest): AsyncIterable<LLMStreamEvent>

Options are the same LLMGenerationOptions that generate() takes, with the same defaults.

Hermes does not support for await...of over NitroModules async iterables. It silently fails to iterate, so no tokens arrive and the UI hangs. Always drive the iterator with manual iterator.next() calls.

Basic usage

import { RunAnywhere } from '@runanywhere/core'

const iterator = RunAnywhere.generateStream('Write a short story about a robot', {
  maxTokens: 200,
})[Symbol.asyncIterator]()

let text = ''
try {
  for (;;) {
    const { value, done } = await iterator.next()
    if (done) break
    text += value.token
    setResponse(text)
    if (value.isFinal) break
  }
} finally {
  await iterator.return?.()
}

Calling iterator.return() cancels the native generation. Breaking out of the loop without it leaves the backend running, so put it in a finally.

LLMStreamEvent

Field	Type	Meaning
`seq`	`number`	Monotonic sequence number
`timestampUs`	`number`	Microseconds since epoch
`token`	`string`	Token text; may be empty on control events
`isFinal`	`boolean`	Terminal event for this generation
`kind`	`TokenKind`	Distinguishes thought tokens from output
`tokenId`	`number`
`logprob`	`number`
`finishReason`	`string`	Set on the terminal event
`errorMessage`	`string`	Non-empty on failure
`result`	`LLMStreamFinalResult \| undefined`	Backend aggregate carried by the final event

LLMStreamFinalResult carries text, promptTokens, completionTokens, totalTokens, totalTimeMs, timeToFirstTokenMs, tokensPerSecond, finishReason, promptEvalTimeMs, and decodeTimeMs. An event with a non-empty errorMessage is followed by the iterator throwing that error, so wrap the loop in try.

aggregateStream

aggregateStream drives the iterator for you, tallies tokens and timing, and returns a populated LLMGenerationResult. It uses the manual-iterator pattern internally and tears the subscription down on early exit.

const events = RunAnywhere.generateStream(prompt, { maxTokens: 300 })

const result = await RunAnywhere.aggregateStream(prompt, events, (transcript) => {
  setResponse(transcript)
})

console.log(result.text, result.tokensPerSecond, result.ttftMs)

The onToken callback receives the full accumulated transcript, not the individual token, which is what you want for a React state setter.

React component

StreamingChat.tsx

import React, { useCallback, useState } from 'react'
import { View, Text, Button, ScrollView } from 'react-native'
import { RunAnywhere } from '@runanywhere/core'

export function StreamingChat() {
  const [response, setResponse] = useState('')
  const [streaming, setStreaming] = useState(false)
  const [metrics, setMetrics] = useState('')

  const handleStream = useCallback(async () => {
    setResponse('')
    setMetrics('')
    setStreaming(true)

    const prompt = 'Explain how neural networks work'
    try {
      const events = RunAnywhere.generateStream(prompt, {
        maxTokens: 300,
        temperature: 0.7,
      })
      const result = await RunAnywhere.aggregateStream(prompt, events, setResponse)
      setMetrics(
        `${result.tokensPerSecond.toFixed(1)} tok/s · ${result.generationTimeMs}ms · ` +
          `${result.tokensGenerated} tokens`
      )
    } catch (error) {
      setResponse(`Error: ${(error as Error).message}`)
    } finally {
      setStreaming(false)
    }
  }, [])

  return (
    <View style={{ flex: 1, padding: 16 }}>
      <Button
        title={streaming ? 'Streaming…' : 'Start'}
        onPress={handleStream}
        disabled={streaming}
      />
      <ScrollView style={{ flex: 1, marginTop: 16 }}>
        <Text style={{ fontSize: 16, lineHeight: 24 }}>{response}</Text>
      </ScrollView>
      {metrics !== '' && <Text style={{ marginTop: 8, color: '#666' }}>{metrics}</Text>}
    </View>
  )
}

Cancelling

await RunAnywhere.cancelGeneration()

iterator.return() already calls the native cancel and awaits it, so back-to-back cancel-then-generate sequences are safe. Call cancelGeneration() directly when you hold no iterator reference, for example from a global stop button.

Throttling UI updates

Very fast generation can outrun React. Batch state writes to one animation frame.

import { useCallback, useRef, useState } from 'react'

function useThrottledText() {
  const [text, setText] = useState('')
  const pending = useRef('')
  const frame = useRef<number | undefined>(undefined)

  const set = useCallback((next: string) => {
    pending.current = next
    if (frame.current === undefined) {
      frame.current = requestAnimationFrame(() => {
        setText(pending.current)
        frame.current = undefined
      })
    }
  }, [])

  return { text, set }
}

Generate

Non-streaming generation and options

Chat

Multi-turn conversations

System Prompts

Control model behavior

Configuration

Events and logging

Getting Started

Swift SDK

Kotlin SDK

React Native SDK

Flutter SDK

Web SDK

Vibe Coding

Basic usage

LLMStreamEvent

aggregateStream

React component

Cancelling

Throttling UI updates

Generate

Chat

System Prompts

Configuration

​Basic usage

​LLMStreamEvent

​aggregateStream

​React component

​Cancelling

​Throttling UI updates

​Related

Generate

Chat

System Prompts

Configuration

Basic usage

LLMStreamEvent

aggregateStream

React component

Cancelling

Throttling UI updates

Related