generateStream()

Stream tokens in real-time for a responsive user experience. Ideal for chat interfaces where users expect to see text appear progressively. generateStream returns a Stream<LLMStreamEvent> directly. It is not a future, and there is no wrapper object holding a separate token stream and metrics future.

final buffer = StringBuffer();

await for (final event in RunAnywhere.generateStream(
  'Tell me a story about a robot',
  LLMGenerationOptions(maxTokens: 500),
)) {
  buffer.write(event.token);
  // Or update UI: setState(() => _response += event.token);
  if (event.isFinal) break;
}

LLMStreamEvent

Property	Type	Description
`token`	`String`	Text delta for this event; empty on control events
`isFinal`	`bool`	True on the terminal event
`finishReason`	`String`	Why generation stopped, set on the final event
`errorMessage`	`String`	Non-empty when the backend failed
`result`	`LLMStreamFinalResult`	Terminal aggregate; guard with `hasResult()`
`kind`	`TokenKind`	Distinguishes ordinary tokens from thought tokens
`toolCall`	`ToolCall`	Set when the model emits a tool call mid-stream

LLMStreamFinalResult carries text, promptTokens, completionTokens, totalTokens, totalTimeMs, timeToFirstTokenMs, tokensPerSecond, finishReason, promptEvalTimeMs, and decodeTimeMs.

Final metrics

The terminal event usually carries an LLMStreamFinalResult. To collect the transcript and metrics in one step as an LLMGenerationResult, hand the stream to RunAnywhere.aggregateStream:

final result = await RunAnywhere.aggregateStream(
  prompt: prompt,
  events: RunAnywhere.generateStream(prompt),
  onToken: (aggregated) async {
    setState(() => _response = aggregated);
  },
);

print('Generated ${result.tokensGenerated} tokens');
print('Speed: ${result.tokensPerSecond.toStringAsFixed(1)} tok/s');
print('TTFT: ${result.ttftMs} ms');

onToken receives the full transcript so far, not the individual delta, so it drops straight into setState. It prefers the backend’s terminal aggregate when one is present and falls back to locally concatenated text and wall-clock timings otherwise.

Request-shaped streaming

For multi-turn history or a conversation cache, stream from a request proto instead:

final events = RunAnywhere.generateStreamRequest(
  LLMGenerateRequest(prompt: userInput, history: history),
);

class StreamingChatWidget extends StatefulWidget {
  @override
  _StreamingChatWidgetState createState() => _StreamingChatWidgetState();
}

class _StreamingChatWidgetState extends State<StreamingChatWidget> {
  String _response = '';
  bool _isStreaming = false;
  StreamSubscription<LLMStreamEvent>? _subscription;

  Future<void> _startStreaming(String prompt) async {
    setState(() {
      _response = '';
      _isStreaming = true;
    });

    final events = RunAnywhere.generateStream(
      prompt,
      LLMGenerationOptions(maxTokens: 300),
    );

    _subscription = events.listen(
      (event) {
        if (event.token.isNotEmpty) {
          setState(() => _response += event.token);
        }
        if (event.isFinal && event.hasResult()) {
          final r = event.result;
          debugPrint('Generated at ${r.tokensPerSecond.toStringAsFixed(1)} tok/s');
        }
      },
      onDone: () => setState(() => _isStreaming = false),
      onError: (Object e) => setState(() {
        _response = 'Error: $e';
        _isStreaming = false;
      }),
    );
  }

  void _cancelStreaming() {
    RunAnywhere.cancelGeneration();
    _subscription?.cancel();
  }

  @override
  Widget build(BuildContext context) {
    return Column(
      children: [
        Expanded(
          child: SingleChildScrollView(
            child: Text(_response),
          ),
        ),
        Row(
          children: [
            ElevatedButton(
              onPressed: _isStreaming ? null : () => _startStreaming('Tell me a joke'),
              child: Text('Generate'),
            ),
            if (_isStreaming)
              TextButton(
                onPressed: _cancelStreaming,
                child: Text('Cancel'),
              ),
          ],
        ),
      ],
    );
  }
}

Cancellation

Cancel ongoing generation at any time. cancelGeneration() is synchronous, returns void, and is a best-effort no-op when nothing is in flight.

final sub = RunAnywhere.generateStream('Long story...').listen(print);

Future.delayed(const Duration(seconds: 5), () {
  RunAnywhere.cancelGeneration();
});

Cancelling the subscription also cancels the native generation: the stream’s onCancel hook calls into the same cancel path.

await sub.cancel();

Best Practices

Use streaming for chat interfaces — Users perceive the app as more responsive when they see tokens appear progressively, even if total generation time is the same.

Update UI incrementally: Append event.token to your state as events arrive
Show a cancel button: Let users stop long generations
Handle cancellation gracefully: The stream completes when cancelled
Get final metrics: Read event.result on the terminal event, or use aggregateStream

Generation starts when the first listener attaches, not when generateStream is called, so no tokens can be produced before your subscriber is ready.

chat()

Simple one-liner

generate()

Non-streaming with metrics

Getting Started

Swift SDK

Kotlin SDK

React Native SDK

Flutter SDK

Web SDK

Vibe Coding

LLMStreamEvent

Final metrics

Request-shaped streaming

Flutter Widget Example

Cancellation

Best Practices

See Also

chat()

generate()

​LLMStreamEvent

​Final metrics

​Request-shaped streaming

​Flutter Widget Example

​Cancellation

​Best Practices

​See Also

chat()

generate()

LLMStreamEvent

Final metrics

Request-shaped streaming

Flutter Widget Example

Cancellation

Best Practices

See Also