generate()

generate runs one non-streaming generation and resolves with the full proto result.

import { RunAnywhere } from '@runanywhere/core'

const result = await RunAnywhere.generate('Explain quantum computing in simple terms', {
  maxTokens: 200,
  temperature: 0.7,
})

console.log(result.text)
console.log(result.tokensGenerated, 'tokens at', result.tokensPerSecond, 'tok/s')
console.log(result.generationTimeMs, 'ms')

Signature

generate is overloaded. Pass a prompt string with optional options, or a whole LLMGenerateRequest.

generate(prompt: string, options?: LLMGenerationOptions): Promise<LLMGenerationResult>
generate(request: LLMGenerateRequest): Promise<LLMGenerationResult>

Both types are generated: LLMGenerationOptions and LLMGenerationResult from @runanywhere/proto-ts/llm_options, LLMGenerateRequest from @runanywhere/proto-ts/llm_service. They are not re-exported from @runanywhere/core. It throws SDKException when the SDK is not initialized, when the native module is missing, or when the backend fails.

Options

The sampling defaults below are declared once in the IDL and read by every SDK. A field left unset gets the default; a field explicitly set to the proto zero value means “let the engine decide”.

Field	Type	Default	Notes
`maxTokens`	`number`	`100`
`temperature`	`number`	`0.8`	0.0 to 2.0; 0.0 is greedy decoding
`topP`	`number`	`1.0`	1.0 disables nucleus truncation
`topK`	`number`	`0`	0 disables top-k
`repetitionPenalty`	`number`	`1.0`	1.0 is no penalty
`stopSequences`	`string[]`	`[]`
`systemPrompt`	`string`	unset
`seed`	`number`	`0`	0 is a backend-chosen random seed
`frequencyPenalty`	`number`	`0.0`
`presencePenalty`	`number`	`0.0`
`repeatLastN`	`number`	`0`	0 is the backend default lookback
`minP`	`number`	`0.0`	0.0 disables
`nThreads`	`number`	`0`	0 is the runtime default
`grammar`	`string`	unset	GBNF or backend-specific rule text
`jsonSchema`	`string`	unset	Superseded by `structuredOutput` when both are set
`disableThinking`	`boolean`	`false`	Commons applies the model’s no-think directive
`preferredFramework`	`InferenceFramework`	unset	Unset picks automatically
`executionTarget`	`ExecutionTarget`	unset
`thinkingPattern`	`ThinkingTagPattern`	unset	Setting it also turns on thought emission

Result

LLMGenerationResult fields you are most likely to read:

Field	Type	Meaning
`text`	`string`	Generated text, thinking content removed
`thinkingContent`	`string \| undefined`	Extracted reasoning, when the model emits any
`inputTokens`	`number`	Prompt tokens
`tokensGenerated`	`number`	Completion tokens
`totalTokens`	`number`	Prompt plus completion
`responseTokens`	`number`	Tokens in the response as opposed to thinking
`thinkingTokens`	`number`	0 when not applicable
`modelUsed`	`string`	Model id
`generationTimeMs`	`number`	Wall clock
`tokensPerSecond`	`number`
`ttftMs`	`number \| undefined`	Time to first token, streaming only
`promptEvalTimeMs`	`number`	Prefill time when the backend reports it
`decodeTimeMs`	`number`	Decode time when the backend reports it
`finishReason`	`string`	`stop`, `length`, `cancelled`, `error`, or empty
`framework`	`string \| undefined`	Framework that ran the generation
`jsonOutput`	`string \| undefined`	Set for structured-output requests
`errorMessage`	`string \| undefined`

There is no latencyMs, tokensUsed, or nested performanceMetrics object on this result.

Temperature

Temperature	Use case
0.0 to 0.3	Factual, deterministic responses
0.4 to 0.7	Balanced, general purpose
0.8 to 1.2	Creative, varied output
1.3 to 2.0	Experimental

const factual = await RunAnywhere.generate('What is the boiling point of water?', {
  temperature: 0.1,
  maxTokens: 50,
})

Stop sequences

const result = await RunAnywhere.generate('List 3 fruits:', {
  maxTokens: 100,
  stopSequences: ['4.', '\n\n'],
})

System prompt

const result = await RunAnywhere.generate('What is the best programming language?', {
  maxTokens: 200,
  systemPrompt: 'You are a helpful coding assistant. Be concise and practical.',
})

See System Prompts.

Thinking models

Models that emit <think> blocks (Qwen3, LFM2, DeepSeek-R1) put the reasoning trace in thinkingContent and the answer in text. Register the model with supportsThinking: true so the registry records the capability.

const result = await RunAnywhere.generate('Solve step by step: what is 15% of 240?', {
  maxTokens: 500,
})

console.log(result.thinkingContent)
console.log(result.text)

Pass disableThinking: true to suppress the thinking phase for one call. Commons injects the model’s own no-think directive at the prompt level, so do not prepend /no_think yourself.

Cancellation

const promise = RunAnywhere.generate('Write a long story…', { maxTokens: 1000 })

setTimeout(() => {
  void RunAnywhere.cancelGeneration()
}, 2000)

cancelGeneration() is a no-op when the SDK is not initialized. A cancelled generation surfaces finishReason: 'cancelled', and isSDKException(e) && e.isExpected is true for the corresponding error code, so you can skip crash reporting on it.

Chat

Multi-turn conversations

Streaming

Token streaming

System Prompts

Control model behavior

Error Handling

SDKException reference

Getting Started

Swift SDK

Kotlin SDK

React Native SDK

Flutter SDK

Web SDK

Vibe Coding

Signature

Options

Result

Temperature

Stop sequences

System prompt

Thinking models

Cancellation

Chat

Streaming

System Prompts

Error Handling

​Signature

​Options

​Result

​Temperature

​Stop sequences

​System prompt

​Thinking models

​Cancellation

​Related

Chat

Streaming

System Prompts

Error Handling

Signature

Options

Result

Temperature

Stop sequences

System prompt

Thinking models

Cancellation

Related