Best Practices

Register backends before initializing

The plugin registry has to know which engines exist before the first model load. Register, then initialize.

LlamaCPP.register()
ONNX.register()
try RunAnywhere.initialize(apiKey: "", baseUrl: "", environment: .development)

Getting this backwards produces “No provider could handle the request”, which reads like a model problem and is not.

Initialize once, early

initialize() is synchronous and cheap. Call it at app launch rather than lazily on the first generation, so the background phase has time to finish before anyone needs it.

Let generation load the model

let result = try await RunAnywhere.llm.generate(
    prompt: prompt,
    options: LlmOptions(model: "qwen3-0.6b")
)

Generation auto-loads and downloads when needed. Reach for models.load when you want to control when the cost is paid, for instance warming a model behind a splash screen.

Stream anything a person waits for

A 200-token reply takes seconds. Streaming turns that into words appearing immediately.

for try await event in try await RunAnywhere.llm.generateStream(prompt: prompt) {
    if case .token(let token) = event { text += token.text }
}

Cap output length

maxOutputTokens is the single biggest lever on latency and heat. Set it to what the interface can actually display.

Use	Reasonable cap
Classification, extraction	20 to 50
Chat reply	150 to 300
Spoken reply	100
Summary	150

Unload what you are not using

Models hold memory for as long as they are resident. On a phone, holding a language model, a speech model, and a vision model at once is how you get .insufficientMemory.

try await RunAnywhere.models.unload(id: "qwen3-0.6b")
try await RunAnywhere.models.unloadAll(category: .language)

Trim conversations

History grows until it exceeds the context window. Keep the system message and the most recent turns.

if messages.count > 21 {
    messages = [messages[0]] + messages.suffix(20)
}

Keep system prompts short

Small models follow three concrete sentences better than a paragraph of persona. Say the format you want and the length you want, and nothing else.

Use structured output rather than asking nicely

Asking for JSON in a system prompt works most of the time, which is the problem. When you need to parse the result, use generateStructured, which validates and can repair.

let result = try await RunAnywhere.llm.generateStructured(
    prompt: prompt,
    schema: schema,
    mode: .repair
)

.constrained is not implemented yet and throws.

Handle cancellation

Cancel the Task when a view disappears. Otherwise a generation keeps running and keeps the model resident for a screen nobody is looking at.

Check capabilities rather than assuming

let capabilities = RunAnywhere.capabilities()

A namespace existing does not mean the engine behind it is linked in your build. MLX on the simulator is the common case: MLX.register() returns false and no MLX model will load.

Test on hardware

The simulator has no arm64 native libraries for some backends and runs no MLX inference at all. Performance numbers from a simulator are meaningless.

Watch memory around images

Image generation is the heaviest operation in the SDK. Run one at a time, show progress, and allow cancellation.

Getting Started

Swift SDK

Kotlin SDK

React Native SDK

Flutter SDK

Web SDK

Electron SDK

Vibe Coding

Register backends before initializing

Initialize once, early

Let generation load the model

Stream anything a person waits for

Cap output length

Unload what you are not using

Trim conversations

Keep system prompts short

Use structured output rather than asking nicely

Handle cancellation

Check capabilities rather than assuming

Test on hardware

Watch memory around images

​Register backends before initializing

​Initialize once, early

​Let generation load the model

​Stream anything a person waits for

​Cap output length

​Unload what you are not using

​Trim conversations

​Keep system prompts short

​Use structured output rather than asking nicely

​Handle cancellation

​Check capabilities rather than assuming

​Test on hardware

​Watch memory around images

Register backends before initializing

Initialize once, early

Let generation load the model

Stream anything a person waits for

Cap output length

Unload what you are not using

Trim conversations

Keep system prompts short

Use structured output rather than asking nicely

Handle cancellation

Check capabilities rather than assuming

Test on hardware

Watch memory around images