> ## Documentation Index
> Fetch the complete documentation index at: https://docs.runanywhere.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Quick Start
> Install a RunAnywhere SDK, register a model, and generate text
Every SDK follows the same three steps: install, register a backend and initialize, generate.
Models are not bundled. You register a model once by URL, then name it in your generation options. The SDK downloads and loads it on first use, so there is no separate download or load step to sequence yourself.
The examples below use SmolLM2 360M (386 MB), the smallest model that produces coherent output.
### 1. Install
```kotlin build.gradle.kts theme={null}
dependencies {
implementation("io.github.sanchitmonga22:runanywhere-sdk:0.20.12")
implementation("io.github.sanchitmonga22:runanywhere-llamacpp:0.20.12")
// Add only if you need STT / TTS / VAD:
implementation("io.github.sanchitmonga22:runanywhere-onnx:0.20.12")
}
```
`mavenCentral()` is the only repository required. Minimum Android API 24.
### 2. Register a backend, then initialize
Register backends before `initialize()`. The C++ plugin registry has to know which engines exist before the first model load, or the load fails with error -422, "No provider could handle the request".
Pass the `Application` context so the SDK can reach Android file storage and the Keystore.
```kotlin theme={null}
import ai.runanywhere.proto.v1.SDKEnvironment
import com.runanywhere.sdk.llm.llamacpp.LlamaCPP
import com.runanywhere.sdk.public.RunAnywhere
import kotlinx.coroutines.*
class MyApplication : Application() {
private val scope = CoroutineScope(SupervisorJob() + Dispatchers.IO)
override fun onCreate() {
super.onCreate()
scope.launch {
LlamaCPP.register()
RunAnywhere.initialize(
context = this@MyApplication,
environment = SDKEnvironment.SDK_ENVIRONMENT_DEVELOPMENT,
)
}
}
}
```
One call is enough. Authentication, device registration, and telemetry run in the background, and local inference is usable as soon as `initialize` returns.
### 3. Register a model and generate
```kotlin theme={null}
import ai.runanywhere.proto.v1.InferenceFramework
import ai.runanywhere.proto.v1.ModelCategory
import com.runanywhere.sdk.public.api.*
RunAnywhere.models.register(
ModelRegistration.url(
id = "smollm2-360m-q8_0",
name = "SmolLM2 360M Q8_0",
url = "https://huggingface.co/prithivMLmods/SmolLM2-360M-GGUF/resolve/main/SmolLM2-360M.Q8_0.gguf",
framework = InferenceFramework.INFERENCE_FRAMEWORK_LLAMA_CPP,
category = ModelCategory.MODEL_CATEGORY_LANGUAGE,
memoryRequirement = 386_404_416L,
),
)
val result = RunAnywhere.llm.generate(
"What is the capital of France?",
LlmOptions(model = "smollm2-360m-q8_0"),
)
println(result.text)
println("${result.outputTokens} tokens at ${result.tokensPerSecond} tok/s")
```
Naming the model in `LlmOptions` downloads and loads it if needed. Later calls skip that work. To watch download progress, call `RunAnywhere.models.download(id)` yourself and collect its `Flow`.
### 4. Stream
```kotlin theme={null}
RunAnywhere.llm.generateStream("Write a haiku about rain.").collect { event ->
when (event) {
is GenerationEvent.Token -> if (event.kind == TokenKind.TEXT) print(event.text)
is GenerationEvent.Completed -> println("\n${event.result.tokensPerSecond} tok/s")
else -> {}
}
}
```
Cancel by cancelling the coroutine collecting the flow.
Installation, LLM, STT, TTS, VAD, VLM, RAG, LoRA, tool calling, voice agent
### 1. Install
Add the package in Xcode via **File → Add Package Dependencies…** with the URL
`https://github.com/RunanywhereAI/runanywhere-sdks`, or declare it in `Package.swift`:
```swift Package.swift theme={null}
dependencies: [
.package(url: "https://github.com/RunanywhereAI/runanywhere-sdks", from: "0.20.12")
],
targets: [
.target(
name: "YourApp",
dependencies: [
.product(name: "RunAnywhere", package: "runanywhere-sdks"),
.product(name: "RunAnywhereLlamaCPP", package: "runanywhere-sdks"),
]
)
]
```
Requires iOS 17.5+ or macOS 14.5+.
### 2. Register a backend, then initialize
Register backends **before** `initialize()`, or the first model load fails with "No provider could handle the request".
```swift theme={null}
import RunAnywhere
import LlamaCPPRuntime
LlamaCPP.register()
try RunAnywhere.initialize(environment: .development)
```
Pass `apiKey:` and `baseUrl:` with `environment: .production` when you have credentials. Authentication and telemetry run in the background from inside this call.
### 3. Register a model and generate
```swift theme={null}
_ = try await RunAnywhere.models.register(
.url(
id: "smollm2-360m-q8_0",
name: "SmolLM2 360M Q8_0",
url: "https://huggingface.co/prithivMLmods/SmolLM2-360M-GGUF/resolve/main/SmolLM2-360M.Q8_0.gguf",
framework: .llamaCpp,
category: .language,
memoryRequirement: 386_404_416
)
)
let result = try await RunAnywhere.llm.generate(
"What is the capital of France?",
options: LlmOptions(model: "smollm2-360m-q8_0")
)
print(result.text)
print("\(result.outputTokens) tokens at \(result.tokensPerSecond) tok/s")
```
Naming the model in `LlmOptions` downloads and loads it if needed. To show progress, iterate `RunAnywhere.models.download(id)` instead.
### 4. Stream
```swift theme={null}
for try await event in try await RunAnywhere.llm.generateStream("Write a haiku about rain.") {
switch event {
case .token(let text, let kind) where kind == .text:
print(text, terminator: "")
case .completed(let result):
print("\n\(result.tokensPerSecond) tok/s")
default:
break
}
}
```
Cancel by cancelling the enclosing `Task`. The stream throws on failure, so wrap it in `do/catch`.
Installation, LLM, STT, TTS, VAD, VLM, tool calling, voice agent
### 1. Install
The core package is `@runanywhere/core`. There is no `@runanywhere/react-native` package.
```bash theme={null}
yarn add @runanywhere/core @runanywhere/llamacpp
# Add only if you need STT / TTS / VAD:
yarn add @runanywhere/onnx
```
Peer dependencies: `react-native >= 0.83.1`, `react >= 19.0.0`,
`react-native-nitro-modules ^0.33.9`, `react-native-blob-util`, `react-native-device-info`,
`react-native-fs`. Run `pod install` in `ios/` after adding the packages.
### 2. Initialize, then register a backend
```typescript theme={null}
import { RunAnywhere, SDKEnvironment } from '@runanywhere/core'
import { LlamaCPP } from '@runanywhere/llamacpp'
await RunAnywhere.initialize({
environment: SDKEnvironment.SDK_ENVIRONMENT_DEVELOPMENT,
})
await LlamaCPP.register()
```
Backend packages export only `register`, `unregister`, and `isRegistered`.
### 3. Register a model and generate
```typescript theme={null}
import { InferenceFramework, ModelCategory } from '@runanywhere/core'
await RunAnywhere.models.register({
kind: 'url',
id: 'smollm2-360m-q8_0',
name: 'SmolLM2 360M Q8_0',
url: 'https://huggingface.co/prithivMLmods/SmolLM2-360M-GGUF/resolve/main/SmolLM2-360M.Q8_0.gguf',
framework: InferenceFramework.INFERENCE_FRAMEWORK_LLAMA_CPP,
category: ModelCategory.MODEL_CATEGORY_LANGUAGE,
memoryRequirement: 386_404_416,
})
const result = await RunAnywhere.llm.generate('What is the capital of France?', {
model: 'smollm2-360m-q8_0',
})
console.log(result.text, result.outputTokens, result.tokensPerSecond)
```
Naming the model downloads and loads it if needed.
### 4. Stream
Hermes cannot iterate a NitroModules async iterable with `for await...of`, so drive the iterator by hand. Calling `return()` cancels the native subscription.
```typescript theme={null}
const iterator = RunAnywhere.llm
.generateStream('Write a haiku about rain.')
[Symbol.asyncIterator]()
for (;;) {
const { value, done } = await iterator.next()
if (done) break
if (value.type === 'token' && value.kind === 'TEXT') process.stdout.write(value.text)
if (value.type === 'completed') console.log(value.result.tokensPerSecond)
}
```
Installation, LLM, STT, TTS, VAD, tool calling, voice agent
### 1. Install
```yaml pubspec.yaml theme={null}
dependencies:
runanywhere: ^0.20.12
runanywhere_llamacpp: ^0.20.12
# Add only if you need STT / TTS / VAD:
runanywhere_onnx: ^0.20.12
```
```bash theme={null}
flutter pub get
```
Requires Dart 3.12.0+ and Flutter 3.44.0+.
### 2. Register a backend, then initialize
```dart theme={null}
import 'package:flutter/widgets.dart';
import 'package:runanywhere/runanywhere.dart';
import 'package:runanywhere_llamacpp/runanywhere_llamacpp.dart';
Future main() async {
WidgetsFlutterBinding.ensureInitialized();
LlamaCpp.register();
await RunAnywhere.initialize(
environment: SDKEnvironment.SDK_ENVIRONMENT_DEVELOPMENT,
);
runApp(const MyApp());
}
```
`LlamaCpp.register()` is synchronous. `Onnx.register()` and `MLX.register()` return futures.
### 3. Register a model and generate
```dart theme={null}
await RunAnywhere.models.register(
ModelRegistration.url(
id: 'smollm2-360m-q8_0',
name: 'SmolLM2 360M Q8_0',
url:
'https://huggingface.co/prithivMLmods/SmolLM2-360M-GGUF/resolve/main/SmolLM2-360M.Q8_0.gguf',
framework: InferenceFramework.INFERENCE_FRAMEWORK_LLAMA_CPP,
category: ModelCategory.MODEL_CATEGORY_LANGUAGE,
memoryRequirement: 386404416,
),
);
final result = await RunAnywhere.llm.generate(
'What is the capital of France?',
options: const LlmOptions(model: 'smollm2-360m-q8_0'),
);
print('${result.text} (${result.outputTokens} tokens)');
```
### 4. Stream
```dart theme={null}
await for (final event in RunAnywhere.llm.generateStream('Write a haiku about rain.')) {
if (event is GenerationToken && event.kind == TokenKind.text) {
stdout.write(event.text);
}
}
```
Installation, LLM, STT, TTS, VAD, tool calling, voice agent
### 1. Install
```bash theme={null}
npm install @runanywhere/web @runanywhere/web-llamacpp
# Add only if you need STT / TTS / VAD:
npm install @runanywhere/web-onnx
```
Inference runs in WebAssembly. Serve your app cross-origin isolated so `SharedArrayBuffer` and threaded WASM work:
```
Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: credentialless
```
Serve `.wasm` with `Content-Type: application/wasm`. Safari does not support `credentialless`; use `require-corp` or the COI service-worker pattern. In Vite, add the packages to `optimizeDeps.exclude` and `'**/*.wasm'` to `assetsInclude`.
### 2. Initialize, then register a backend
On Web the backend packages install onto core adapters, so `initialize()` runs first. Each backend `register()` loads its own WASM binary.
```typescript theme={null}
import { RunAnywhere, SDKEnvironment } from '@runanywhere/web'
import { LlamaCPP } from '@runanywhere/web-llamacpp'
await RunAnywhere.initialize({
environment: SDKEnvironment.SDK_ENVIRONMENT_DEVELOPMENT,
})
await LlamaCPP.register({ acceleration: 'auto' })
```
`acceleration: 'auto'` picks WebGPU when the browser supports it and falls back to CPU.
### 3. Register a model and generate
```typescript theme={null}
import { InferenceFramework, ModelCategory } from '@runanywhere/web'
RunAnywhere.models.register({
kind: 'url',
id: 'smollm2-360m-q8_0',
name: 'SmolLM2 360M Q8_0',
url: 'https://huggingface.co/prithivMLmods/SmolLM2-360M-GGUF/resolve/main/SmolLM2-360M.Q8_0.gguf',
framework: InferenceFramework.INFERENCE_FRAMEWORK_LLAMA_CPP,
category: ModelCategory.MODEL_CATEGORY_LANGUAGE,
memoryRequirement: 386_404_416,
})
const result = await RunAnywhere.llm.generate({
prompt: 'What is the capital of France?',
model: 'smollm2-360m-q8_0',
})
console.log(result.text, result.tokensPerSecond)
```
Downloaded models persist in OPFS, so a reload does not re-download.
### 4. Stream
On Web the prompt is a field in the options object. The stream is a plain async iterable.
```typescript theme={null}
for await (const event of RunAnywhere.llm.generateStream({
prompt: 'Write a haiku about rain.',
})) {
if (event.type === 'token' && event.kind === 'TEXT') output.textContent += event.text
if (event.type === 'completed') console.log(event.result.tokensPerSecond)
}
```
Installation, LLM, STT, TTS, VAD, VLM, tool calling, voice agent
## What to read next
Platform requirements and which features each SDK ships
Generation options, streaming, structured output
Transcription and streaming transcription
One session that runs VAD, STT, LLM, and TTS together