Quick Start

Every SDK follows the same three steps: install, register a backend and initialize, generate. Models are not bundled. You register a model once by URL, then name it in your generation options. The SDK downloads and loads it on first use, so there is no separate download or load step to sequence yourself. The examples below use SmolLM2 360M (386 MB), the smallest model that produces coherent output.

Kotlin
Swift
React Native
Flutter
Web

1. Install

build.gradle.kts

dependencies {
    implementation("io.github.sanchitmonga22:runanywhere-sdk:0.20.12")
    implementation("io.github.sanchitmonga22:runanywhere-llamacpp:0.20.12")
    // Add only if you need STT / TTS / VAD:
    implementation("io.github.sanchitmonga22:runanywhere-onnx:0.20.12")
}

mavenCentral() is the only repository required. Minimum Android API 24.

2. Register a backend, then initialize

Register backends before initialize(). The C++ plugin registry has to know which engines exist before the first model load, or the load fails with error -422, “No provider could handle the request”.Pass the Application context so the SDK can reach Android file storage and the Keystore.

import ai.runanywhere.proto.v1.SDKEnvironment
import com.runanywhere.sdk.llm.llamacpp.LlamaCPP
import com.runanywhere.sdk.public.RunAnywhere
import kotlinx.coroutines.*

class MyApplication : Application() {
    private val scope = CoroutineScope(SupervisorJob() + Dispatchers.IO)

    override fun onCreate() {
        super.onCreate()
        scope.launch {
            LlamaCPP.register()
            RunAnywhere.initialize(
                context = this@MyApplication,
                environment = SDKEnvironment.SDK_ENVIRONMENT_DEVELOPMENT,
            )
        }
    }
}

One call is enough. Authentication, device registration, and telemetry run in the background, and local inference is usable as soon as initialize returns.

3. Register a model and generate

import ai.runanywhere.proto.v1.InferenceFramework
import ai.runanywhere.proto.v1.ModelCategory
import com.runanywhere.sdk.public.api.*

RunAnywhere.models.register(
    ModelRegistration.url(
        id = "smollm2-360m-q8_0",
        name = "SmolLM2 360M Q8_0",
        url = "https://huggingface.co/prithivMLmods/SmolLM2-360M-GGUF/resolve/main/SmolLM2-360M.Q8_0.gguf",
        framework = InferenceFramework.INFERENCE_FRAMEWORK_LLAMA_CPP,
        category = ModelCategory.MODEL_CATEGORY_LANGUAGE,
        memoryRequirement = 386_404_416L,
    ),
)

val result = RunAnywhere.llm.generate(
    "What is the capital of France?",
    LlmOptions(model = "smollm2-360m-q8_0"),
)
println(result.text)
println("${result.outputTokens} tokens at ${result.tokensPerSecond} tok/s")

Naming the model in LlmOptions downloads and loads it if needed. Later calls skip that work. To watch download progress, call RunAnywhere.models.download(id) yourself and collect its Flow<DownloadEvent>.

4. Stream

RunAnywhere.llm.generateStream("Write a haiku about rain.").collect { event ->
    when (event) {
        is GenerationEvent.Token -> if (event.kind == TokenKind.TEXT) print(event.text)
        is GenerationEvent.Completed -> println("\n${event.result.tokensPerSecond} tok/s")
        else -> {}
    }
}

Cancel by cancelling the coroutine collecting the flow.

Full Kotlin documentation

Installation, LLM, STT, TTS, VAD, VLM, RAG, LoRA, tool calling, voice agent

1. Install

Add the package in Xcode via File → Add Package Dependencies… with the URL https://github.com/RunanywhereAI/runanywhere-sdks, or declare it in Package.swift:

Package.swift

dependencies: [
    .package(url: "https://github.com/RunanywhereAI/runanywhere-sdks", from: "0.20.12")
],
targets: [
    .target(
        name: "YourApp",
        dependencies: [
            .product(name: "RunAnywhere", package: "runanywhere-sdks"),
            .product(name: "RunAnywhereLlamaCPP", package: "runanywhere-sdks"),
        ]
    )
]

Requires iOS 17.5+ or macOS 14.5+.

2. Register a backend, then initialize

import RunAnywhere
import LlamaCPPRuntime

LlamaCPP.register()
try RunAnywhere.initialize(environment: .development)

Pass apiKey: and baseUrl: with environment: .production when you have credentials. Authentication and telemetry run in the background from inside this call.

3. Register a model and generate

_ = try await RunAnywhere.models.register(
    .url(
        id: "smollm2-360m-q8_0",
        name: "SmolLM2 360M Q8_0",
        url: "https://huggingface.co/prithivMLmods/SmolLM2-360M-GGUF/resolve/main/SmolLM2-360M.Q8_0.gguf",
        framework: .llamaCpp,
        category: .language,
        memoryRequirement: 386_404_416
    )
)

let result = try await RunAnywhere.llm.generate(
    "What is the capital of France?",
    options: LlmOptions(model: "smollm2-360m-q8_0")
)
print(result.text)
print("\(result.outputTokens) tokens at \(result.tokensPerSecond) tok/s")

Naming the model in LlmOptions downloads and loads it if needed. To show progress, iterate RunAnywhere.models.download(id) instead.

4. Stream

for try await event in try await RunAnywhere.llm.generateStream("Write a haiku about rain.") {
    switch event {
    case .token(let text, let kind) where kind == .text:
        print(text, terminator: "")
    case .completed(let result):
        print("\n\(result.tokensPerSecond) tok/s")
    default:
        break
    }
}

Cancel by cancelling the enclosing Task. The stream throws on failure, so wrap it in do/catch.

Full Swift documentation

Installation, LLM, STT, TTS, VAD, VLM, tool calling, voice agent

1. Install

The core package is @runanywhere/core. There is no @runanywhere/react-native package.

yarn add @runanywhere/core @runanywhere/llamacpp
# Add only if you need STT / TTS / VAD:
yarn add @runanywhere/onnx

Peer dependencies: react-native >= 0.83.1, react >= 19.0.0, react-native-nitro-modules ^0.33.9, react-native-blob-util, react-native-device-info, react-native-fs. Run pod install in ios/ after adding the packages.

2. Initialize, then register a backend

import { RunAnywhere, SDKEnvironment } from '@runanywhere/core'
import { LlamaCPP } from '@runanywhere/llamacpp'

await RunAnywhere.initialize({
  environment: SDKEnvironment.SDK_ENVIRONMENT_DEVELOPMENT,
})
await LlamaCPP.register()

Backend packages export only register, unregister, and isRegistered.

3. Register a model and generate

import { InferenceFramework, ModelCategory } from '@runanywhere/core'

await RunAnywhere.models.register({
  kind: 'url',
  id: 'smollm2-360m-q8_0',
  name: 'SmolLM2 360M Q8_0',
  url: 'https://huggingface.co/prithivMLmods/SmolLM2-360M-GGUF/resolve/main/SmolLM2-360M.Q8_0.gguf',
  framework: InferenceFramework.INFERENCE_FRAMEWORK_LLAMA_CPP,
  category: ModelCategory.MODEL_CATEGORY_LANGUAGE,
  memoryRequirement: 386_404_416,
})

const result = await RunAnywhere.llm.generate('What is the capital of France?', {
  model: 'smollm2-360m-q8_0',
})
console.log(result.text, result.outputTokens, result.tokensPerSecond)

Naming the model downloads and loads it if needed.

4. Stream

Hermes cannot iterate a NitroModules async iterable with for await...of, so drive the iterator by hand. Calling return() cancels the native subscription.

const iterator = RunAnywhere.llm
  .generateStream('Write a haiku about rain.')
  [Symbol.asyncIterator]()

for (;;) {
  const { value, done } = await iterator.next()
  if (done) break
  if (value.type === 'token' && value.kind === 'TEXT') process.stdout.write(value.text)
  if (value.type === 'completed') console.log(value.result.tokensPerSecond)
}

Full React Native documentation

Installation, LLM, STT, TTS, VAD, tool calling, voice agent

1. Install

pubspec.yaml

dependencies:
  runanywhere: ^0.20.12
  runanywhere_llamacpp: ^0.20.12
  # Add only if you need STT / TTS / VAD:
  runanywhere_onnx: ^0.20.12

flutter pub get

Requires Dart 3.12.0+ and Flutter 3.44.0+.

2. Register a backend, then initialize

import 'package:flutter/widgets.dart';
import 'package:runanywhere/runanywhere.dart';
import 'package:runanywhere_llamacpp/runanywhere_llamacpp.dart';

Future<void> main() async {
  WidgetsFlutterBinding.ensureInitialized();

  LlamaCpp.register();

  await RunAnywhere.initialize(
    environment: SDKEnvironment.SDK_ENVIRONMENT_DEVELOPMENT,
  );

  runApp(const MyApp());
}

LlamaCpp.register() is synchronous. Onnx.register() and MLX.register() return futures.

3. Register a model and generate

await RunAnywhere.models.register(
  ModelRegistration.url(
    id: 'smollm2-360m-q8_0',
    name: 'SmolLM2 360M Q8_0',
    url:
        'https://huggingface.co/prithivMLmods/SmolLM2-360M-GGUF/resolve/main/SmolLM2-360M.Q8_0.gguf',
    framework: InferenceFramework.INFERENCE_FRAMEWORK_LLAMA_CPP,
    category: ModelCategory.MODEL_CATEGORY_LANGUAGE,
    memoryRequirement: 386404416,
  ),
);

final result = await RunAnywhere.llm.generate(
  'What is the capital of France?',
  options: const LlmOptions(model: 'smollm2-360m-q8_0'),
);
print('${result.text} (${result.outputTokens} tokens)');

4. Stream

await for (final event in RunAnywhere.llm.generateStream('Write a haiku about rain.')) {
  if (event is GenerationToken && event.kind == TokenKind.text) {
    stdout.write(event.text);
  }
}

Full Flutter documentation

Installation, LLM, STT, TTS, VAD, tool calling, voice agent

1. Install

npm install @runanywhere/web @runanywhere/web-llamacpp
# Add only if you need STT / TTS / VAD:
npm install @runanywhere/web-onnx

Inference runs in WebAssembly. Serve your app cross-origin isolated so SharedArrayBuffer and threaded WASM work:

Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: credentialless

Serve .wasm with Content-Type: application/wasm. Safari does not support credentialless; use require-corp or the COI service-worker pattern. In Vite, add the packages to optimizeDeps.exclude and '**/*.wasm' to assetsInclude.

2. Initialize, then register a backend

On Web the backend packages install onto core adapters, so initialize() runs first. Each backend register() loads its own WASM binary.

import { RunAnywhere, SDKEnvironment } from '@runanywhere/web'
import { LlamaCPP } from '@runanywhere/web-llamacpp'

await RunAnywhere.initialize({
  environment: SDKEnvironment.SDK_ENVIRONMENT_DEVELOPMENT,
})
await LlamaCPP.register({ acceleration: 'auto' })

acceleration: 'auto' picks WebGPU when the browser supports it and falls back to CPU.

3. Register a model and generate

import { InferenceFramework, ModelCategory } from '@runanywhere/web'

RunAnywhere.models.register({
  kind: 'url',
  id: 'smollm2-360m-q8_0',
  name: 'SmolLM2 360M Q8_0',
  url: 'https://huggingface.co/prithivMLmods/SmolLM2-360M-GGUF/resolve/main/SmolLM2-360M.Q8_0.gguf',
  framework: InferenceFramework.INFERENCE_FRAMEWORK_LLAMA_CPP,
  category: ModelCategory.MODEL_CATEGORY_LANGUAGE,
  memoryRequirement: 386_404_416,
})

const result = await RunAnywhere.llm.generate({
  prompt: 'What is the capital of France?',
  model: 'smollm2-360m-q8_0',
})
console.log(result.text, result.tokensPerSecond)

Downloaded models persist in OPFS, so a reload does not re-download.

4. Stream

On Web the prompt is a field in the options object. The stream is a plain async iterable.

for await (const event of RunAnywhere.llm.generateStream({
  prompt: 'Write a haiku about rain.',
})) {
  if (event.type === 'token' && event.kind === 'TEXT') output.textContent += event.text
  if (event.type === 'completed') console.log(event.result.tokensPerSecond)
}

Full Web documentation

Installation, LLM, STT, TTS, VAD, VLM, tool calling, voice agent

SDK overview

Platform requirements and which features each SDK ships

Text generation

Generation options, streaming, structured output

Speech-to-text

Transcription and streaming transcription

Voice agent

One session that runs VAD, STT, LLM, and TTS together

Getting Started

Swift SDK

Kotlin SDK

React Native SDK

Flutter SDK

Web SDK

Vibe Coding

1. Install

2. Register a backend, then initialize

3. Register a model and generate

4. Stream

Full Kotlin documentation

1. Install

2. Register a backend, then initialize

3. Register a model and generate

4. Stream

Full Swift documentation

1. Install

2. Initialize, then register a backend

3. Register a model and generate

4. Stream

Full React Native documentation

1. Install

2. Register a backend, then initialize

3. Register a model and generate

4. Stream

Full Flutter documentation

1. Install

2. Initialize, then register a backend

3. Register a model and generate

4. Stream

Full Web documentation

What to read next

SDK overview

Text generation

Speech-to-text

Voice agent

​1. Install

​2. Register a backend, then initialize

​3. Register a model and generate

​4. Stream

Full Kotlin documentation

​1. Install

​2. Register a backend, then initialize

​3. Register a model and generate

​4. Stream

Full Swift documentation

​1. Install

​2. Initialize, then register a backend

​3. Register a model and generate

​4. Stream

Full React Native documentation

​1. Install

​2. Register a backend, then initialize

​3. Register a model and generate

​4. Stream

Full Flutter documentation

​1. Install

​2. Initialize, then register a backend

​3. Register a model and generate

​4. Stream

Full Web documentation

​What to read next

SDK overview

Text generation

Speech-to-text

Voice agent

1. Install

2. Register a backend, then initialize

3. Register a model and generate

4. Stream

1. Install

2. Register a backend, then initialize

3. Register a model and generate

4. Stream

1. Install

2. Initialize, then register a backend

3. Register a model and generate

4. Stream

1. Install

2. Register a backend, then initialize

3. Register a model and generate

4. Stream

1. Install

2. Initialize, then register a backend

3. Register a model and generate

4. Stream

What to read next