Quick Start

Complete example

Application.kt

import ai.runanywhere.proto.v1.InferenceFramework
import ai.runanywhere.proto.v1.ModelCategory
import ai.runanywhere.proto.v1.SDKEnvironment
import com.runanywhere.sdk.core.onnx.ONNX
import com.runanywhere.sdk.llm.llamacpp.LlamaCPP
import com.runanywhere.sdk.npu.qhexrt.QHexRT
import com.runanywhere.sdk.public.RunAnywhere
import com.runanywhere.sdk.public.extensions.downloadModelStream
import com.runanywhere.sdk.public.extensions.generate
import com.runanywhere.sdk.public.extensions.loadModel
import com.runanywhere.sdk.public.extensions.refreshModelRegistry
import com.runanywhere.sdk.public.extensions.registerModel
import com.runanywhere.sdk.public.types.RAModelLoadRequest

class MyApplication : Application() {
    private val scope = CoroutineScope(SupervisorJob() + Dispatchers.IO)

    override fun onCreate() {
        super.onCreate()
        scope.launch { setupSdk() }
    }

    private suspend fun setupSdk() {
        // 1. Register the CPU backends BEFORE initialize().
        runCatching { LlamaCPP.register() }
            .onFailure { Log.w("SDK", "llama.cpp backend unavailable: ${it.message}") }
        runCatching { ONNX.register() }
            .onFailure { Log.w("SDK", "ONNX/Sherpa backend unavailable: ${it.message}") }

        // 2. Initialize. The Context overload installs the platform context for you.
        RunAnywhere.initialize(
            context = this@MyApplication,
            environment = SDKEnvironment.SDK_ENVIRONMENT_DEVELOPMENT,
        )

        // 3. Register the NPU backend AFTER initialize().
        QHexRT.register()

        // 4. Seed the model registry, then refresh it.
        val model = RunAnywhere.registerModel(
            id = "smollm2-360m-instruct-q8_0",
            name = "SmolLM2 360M Instruct Q8_0",
            url = "https://huggingface.co/.../SmolLM2-360M-Instruct-Q8_0.gguf",
            framework = InferenceFramework.INFERENCE_FRAMEWORK_LLAMA_CPP,
            modality = ModelCategory.MODEL_CATEGORY_LANGUAGE,
            memoryRequirement = 400_000_000L,
        )
        RunAnywhere.refreshModelRegistry()

        // 5. Download, load, generate.
        RunAnywhere.downloadModelStream(model).collect { progress -> report(progress) }
        RunAnywhere.loadModel(RAModelLoadRequest(model_id = model.id))

        val response = RunAnywhere.generate("What is 2+2?").text
        Log.i("SDK", response)
    }
}

Step by step

1. Initialize

The Android overload of initialize() takes a Context and installs the platform context (secure storage, model path resolution) before phase 1 starts.

RunAnywhere.initialize(
    context = applicationContext,
    environment = SDKEnvironment.SDK_ENVIRONMENT_DEVELOPMENT,
)

You do not call AndroidPlatformContext.initialize() or CppBridgeModelPaths.setBaseDirectory(). Both are SDK-internal. initialize() is synchronous and returns in single-digit milliseconds. Phase 2 (authentication, model assignments, device registration) runs in the background and is awaited automatically on first feature use. For production, pass credentials:

RunAnywhere.initialize(
    context = applicationContext,
    apiKey = BuildConfig.RUNANYWHERE_API_KEY,
    baseURL = BuildConfig.RUNANYWHERE_BASE_URL,
    environment = SDKEnvironment.SDK_ENVIRONMENT_PRODUCTION,
)

Environment options

SDKEnvironment is the Wire-generated proto enum ai.runanywhere.proto.v1.SDKEnvironment. Values are fully qualified: SDKEnvironment.SDK_ENVIRONMENT_DEVELOPMENT, not SDKEnvironment.DEVELOPMENT.

Environment	Behavior
`SDK_ENVIRONMENT_DEVELOPMENT`	Debug logging, no api key required
`SDK_ENVIRONMENT_PRODUCTION`	Production backend, minimal logging

The only other value is SDK_ENVIRONMENT_UNSPECIFIED. There is no staging value.

2. Register backends

Order matters. Register llama.cpp and ONNX before initialize(): once initialization returns, a concurrent caller can reach loadModel() while only the platform backend is registered, and the request fails with “no provider could handle the request”. Register QHexRT after initialize(), because its DSP skel installer needs the application context the SDK holds.

runCatching { LlamaCPP.register() }
runCatching { ONNX.register() }

RunAnywhere.initialize(context = app, environment = environment)

QHexRT.register()

register() is a suspend function on all three modules and takes no arguments. There is no priority parameter. Wrap the CPU backends in runCatching: a build that ships only some native libraries throws on the missing one, and an unguarded call aborts SDK setup. QHexRT.register() rejects itself on unsupported parts and logs instead of throwing.

3. Register models

registerModel() is a suspend function with three overloads. Single file:

val llm = RunAnywhere.registerModel(
    id = "smollm2-360m-instruct-q8_0",
    name = "SmolLM2 360M Instruct Q8_0",
    url = "https://huggingface.co/.../SmolLM2-360M-Instruct-Q8_0.gguf",
    framework = InferenceFramework.INFERENCE_FRAMEWORK_LLAMA_CPP,
    modality = ModelCategory.MODEL_CATEGORY_LANGUAGE,
    memoryRequirement = 400_000_000L,
)

Archive, for Sherpa speech models that ship as .tar.gz:

val stt = RunAnywhere.registerModel(
    id = "sherpa-onnx-whisper-tiny.en",
    name = "Sherpa Whisper Tiny (ONNX)",
    archiveUrl = "https://.../sherpa-onnx-whisper-tiny.en.tar.gz",
    framework = InferenceFramework.INFERENCE_FRAMEWORK_SHERPA,
    modality = ModelCategory.MODEL_CATEGORY_SPEECH_RECOGNITION,
    archiveType = ArchiveType.ARCHIVE_TYPE_TAR_GZ,
    structure = ArchiveStructure.ARCHIVE_STRUCTURE_NESTED_DIRECTORY,
)

Multi-file, for VLMs that need a model GGUF plus an mmproj projector:

val vlm = RunAnywhere.registerModel(
    multiFile = listOf(
        ModelFileDescriptor(url = "https://...SmolVLM2-256M-Video-Instruct-Q8_0.gguf", filename = "SmolVLM2-256M-Video-Instruct-Q8_0.gguf"),
        ModelFileDescriptor(url = "https://...mmproj-SmolVLM2-256M-Video-Instruct-Q8_0.gguf", filename = "mmproj-SmolVLM2-256M-Video-Instruct-Q8_0.gguf"),
    ),
    id = "smolvlm2-256m-video-instruct-q8_0",
    name = "SmolVLM2 256M Video Instruct Q8_0",
    framework = InferenceFramework.INFERENCE_FRAMEWORK_LLAMA_CPP,
    modality = ModelCategory.MODEL_CATEGORY_MULTIMODAL,
    memoryRequirement = 450_000_000L,
)

There is no registerMultiFileModel; the multi-file form is an overload of registerModel. Registration is idempotent. The registry merges on re-save and preserves runtime fields such as is_downloaded and per-file local paths, so a catalog can be re-seeded on every launch. Call RunAnywhere.refreshModelRegistry() afterwards so a local rescan picks up what is already on disk.

4. Download and load

RunAnywhere.downloadModelStream(llm).collect { progress ->
    val percent = if (progress.total_bytes > 0) {
        (progress.bytes_downloaded * 100 / progress.total_bytes).toInt()
    } else {
        (progress.stage_progress.coerceIn(0f, 1f) * 100).toInt()
    }
    updateProgress(percent)
}

downloadModelStream(model: RAModelInfo) returns Flow<DownloadProgress>. The suspend form is downloadModel(model, onProgress), which returns the terminal progress. Both take a ModelInfo, not a model id string. total_bytes is 0 until the planner has sized the artifact. loadModel() is one entry point for every modality. Category is inferred from the registry entry and is only needed to disambiguate a model registered under more than one:

RunAnywhere.loadModel(RAModelLoadRequest(model_id = "smollm2-360m-instruct-q8_0"))
RunAnywhere.loadModel(RAModelLoadRequest(model_id = "sherpa-onnx-whisper-tiny.en"))
RunAnywhere.loadModel(RAModelLoadRequest(model_id = "vits-piper-en_US-lessac-medium"))
RunAnywhere.loadModel(RAModelLoadRequest(model_id = "smolvlm2-256m-video-instruct-q8_0"))

RunAnywhere.loadModel(
    RAModelLoadRequest(
        model_id = "sherpa-onnx-whisper-tiny.en",
        category = ModelCategory.MODEL_CATEGORY_SPEECH_RECOGNITION,
    ),
)

loadModel() returns a ModelLoadResult with a success flag rather than throwing:

val result = RunAnywhere.loadModel(request)
if (!result.success) showError(result.error_message)

5. Run inference

// Text generation
val text = RunAnywhere.generate("What is 2+2?").text

// Transcription, from 16 kHz mono 16-bit PCM bytes
val transcript = RunAnywhere.transcribe(audioBytes).text

// Speech synthesis and playback
RunAnywhere.speak("Hello!", RATTSOptions(language_code = "en-US"))

// Synthesis to a buffer
val output = RunAnywhere.synthesize("Hello!", RATTSOptions(language_code = "en-US"))
val pcm = output.audio_data.toByteArray()

generate() returns an LLMGenerationResult and transcribe() an STTOutput, so both need .text. Proto field names are snake_case throughout.

6. Stream

RunAnywhere.generateStream("Tell me a story about AI").collect { event ->
    if (event.token.isNotEmpty()) textView.append(event.token)
}

generateStream() emits LLMStreamEvent, not String, and is not a suspend function. See Streaming.

What’s next?

LLM chat

Multi-turn conversations

Streaming

Token streaming with Flows

Speech-to-text

Transcribe audio to text

Text-to-speech

Convert text to spoken audio

Getting Started

Swift SDK

Kotlin SDK

React Native SDK

Flutter SDK

Web SDK

Vibe Coding

Complete example

Step by step

1. Initialize

Environment options

2. Register backends

3. Register models

4. Download and load

5. Run inference

6. Stream

What’s next?

LLM chat

Streaming

Speech-to-text

Text-to-speech

​Complete example

​Step by step

​1. Initialize

​Environment options

​2. Register backends

​3. Register models

​4. Download and load

​5. Run inference

​6. Stream

​What’s next?

LLM chat

Streaming

Speech-to-text

Text-to-speech

Complete example

Step by step

1. Initialize

Environment options

2. Register backends

3. Register models

4. Download and load

5. Run inference

6. Stream

What’s next?