Quick Start

Early Beta — The Web SDK is in early beta. APIs may change between releases.

Complete example

Four steps get you from an empty page to streamed on-device tokens: initialize the SDK, register a backend, register and download a model, generate.

runanywhere.ts

import { RunAnywhere, SDKEnvironment, ModelCategory } from '@runanywhere/web'
import { InferenceFramework, ModelFormat } from '@runanywhere/proto-ts/model_types'
import { LlamaCPP } from '@runanywhere/web-llamacpp'

export const MODEL_ID = 'lfm2-350m-q4_k_m'

let initPromise: Promise<void> | null = null

export function initSDK(): Promise<void> {
  initPromise ??= (async () => {
    // 1. Core SDK. Loads the commons WASM and starts background service init.
    await RunAnywhere.initialize({
      environment: SDKEnvironment.SDK_ENVIRONMENT_DEVELOPMENT,
    })

    // 2. Backend. Loads the llama.cpp WASM and registers its vtable.
    await LlamaCPP.register({ acceleration: 'auto' })

    // 3. Wait for the async service phase to finish.
    await RunAnywhere.completeServicesInitialization()

    // 4. Catalog. The SDK builds the ModelInfo proto from these arguments.
    RunAnywhere.registerModel(
      'https://huggingface.co/LiquidAI/LFM2-350M-GGUF/resolve/main/LFM2-350M-Q4_K_M.gguf',
      'LiquidAI LFM2 350M Q4_K_M',
      InferenceFramework.INFERENCE_FRAMEWORK_LLAMA_CPP,
      {
        id: MODEL_ID,
        description: 'LiquidAI compact LLM tuned for fast on-device chat.',
        modality: ModelCategory.MODEL_CATEGORY_LANGUAGE,
        format: ModelFormat.MODEL_FORMAT_GGUF,
        downloadSizeBytes: 229_309_376,
        memoryRequirement: 300_000_000,
        contextLength: 2048,
      }
    )
  })()

  return initPromise
}

App.tsx

import { useEffect, useState, useCallback } from 'react'
import { RunAnywhere, ModelCategory } from '@runanywhere/web'
import { initSDK, MODEL_ID } from './runanywhere'

export function App() {
  const [ready, setReady] = useState(false)
  const [progress, setProgress] = useState('')
  const [response, setResponse] = useState('')

  useEffect(() => {
    initSDK().then(() => setReady(true))
  }, [])

  const handleGenerate = useCallback(async () => {
    await RunAnywhere.downloadModel({
      modelId: MODEL_ID,
      onProgress: (p) => setProgress(`${Math.round(p.overallProgress * 100)}%`),
    })

    const load = await RunAnywhere.loadModel({
      modelId: MODEL_ID,
      category: ModelCategory.MODEL_CATEGORY_LANGUAGE,
      forceReload: false,
      validateAvailability: true,
    })
    if (!load?.success) {
      setResponse(load?.errorMessage ?? 'Model load failed')
      return
    }

    const { stream, result } = await RunAnywhere.generateStream({
      prompt: 'Explain quantum computing briefly.',
      maxTokens: 200,
      temperature: 0.7,
    })

    let text = ''
    for await (const token of stream) {
      text += token
      setResponse(text)
    }

    const metrics = await result
    console.log(`${metrics.tokensPerSecond.toFixed(1)} tok/s | ${metrics.generationTimeMs}ms`)
  }, [])

  return (
    <div>
      <button onClick={handleGenerate} disabled={!ready}>
        {ready ? 'Generate' : 'Loading SDK...'}
      </button>
      <div>{progress}</div>
      <div>{response}</div>
    </div>
  )
}

Step by step

1. Install

npm install @runanywhere/web @runanywhere/web-llamacpp

Add @runanywhere/web-onnx for speech (STT, TTS, VAD). See Installation for bundler configuration and the cross-origin isolation headers.

2. Initialize

import { RunAnywhere, SDKEnvironment } from '@runanywhere/web'
import { LlamaCPP } from '@runanywhere/web-llamacpp'

await RunAnywhere.initialize({ environment: SDKEnvironment.SDK_ENVIRONMENT_DEVELOPMENT })
await LlamaCPP.register({ acceleration: 'auto' })
await RunAnywhere.completeServicesInitialization()

initialize() is fail-closed: if the commons WASM cannot load, it throws and isInitialized stays false. It also starts the service phase in the background, so completeServicesInitialization() awaits work already in flight rather than starting it. Registering a backend after initialize() is the documented order on Web, because backend packages install onto core adapters. SDKInitOptions accepts environment, apiKey, baseURL, buildToken, appIdentifier, appName, appVersion, and appBuild. There is no debug flag; log level follows the environment, and RunAnywhere.setLogLevel() overrides it. Wrap the whole sequence in a cached promise so it is safe to call from several components.

Environments

Enum value	Wire value	Default log level
`SDKEnvironment.SDK_ENVIRONMENT_UNSPECIFIED`	0
`SDKEnvironment.SDK_ENVIRONMENT_DEVELOPMENT`	1	Debug
`SDKEnvironment.SDK_ENVIRONMENT_PRODUCTION`	3	Warning

There is no staging member. Wire value 2 was removed.

3. Register a model

Three registration entry points, picked by artifact shape:

Function	Use for
`RunAnywhere.registerModel(url, name, framework, options?)`	One file, such as a GGUF.
`RunAnywhere.registerModelArchive(url, name, framework, archiveType, options?)`	`.tar.gz` bundles, typical for STT and TTS.
`RunAnywhere.registerModelMultiFile(options)`	Several files with roles, such as a VLM’s primary GGUF plus mmproj.

Each returns the assembled ModelInfo. The app supplies flat arguments; the SDK builds the proto, the MultiFileArtifact, and the ExpectedModelFiles manifest.

4. Download and load

import { RunAnywhere, ModelCategory, DownloadStage } from '@runanywhere/web'

await RunAnywhere.downloadModel({
  modelId: 'lfm2-350m-q4_k_m',
  onProgress: (p) => console.log(DownloadStage[p.stage], Math.round(p.overallProgress * 100)),
})

await RunAnywhere.loadModel({
  modelId: 'lfm2-350m-q4_k_m',
  category: ModelCategory.MODEL_CATEGORY_LANGUAGE,
  forceReload: false,
  validateAvailability: true,
})

downloadModel also accepts a bare model id string. downloadModelStream yields DownloadProgress values instead of taking a callback. Downloads land in OPFS and survive reloads. downloadModel requires persistent browser storage; if RunAnywhere.storage.backend is 'memory', it throws ERROR_CODE_STORAGE_ERROR because independent backend WASM instances cannot share a private MEMFS.

5. Generate

const { stream, result } = await RunAnywhere.generateStream({
  prompt: 'What is 2+2?',
  maxTokens: 50,
})

let text = ''
for await (const token of stream) {
  text += token
}

const final = await result
console.log(final.text, final.tokensGenerated, final.tokensPerSecond, final.generationTimeMs)

The prompt lives inside the options object. See generate() for every option and its default.

Tracking download progress

Two options. The onProgress callback above, or the SDK event bus.

const unsubscribe = RunAnywhere.events.on('model.downloadProgress', (evt) => {
  console.log(`${evt.modelId}: ${Math.round(evt.progress * 100)}%`)
})

// later
unsubscribe()

RunAnywhere.events is the shared EventBus. Payload fields sit directly on the event object.

Checking acceleration

import { RunAnywhere } from '@runanywhere/web'
import { LlamaCPP } from '@runanywhere/web-llamacpp'

console.log(LlamaCPP.isRegistered)
console.log(LlamaCPP.accelerationMode) // 'cpu' | 'webgpu'
console.log(RunAnywhere.runtime.active) // what the SDK actually resolved

Prefer RunAnywhere.runtime.active: when a backend worker owns the GPU, the main-thread bridge mode can still read cpu.

What’s next

LLM generate

Full generation with options and metrics

Streaming

Real-time token streaming

Speech-to-text

Transcribe audio in the browser

Text-to-speech

Convert text to spoken audio

Getting Started

Swift SDK

Kotlin SDK

React Native SDK

Flutter SDK

Web SDK

Vibe Coding

Complete example

Step by step

1. Install

2. Initialize

Environments

3. Register a model

4. Download and load

5. Generate

Tracking download progress

Checking acceleration

What’s next

LLM generate

Streaming

Speech-to-text

Text-to-speech

​Complete example

​Step by step

​1. Install

​2. Initialize

​Environments

​3. Register a model

​4. Download and load

​5. Generate

​Tracking download progress

​Checking acceleration

​What’s next

LLM generate

Streaming

Speech-to-text

Text-to-speech

Complete example

Step by step

1. Install

2. Initialize

Environments

3. Register a model

4. Download and load

5. Generate

Tracking download progress

Checking acceleration

What’s next