Models

RunAnywhere.models is the model catalog and its residency.

The short version

Generation loads what it needs and downloads when options.model names a model that is not on disk:

const result = await RunAnywhere.llm.generate('Hello', { model: 'qwen3-0.6b' })

Reach for the models namespace to control when that cost is paid, show progress, or let a person choose.

What is available

const models = await RunAnywhere.models.list()
const language = await RunAnywhere.models.list({
  category: ModelCategory.MODEL_CATEGORY_LANGUAGE,
})

const resident = await RunAnywhere.models.loaded(ModelCategory.MODEL_CATEGORY_LANGUAGE)

Downloading, with progress

for await (const event of RunAnywhere.models.download('qwen3-0.6b')) {
  if (event.type === 'progress') {
    setFraction(event.fraction ?? 0)
    setSpeed(event.bytesPerSecond)
  }
}

Progress carries bytesDone, bytesTotal, fraction, percent, bytesPerSecond, etaSeconds, retryAttempt, currentFileIndex, totalFiles, and overallProgress. Downloads run through the SDK’s native download bridge, not a JavaScript filesystem library, so they survive a JS reload. What does not survive is your for await loop: after a fast-refresh the download keeps going but nothing is listening. Re-attach on mount rather than assuming a fresh start. Show the size first. On cellular, a 2GB model is the user’s money.

Loading

const loaded = await RunAnywhere.models.load('qwen3-0.6b')

const tuned = await RunAnywhere.models.load('qwen3-0.6b', {
  contextLength: 4096,
  threads: 4,
})

Field	Meaning
`backendPreferences`	Ordered preference; each can be required
`accelerator`	Accelerator policy
`contextLength`	Context window to allocate
`threads`	CPU threads
`forceReload`	Reload even if already resident
`framework`	Pin to one backend
`useGpu`	Request GPU execution

contextLength costs memory whether you use it or not.

MLX and QHexRT

@runanywhere/mlx registers only on physical iOS hardware. @runanywhere/qhexrt is Android arm64 only. Pinning framework to either on the wrong device throws rather than falling back, which is what you want when measuring one engine. Note that capabilities() on React Native is a static literal and lists only llama.cpp and ONNX, even when MLX or QHexRT are installed. Do not use it to decide whether a backend is present.

Switching models

let current: string | null = null

async function use(id: string): Promise<void> {
  if (current && current !== id) {
    await RunAnywhere.models.unload(current)
  }
  await RunAnywhere.models.load(id)
  current = id
}

On a phone, unload before loading. Holding two language models is how you get insufficientMemory. Or switch per call, without touching residency:

await RunAnywhere.llm.generate(prompt, { model: 'qwen3-0.6b' })
await RunAnywhere.llm.generate(prompt, { model: 'llama-3.2-1b' })

Unloading

await RunAnywhere.models.unload('qwen3-0.6b')
await RunAnywhere.models.unloadAll()

Do this when the app backgrounds, not only on unmount:

AppState.addEventListener('change', (state) => {
  if (state !== 'active') void RunAnywhere.models.unloadAll()
})

What is resident right now

const state = await RunAnywhere.models.state()

Deleting

await RunAnywhere.models.delete('qwen3-0.6b')
await RunAnywhere.models.unregister('qwen3-0.6b')

delete removes the bytes. unregister removes the catalog entry.

Registering your own model

const info = await RunAnywhere.models.register({
  id: 'my-model',
  name: 'My model',
  url: 'https://example.com/model.gguf',
  framework: InferenceFramework.INFERENCE_FRAMEWORK_LLAMA_CPP,
  category: ModelCategory.MODEL_CATEGORY_LANGUAGE,
})

Storage

const info = await RunAnywhere.storage.info() // StorageInfo | null
await RunAnywhere.storage.clearCache()
await RunAnywhere.storage.cleanTempFiles()

storage.info() reports device, app, and per-model usage, which is what a “manage storage” screen renders.

A complete picker

import { RunAnywhere, ModelCategory } from '@runanywhere/core'
import type { ModelInfo } from '@runanywhere/core'
import { useCallback, useEffect, useState } from 'react'

export function useModelPicker() {
  const [models, setModels] = useState<ModelInfo[]>([])
  const [downloading, setDownloading] = useState<string | null>(null)
  const [fraction, setFraction] = useState(0)

  const refresh = useCallback(async () => {
    setModels(
      await RunAnywhere.models.list({
        category: ModelCategory.MODEL_CATEGORY_LANGUAGE,
      })
    )
  }, [])

  useEffect(() => {
    void refresh()
  }, [refresh])

  const download = useCallback(
    async (id: string) => {
      setDownloading(id)
      setFraction(0)
      try {
        for await (const event of RunAnywhere.models.download(id)) {
          if (event.type === 'progress') setFraction(event.fraction ?? 0)
        }
        await refresh()
      } finally {
        setDownloading(null)
      }
    },
    [refresh]
  )

  const use = useCallback(async (id: string) => {
    await RunAnywhere.models.unloadAll()
    await RunAnywhere.models.load(id)
  }, [])

  return { models, downloading, fraction, download, use, refresh }
}

Errors worth handling

Code	Cause
`modelNotFound`	The id is not in the registry
`insufficientStorage`	Not enough disk for the download
`insufficientMemory`	The model will not fit in RAM
`networkUnavailable`	A download needs a connection

Swift SDK

Kotlin SDK

React Native SDK

Flutter SDK

Web SDK

Electron SDK

The short version

What is available

Downloading, with progress

Loading

MLX and QHexRT

Switching models

Unloading

What is resident right now

Deleting

Registering your own model

Storage

A complete picker

Errors worth handling

​The short version

​What is available

​Downloading, with progress

​Loading

​MLX and QHexRT

​Switching models

​Unloading

​What is resident right now

​Deleting

​Registering your own model

​Storage

​A complete picker

​Errors worth handling

The short version

What is available

Downloading, with progress

Loading

MLX and QHexRT

Switching models

Unloading

What is resident right now

Deleting

Registering your own model

Storage

A complete picker

Errors worth handling