Models

RunAnywhere.models is the model catalog and its residency. In a browser it matters more than anywhere else, because the download is the user’s bandwidth and the storage is their disk.

The short version

Generation loads what it needs and downloads when options.model names a model that is not present:

const result = await RunAnywhere.llm.generate('Hello', { model: 'qwen3-0.6b' })

Reach for the models namespace when you want to control when that cost is paid, show progress, or let a person choose.

What is available

list, get, and register are synchronous on Web and async on every other SDK. Code ported from Swift or React Native will have await in the wrong places.

const models = RunAnywhere.models.list()
const language = RunAnywhere.models.list({ category: ModelCategory.MODEL_CATEGORY_LANGUAGE })
const model = RunAnywhere.models.get('qwen3-0.6b') // ModelInfo | null

Downloading, with progress

for await (const event of RunAnywhere.models.download('qwen3-0.6b')) {
  if (event.type === 'progress') {
    bar.value = event.fraction ?? 0
    label.textContent = `${Math.round(event.percent ?? 0)}% · ${format(event.bytesPerSecond)}/s`
  }
}

Progress carries bytesDone, bytesTotal, fraction, percent, bytesPerSecond, etaSeconds, retryAttempt, currentFileIndex, totalFiles, and overallProgress. Show the size before you start. A 2GB download over a metered connection is a real cost, and a browser gives the user no other warning.

const model = RunAnywhere.models.get('qwen3-0.6b')
if (!confirm(`Download ${model?.name} (${formatBytes(model?.downloadSize)})?`)) return

Where downloads go

This is the browser-specific part, and it decides whether a returning user re-downloads everything.

RunAnywhere.storage.backend // 'fsAccess' | 'opfs' | 'memory'
RunAnywhere.storage.directoryName // string | null

Backend	Meaning
`opfs`	Origin-private storage. The default. Wiped when the user clears site data.
`fsAccess`	A real directory the user granted. Survives clearing site data.
`memory`	No persistence. Everything re-downloads on reload.

Offer a real directory to anyone who will come back:

button.addEventListener('click', async () => {
  await RunAnywhere.storage.chooseDirectory()
})

chooseDirectory() requires a user gesture. Calling it outside a click handler is rejected by the browser, not by the SDK.

After a reload, a granted directory has to be re-acquired, and a lapsed permission re-asked:

await RunAnywhere.storage.restore() // after a reload
await RunAnywhere.storage.requestAccess() // permission lapsed

Neither happens automatically.

Loading

Downloading puts the model in storage. Loading puts it in memory.

const loaded = await RunAnywhere.models.load('qwen3-0.6b')

const tuned = await RunAnywhere.models.load('qwen3-0.6b', {
  contextLength: 4096,
  threads: 4,
})

Field	Meaning
`backendPreferences`	Ordered preference; each can be required
`accelerator`	Accelerator policy
`contextLength`	Context window to allocate
`threads`	CPU threads
`forceReload`	Reload even if already resident
`framework`	Pin to one backend
`useGpu`	Request GPU execution

contextLength costs memory whether you use it or not. A tab has less headroom than a phone, so allocate what your longest conversation actually needs.

CPU or WebGPU

autoRegister picks the right build for the browser, so you normally do not choose. When a WebGPU model produces garbage, switch that model to its CPU variant rather than disabling WebGPU globally.

Will it fit

const verdict = await RunAnywhere.models.checkCompatibility(/* … */)

checkCompatibility exists only on Web. Use it before offering a large model.

Switching models

let current: string | null = null

async function use(id: string): Promise<void> {
  if (current && current !== id) {
    await RunAnywhere.models.unload(current)
  }
  await RunAnywhere.models.load(id)
  current = id
}

In a tab, unload before loading. Holding two language models is how you get insufficientMemory. Or switch per call, without touching residency:

await RunAnywhere.llm.generate(prompt, { model: 'qwen3-0.6b' })
await RunAnywhere.llm.generate(prompt, { model: 'llama-3.2-1b' })

Unloading

await RunAnywhere.models.unload('qwen3-0.6b')
await RunAnywhere.models.unloadAll()

Frees memory and leaves the bytes in storage, so the next load is fast.

What is resident right now

const state = await RunAnywhere.models.state()

Deleting and reclaiming space

await RunAnywhere.models.delete('qwen3-0.6b')
await RunAnywhere.models.unregister('qwen3-0.6b')

await RunAnywhere.storage.refresh() // reconcile catalog against disk
await RunAnywhere.storage.clearCaches()

storage.refresh() returns the number of entries reconciled. Call it after the user has deleted files outside the app, or after a storage backend change.

Registering your own model

const info = RunAnywhere.models.register({
  id: 'my-model',
  name: 'My model',
  url: 'https://example.com/model.gguf',
  framework: InferenceFramework.INFERENCE_FRAMEWORK_LLAMA_CPP,
  category: ModelCategory.MODEL_CATEGORY_LANGUAGE,
})

Synchronous, like list and get. The URL has to be reachable from the page, which means CORS applies. A model host that does not send permissive headers will fail the fetch, and that failure looks like a network error rather than a configuration one. This bites more on Web than on any other platform.

Gated repositories

RunAnywhere.setHuggingFaceToken(token) // null to clear

Never ship a server-side secret this way. Vite inlines build-time variables into the bundle, so anything you put there is public.

A complete picker

const list = document.querySelector<HTMLUListElement>('#models')!

function render(): void {
  list.replaceChildren()

  for (const model of RunAnywhere.models.list()) {
    const item = document.createElement('li')
    item.textContent = model.name

    const action = document.createElement('button')
    action.textContent = model.isDownloaded ? 'Use' : 'Download'
    action.addEventListener('click', async () => {
      if (model.isDownloaded) {
        await RunAnywhere.models.unloadAll()
        await RunAnywhere.models.load(model.id)
      } else {
        for await (const event of RunAnywhere.models.download(model.id)) {
          if (event.type === 'progress') {
            action.textContent = `${Math.round(event.percent ?? 0)}%`
          }
        }
        render()
      }
    })

    item.append(action)
    list.append(item)
  }
}

Errors worth handling

Code	Cause
`modelNotFound`	The id is not in the registry
`insufficientStorage`	The origin’s storage quota, not the disk
`insufficientMemory`	The model will not fit in the tab
`networkUnavailable`	A download needs a connection

Storage errors on Web are about the origin’s quota, which is smaller than the disk and varies by browser. Offering a real directory through chooseDirectory() sidesteps it.

Getting Started

Swift SDK

Kotlin SDK

React Native SDK

Flutter SDK

Web SDK

Electron SDK

Vibe Coding

The short version

What is available

Downloading, with progress

Where downloads go

Loading

CPU or WebGPU

Will it fit

Switching models

Unloading

What is resident right now

Deleting and reclaiming space

Registering your own model

Gated repositories

A complete picker

Errors worth handling

​The short version

​What is available

​Downloading, with progress

​Where downloads go

​Loading

​CPU or WebGPU

​Will it fit

​Switching models

​Unloading

​What is resident right now

​Deleting and reclaiming space

​Registering your own model

​Gated repositories

​A complete picker

​Errors worth handling

The short version

What is available

Downloading, with progress

Where downloads go

Loading

CPU or WebGPU

Will it fit

Switching models

Unloading

What is resident right now

Deleting and reclaiming space

Registering your own model

Gated repositories

A complete picker

Errors worth handling