> ## Documentation Index
> Fetch the complete documentation index at: https://docs.runanywhere.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Models

> Find, download, load, switch, and remove models

`RunAnywhere.models` is the model catalog and its residency. In a browser it matters more than
anywhere else, because the download is the user's bandwidth and the storage is their disk.

## The short version

Generation loads what it needs and downloads when `options.model` names a model that is not
present:

```ts theme={null}
const result = await RunAnywhere.llm.generate('Hello', { model: 'qwen3-0.6b' })
```

Reach for the `models` namespace when you want to control **when** that cost is paid, show
progress, or let a person choose.

## What is available

<Note>
  `list`, `get`, and `register` are **synchronous** on Web and async on every other SDK. Code ported
  from Swift or React Native will have `await` in the wrong places.
</Note>

```ts theme={null}
const models = RunAnywhere.models.list()
const language = RunAnywhere.models.list({ category: ModelCategory.MODEL_CATEGORY_LANGUAGE })
const model = RunAnywhere.models.get('qwen3-0.6b') // ModelInfo | null
```

## Downloading, with progress

```ts theme={null}
for await (const event of RunAnywhere.models.download('qwen3-0.6b')) {
  if (event.type === 'progress') {
    bar.value = event.fraction ?? 0
    label.textContent = `${Math.round(event.percent ?? 0)}% · ${format(event.bytesPerSecond)}/s`
  }
}
```

Progress carries `bytesDone`, `bytesTotal`, `fraction`, `percent`, `bytesPerSecond`,
`etaSeconds`, `retryAttempt`, `currentFileIndex`, `totalFiles`, and `overallProgress`.

**Show the size before you start.** A 2GB download over a metered connection is a real cost,
and a browser gives the user no other warning.

```ts theme={null}
const model = RunAnywhere.models.get('qwen3-0.6b')
if (!confirm(`Download ${model?.name} (${formatBytes(model?.downloadSize)})?`)) return
```

## Where downloads go

This is the browser-specific part, and it decides whether a returning user re-downloads
everything.

```ts theme={null}
RunAnywhere.storage.backend // 'fsAccess' | 'opfs' | 'memory'
RunAnywhere.storage.directoryName // string | null
```

| Backend    | Meaning                                                                    |
| ---------- | -------------------------------------------------------------------------- |
| `opfs`     | Origin-private storage. The default. Wiped when the user clears site data. |
| `fsAccess` | A real directory the user granted. Survives clearing site data.            |
| `memory`   | No persistence. Everything re-downloads on reload.                         |

Offer a real directory to anyone who will come back:

```ts theme={null}
button.addEventListener('click', async () => {
  await RunAnywhere.storage.chooseDirectory()
})
```

<Warning>
  `chooseDirectory()` requires a user gesture. Calling it outside a click handler is rejected by the
  browser, not by the SDK.
</Warning>

After a reload, a granted directory has to be re-acquired, and a lapsed permission re-asked:

```ts theme={null}
await RunAnywhere.storage.restore() // after a reload
await RunAnywhere.storage.requestAccess() // permission lapsed
```

Neither happens automatically.

## Loading

Downloading puts the model in storage. Loading puts it in memory.

```ts theme={null}
const loaded = await RunAnywhere.models.load('qwen3-0.6b')

const tuned = await RunAnywhere.models.load('qwen3-0.6b', {
  contextLength: 4096,
  threads: 4,
})
```

| Field                | Meaning                                  |
| -------------------- | ---------------------------------------- |
| `backendPreferences` | Ordered preference; each can be required |
| `accelerator`        | Accelerator policy                       |
| `contextLength`      | Context window to allocate               |
| `threads`            | CPU threads                              |
| `forceReload`        | Reload even if already resident          |
| `framework`          | Pin to one backend                       |
| `useGpu`             | Request GPU execution                    |

`contextLength` costs memory whether you use it or not. A tab has less headroom than a phone,
so allocate what your longest conversation actually needs.

### CPU or WebGPU

`autoRegister` picks the right build for the browser, so you normally do not choose. When a
WebGPU model produces garbage, switch that model to its CPU variant rather than disabling
WebGPU globally.

### Will it fit

```ts theme={null}
const verdict = await RunAnywhere.models.checkCompatibility(/* … */)
```

`checkCompatibility` exists only on Web. Use it before offering a large model.

## Switching models

```ts theme={null}
let current: string | null = null

async function use(id: string): Promise<void> {
  if (current && current !== id) {
    await RunAnywhere.models.unload(current)
  }
  await RunAnywhere.models.load(id)
  current = id
}
```

In a tab, unload before loading. Holding two language models is how you get
`insufficientMemory`.

Or switch per call, without touching residency:

```ts theme={null}
await RunAnywhere.llm.generate(prompt, { model: 'qwen3-0.6b' })
await RunAnywhere.llm.generate(prompt, { model: 'llama-3.2-1b' })
```

## Unloading

```ts theme={null}
await RunAnywhere.models.unload('qwen3-0.6b')
await RunAnywhere.models.unloadAll()
```

Frees memory and leaves the bytes in storage, so the next load is fast.

## What is resident right now

```ts theme={null}
const state = await RunAnywhere.models.state()
```

## Deleting and reclaiming space

```ts theme={null}
await RunAnywhere.models.delete('qwen3-0.6b')
await RunAnywhere.models.unregister('qwen3-0.6b')

await RunAnywhere.storage.refresh() // reconcile catalog against disk
await RunAnywhere.storage.clearCaches()
```

`storage.refresh()` returns the number of entries reconciled. Call it after the user has
deleted files outside the app, or after a storage backend change.

## Registering your own model

```ts theme={null}
const info = RunAnywhere.models.register({
  id: 'my-model',
  name: 'My model',
  url: 'https://example.com/model.gguf',
  framework: InferenceFramework.INFERENCE_FRAMEWORK_LLAMA_CPP,
  category: ModelCategory.MODEL_CATEGORY_LANGUAGE,
})
```

Synchronous, like `list` and `get`.

The URL has to be reachable from the page, which means **CORS applies**. A model host that
does not send permissive headers will fail the fetch, and that failure looks like a network
error rather than a configuration one. This bites more on Web than on any other platform.

### Gated repositories

```ts theme={null}
RunAnywhere.setHuggingFaceToken(token) // null to clear
```

Never ship a server-side secret this way. Vite inlines build-time variables into the bundle,
so anything you put there is public.

## A complete picker

```ts theme={null}
const list = document.querySelector<HTMLUListElement>('#models')!

function render(): void {
  list.replaceChildren()

  for (const model of RunAnywhere.models.list()) {
    const item = document.createElement('li')
    item.textContent = model.name

    const action = document.createElement('button')
    action.textContent = model.isDownloaded ? 'Use' : 'Download'
    action.addEventListener('click', async () => {
      if (model.isDownloaded) {
        await RunAnywhere.models.unloadAll()
        await RunAnywhere.models.load(model.id)
      } else {
        for await (const event of RunAnywhere.models.download(model.id)) {
          if (event.type === 'progress') {
            action.textContent = `${Math.round(event.percent ?? 0)}%`
          }
        }
        render()
      }
    })

    item.append(action)
    list.append(item)
  }
}
```

## Errors worth handling

| Code                  | Cause                                    |
| --------------------- | ---------------------------------------- |
| `modelNotFound`       | The id is not in the registry            |
| `insufficientStorage` | The origin's storage quota, not the disk |
| `insufficientMemory`  | The model will not fit in the tab        |
| `networkUnavailable`  | A download needs a connection            |

Storage errors on Web are about the **origin's quota**, which is smaller than the disk and
varies by browser. Offering a real directory through `chooseDirectory()` sidesteps it.