Models

RunAnywhere.models is the model catalog and its residency. Everything about getting a model onto the device and into memory lives here.

The short version

You often do not need this page. Generation loads what it needs and downloads when options.model names a model that is not on disk:

let result = try await RunAnywhere.llm.generate(
    prompt: "Hello",
    options: LlmOptions(model: "qwen3-0.6b")
)

Reach for the models namespace when you want to control when that cost is paid, show download progress, or let a person choose.

What is available

let models = try await RunAnywhere.models.list()

Filter to one modality:

let language = try await RunAnywhere.models.list(filter: ModelFilter(category: .language))

One model by id:

if let model = await RunAnywhere.models.get(id: "qwen3-0.6b") {
    print(model.name)
}

Downloading, with progress

download returns a stream of DownloadEvent. Models are hundreds of megabytes to several gigabytes, so show this rather than a spinner.

for try await event in try await RunAnywhere.models.download(id: "qwen3-0.6b") {
    switch event {
    case .progress(let progress):
        self.fraction = progress.fraction ?? 0
        self.speed = progress.bytesPerSecond
        self.eta = progress.etaSeconds
    case .completed:
        self.isReady = true
    default:
        break
    }
}

DownloadProgress carries bytesDone, bytesTotal, fraction, percent, bytesPerSecond, etaSeconds, retryAttempt, currentFileIndex, totalFiles, and overallProgress. A multi-file model reports both the current file and the overall figure.

Resuming

An interrupted download can often continue rather than starting over:

if await RunAnywhere.models.isResumable(id: "qwen3-0.6b") {
    // calling download again continues from where it stopped
}

Loading

Downloading puts the model on disk. Loading puts it in memory.

let loaded = try await RunAnywhere.models.load(id: "qwen3-0.6b")

With options:

let loaded = try await RunAnywhere.models.load(
    id: "qwen3-0.6b",
    options: LoadOptions(contextLength: 4096, threads: 4)
)

Field	Type	Meaning
`backendPreferences`	`[BackendPreference]`	Ordered preference; each can be `required`
`accelerator`	`AcceleratorPolicy?`	Accelerator policy
`contextLength`	`Int?`	Context window to allocate
`threads`	`Int?`	CPU threads
`forceReload`	`Bool`	Reload even if already resident
`framework`	`InferenceFramework?`	Pin to one backend
`useGpu`	`Bool?`	Request GPU execution

contextLength is the one that matters most. A larger window costs memory whether you use it or not, so allocate what your longest conversation actually needs. Pinning a backend:

LoadOptions(
    backendPreferences: [BackendPreference(backend: .llamaCpp, required: true)]
)

With required: false the SDK falls back to another backend if that one cannot serve the model. With required: true it fails instead, which is what you want when you are measuring one engine specifically.

Switching models

Load the new one, unload the old one. Doing it in that order keeps a model available throughout, at the cost of holding both briefly. On a phone, reverse it.

@MainActor
@Observable
final class ModelSwitcher {
    private(set) var current: String?

    func use(_ id: String) async throws {
        if let current, current != id {
            try await RunAnywhere.models.unload(id: current)
        }
        _ = try await RunAnywhere.models.load(id: id)
        current = id
    }
}

You can also switch per call, without touching residency, by naming the model in options:

let a = try await RunAnywhere.llm.generate(prompt: p, options: LlmOptions(model: "qwen3-0.6b"))
let b = try await RunAnywhere.llm.generate(prompt: p, options: LlmOptions(model: "llama-3.2-1b"))

The SDK loads and unloads as needed. That is simpler, and slower when you alternate.

Unloading

try await RunAnywhere.models.unload(id: "qwen3-0.6b")
try await RunAnywhere.models.unloadAll(category: .language)
try await RunAnywhere.models.unloadAll()

Unloading frees memory and leaves the file on disk, so the next load is fast. This is what you do when a screen goes away, and it is the fix for .insufficientMemory.

What is resident right now

let state = await RunAnywhere.models.state()

Use it to render a “loaded” badge, or to decide whether a load is needed before a latency-sensitive path.

Deleting

try await RunAnywhere.models.delete(id: "qwen3-0.6b")

Removes the file from disk. Unloads it first if it is resident.

Registering your own model

Models outside the curated catalog are registered before use. There are three shapes. A single file by URL:

let info = try await RunAnywhere.registerModel(
    name: "My model",
    url: "https://example.com/model.gguf",
    framework: .llamaCpp,
    modality: .language
)

An archive, where you declare the structure inside:

let info = try await RunAnywhere.registerModel(
    archive: "https://example.com/model.tar.gz",
    structure: structure,
    name: "My model",
    framework: .onnx,
    modality: .speechRecognition
)

A multi-file model, described by file descriptors:

let info = try await RunAnywhere.registerModel(
    multiFile: descriptors,
    id: "my-vlm",
    name: "My VLM",
    framework: .llamaCpp,
    modality: .vision
)

A vision model with a separate projector, or a speech model with encoder and decoder files, is the multi-file case.

Gated repositories

For a private or gated Hugging Face repo, set the token first:

RunAnywhere.setHfToken(token)

Keep it in the Keychain, never in source.

Refreshing the registry

await RunAnywhere.models.refresh(rescanLocal: true)

Reconciles the catalog against the bytes actually on disk. Worth calling after the app has been offline, or if a user deleted files outside the app.

Parameter	Default	Effect
`rescanLocal`	`true`	Re-read what is on disk
`includeRemoteCatalog`	`false`	Also fetch the remote catalog
`pruneOrphans`	`false`	Drop registry entries with no files

A complete picker

@MainActor
@Observable
final class ModelPicker {
    var models: [ModelInfo] = []
    var downloading: String?
    var fraction: Double = 0

    func load() async throws {
        models = try await RunAnywhere.models.list(filter: ModelFilter(category: .language))
    }

    func download(_ id: String) async throws {
        downloading = id
        fraction = 0
        defer { downloading = nil }

        for try await event in try await RunAnywhere.models.download(id: id) {
            if case .progress(let p) = event {
                fraction = Double(p.fraction ?? 0)
            }
        }

        try await load()
    }

    func use(_ id: String) async throws {
        try await RunAnywhere.models.unloadAll(category: .language)
        _ = try await RunAnywhere.models.load(id: id)
    }

    func delete(_ id: String) async throws {
        try await RunAnywhere.models.delete(id: id)
        try await load()
    }
}

Errors worth handling

Code	Cause
`.modelNotFound`	The id is not in the registry
`.insufficientStorage`	Not enough disk for the download
`.insufficientMemory`	The model will not fit in RAM
`.networkUnavailable`	A download needs a connection

.insufficientMemory on load usually means something else is still resident. Unload first.

Getting Started

Swift SDK

Kotlin SDK

React Native SDK

Flutter SDK

Web SDK

Electron SDK

Vibe Coding

The short version

What is available

Downloading, with progress

Resuming

Loading

Switching models

Unloading

What is resident right now

Deleting

Registering your own model

Gated repositories

Refreshing the registry

A complete picker

Errors worth handling

​The short version

​What is available

​Downloading, with progress

​Resuming

​Loading

​Switching models

​Unloading

​What is resident right now

​Deleting

​Registering your own model

​Gated repositories

​Refreshing the registry

​A complete picker

​Errors worth handling

The short version

What is available

Downloading, with progress

Resuming

Loading

Switching models

Unloading

What is resident right now

Deleting

Registering your own model

Gated repositories

Refreshing the registry

A complete picker

Errors worth handling