# Create a chat completion Source: https://docs.runanywhere.ai/api-reference/create-a-chat-completion /api-reference/openapi.json post /chat/completions # List models available to the current key Source: https://docs.runanywhere.ai/api-reference/list-models-available-to-the-current-key /api-reference/openapi.json get /models # Best Practices Source: https://docs.runanywhere.ai/electron/best-practices Getting good behaviour out of a desktop integration ## Put inference in the utility host `RpcBackend` forwards to the utility process; `NativeBackend` loads the addon in the current one. Use the host for anything heavy, or a model load blocks the main process and the window stops responding. ## Unpack the natives Everything under `node_modules/@runanywhere/*/prebuilds/` has to be `asarUnpack`ed, both by extension and by the whole tree. The tree rule is what covers QHexRT's `libqnnhtpv81.cat`, whose absence reads as a corrupt model rather than a packaging fault. ## Read capabilities, do not assume ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const capabilities = await RunAnywhere.capabilities() ``` Generated from packaging facts about the linked addon. On `win32-arm64` only QHexRT loads, with no CPU fallback, so vision, segmentation, and speech are unavailable there. Check before putting a feature in a menu. ## Offer pause and resume Download control exists only on Electron. A user closing a laptop lid mid-download should not lose four gigabytes. ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const stalled = await RunAnywhere.models.interrupted() ``` Call this on launch and offer to continue. ## Show a delete plan ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const plan = await RunAnywhere.storage.deletePlan() ``` Show what would go before deleting, rather than deleting and reporting afterwards. Nothing else in the SDK family can do this. ## Use residency policy with several models A chat model and a voice session compete. Without a policy, one evicts the other on every turn and the user waits through a reload each time. ## Close sessions on window close, not only on quit ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} win.on('closed', () => void session?.close()) app.on('before-quit', () => void session?.close()) ``` Closing a window does not end the process. A session tied only to quit keeps the microphone live after the user thinks they stopped. ## Validate tool arguments Tool executors here have the filesystem, child processes, and the network with no CORS. A model choosing which file to read is a model choosing which file to read. Confine paths to a directory you picked, and never build a shell string from model output. ## Batch what crosses the bridge Token deltas are small enough to forward individually. Audio chunks are not. Send audio once per chunk and let the renderer schedule playback against a running clock. ## Send error codes, not error objects Structured clone drops the prototype, so an `SDKException` sent to the renderer arrives as a plain object and `instanceof` fails. Forward `code` and `message` explicitly. ## Size the load for the machine A desktop can afford a larger `contextLength` and higher `threads` than a phone. Both still cost memory, so read what the machine has rather than hardcoding numbers that work on yours. ## Sign your Windows build The published installers carry no Authenticode certificate, so SmartScreen warns on first run. If you ship your own build, signing it is the difference between a user proceeding and not. # Configuration Source: https://docs.runanywhere.ai/electron/configuration Models, storage, logging, and identity ## Lifecycle ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} await RunAnywhere.initialize(options?: InitializeOptions); await RunAnywhere.reset(); await RunAnywhere.capabilities(); await RunAnywhere.componentLifecycleSnapshot(component); ``` | Accessor | Type | Note | | - | - | - | | `isReady` | `boolean` | | | `version` | `string` | **Empty** until `initialize()` resolves | | `deviceId` | `string` | **Empty** until `initialize()` resolves | | `environment` | `Environment` | | | `events` | `AsyncIterableIterator` | | `componentLifecycleSnapshot` treats an unloaded component as an answer rather than a failure: it comes back with `COMPONENT_LIFECYCLE_STATE_NOT_LOADED` and no model id. ## Hugging Face token ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} await RunAnywhere.setHfToken(token) // null to fall back, '' to clear ``` This one has behaviour worth knowing: * `null` falls back to the environment lookup the core resolves: `HF_TOKEN`, then `$HF_TOKEN_PATH`, then `$HF_HOME/token`, then `~/.cache/huggingface/token`. That is the order `huggingface_hub` uses, so `hf auth login` is honoured. * An empty string clears the token **and** disables that fallback. * The token lives in the platform secure store, not in settings, and is re-applied on the next `initialize()` so a gated model still downloads after a cold start. * It is attached only to https requests whose host is exactly `huggingface.co` or `hf.co`, never to a CDN or LFS redirect target. ## Models ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} await RunAnywhere.models.list(filter?); await RunAnywhere.models.get(id); await RunAnywhere.models.register(model); await RunAnywhere.models.unregister(id); await RunAnywhere.models.refresh(options?); await RunAnywhere.models.delete(id); await RunAnywhere.models.load(id, options?); await RunAnywhere.models.unload(id); ``` ### Download control ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} for await (const event of RunAnywhere.models.download(id)) { /* … */ } await RunAnywhere.models.pause(id) await RunAnywhere.models.resume(id) await RunAnywhere.models.cancel(id) await RunAnywhere.models.interrupted() // ids left half-downloaded await RunAnywhere.models.isResumable(id) ``` `interrupted()` after a crash or a forced quit tells you what to offer resuming, which is the difference between a user losing a 4GB download and not. ## Storage ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} await RunAnywhere.storage.info(request?); await RunAnywhere.storage.availability(request); await RunAnywhere.storage.deletePlan(request?); await RunAnywhere.storage.delete(request); await RunAnywhere.storage.clearCache(); await RunAnywhere.storage.cleanTempFiles(); ``` `deletePlan` is a dry run. Call it before `delete` and show the user exactly what is about to go, rather than deleting and reporting afterwards. `availability` answers "will this fit" before a download starts. ## Logging ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} await RunAnywhere.logging.configure(config) await RunAnywhere.logging.configuration() await RunAnywhere.logging.setLevel(level) await RunAnywhere.logging.level() await RunAnywhere.logging.setLocalEnabled(enabled) await RunAnywhere.logging.setDebugMode(enabled) await RunAnywhere.logging.flush() const unsubscribe = await RunAnywhere.logging.addDestination(destination) unsubscribe() for await (const entry of RunAnywhere.logging.records()) { console.log(entry) } ``` `addDestination` returns an unsubscribe function rather than taking a removal call. `records()` is a live stream, which is what a debug pane in the app can render directly. ## LoRA Electron carries the full catalog surface, not the short form: ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} await RunAnywhere.lora.apply(adapterId, scale?); await RunAnywhere.lora.remove(adapterId?); await RunAnywhere.lora.removeAll(); await RunAnywhere.lora.list(); await RunAnywhere.lora.state(); await RunAnywhere.lora.checkCompatibility(config); await RunAnywhere.lora.register(entry); await RunAnywhere.lora.registerArtifact(/* … */); await RunAnywhere.lora.download(/* … */); await RunAnywhere.lora.importAdapter(sourcePath); await RunAnywhere.lora.listCatalog(query?); await RunAnywhere.lora.queryCatalog(query); ``` ## Embeddings helpers ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const vectors = await RunAnywhere.embeddings.embed(texts, options?); const similarity = await RunAnywhere.embeddings.cosineSimilarity(a, b); const norm = await RunAnywhere.embeddings.computeNorm(vector); ``` `cosineSimilarity` and `computeNorm` on the namespace are Electron-only. ## Identity and telemetry ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} RunAnywhere.auth // control-plane identity RunAnywhere.secure // platform secure store RunAnywhere.telemetry // telemetry controls ``` These become meaningful only once the SDK is initialized with control-plane credentials. In keyless local mode they report an unauthenticated state rather than throwing. ## Model residency ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} import { ResidencyPolicy } from '@runanywhere/electron' ``` `ResidencyPolicy`, `ResidencyDecision`, `ResidencySlots`, and `ResidentModel` control which models stay loaded when memory is contended. On a desktop with several models in play, this is what stops a voice session evicting the chat model on every turn. # Diarization Source: https://docs.runanywhere.ai/electron/diarization Work out who spoke when Diarization segments a recording by speaker. It answers "who spoke when", not "what did they say"; pair it with [transcription](/electron/stt/transcribe) for both. ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const result = await RunAnywhere.diarization.diarize(RunAnywhere.audio.file(path)) ``` ## Options | Field | Meaning | | - | - | | `threshold` | How different two voices must be to count as separate speakers | | `minimumDurationMs` | Ignore segments shorter than this | | `mergeGapMs` | Join two segments from one speaker separated by less than this | `mergeGapMs` is the one to tune. Natural speech has pauses inside a turn, and without merging you get one speaker fragmented into a dozen segments. ## Diarization inside transcription For a transcript labelled by speaker rather than a separate segment list, ask the STT namespace instead: ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const transcription = await RunAnywhere.stt.transcribe(audio, { diarization: true, maxSpeakers: 2, }) ``` Use the STT route when you want a readable transcript. Use the diarization namespace when you want timing, for example to drive a speaker timeline or to split a recording into per-speaker files. ## Splitting a recording Desktop is the platform where this is worth doing, because you can write the output straight to disk: ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} import { writeFile } from 'node:fs/promises' const result = await RunAnywhere.diarization.diarize(RunAnywhere.audio.file(path)) // slice the source audio by each segment's timing and write per-speaker files ``` ## Knowing the speaker count helps `maxSpeakers` on the STT path, and a sensible `threshold` here, both improve accuracy a lot when you know how many people are in the room. Diarization over-segments when left to guess. Electron has no streaming diarizer. Diarize a completed recording, or use the STT path with `diarization: true` for live labelling. # Embeddings & Reranking Source: https://docs.runanywhere.ai/electron/embeddings Turn text into vectors, and reorder results against a query Two namespaces that work together. `embeddings` turns text into vectors you can compare. `rerank` takes a query and a list of candidates and reorders them by relevance. ## Embedding text ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const vectors = await RunAnywhere.embeddings.embed(['hello', 'world']) console.log(vectors[0].vector.length) ``` Vectors come back in input order, so index `i` of the result matches index `i` of the input. Batch rather than looping: one call with fifty strings is far cheaper than fifty calls. ## Comparison helpers Electron is the only SDK with these on the namespace: ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const similarity = await RunAnywhere.embeddings.cosineSimilarity(a, b) const norm = await RunAnywhere.embeddings.computeNorm(vector) ``` Both accept an `Embedding` or a raw `Float32Array`. Elsewhere you write these yourself, or use helpers on the vector type. ## Options | Field | Default | Meaning | | - | - | - | | `normalize` | `true` | Unit-length vectors, so cosine similarity is a dot product | | `pooling` | `mean` | How token vectors collapse into one | Leave `normalize` on unless you have a reason. With it off, similarity scores are not comparable across texts of different lengths. ## Reranking ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const ranked = await RunAnywhere.rerank.rerank('refund policy', candidates, 5) ``` Reranking is a different model from embedding. An embedding model encodes each text once and compares vectors, which is fast and approximate. A reranker reads the query and the candidate together, which is slower and much more accurate. The usual shape is both: retrieve widely with embeddings, then rerank the top handful. ## Indexing a folder A desktop app can embed a real corpus rather than a handful of pasted strings: ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} import { readdir, readFile } from 'node:fs/promises' import { join } from 'node:path' const names = await readdir(folder) const texts = await Promise.all(names.map((n) => readFile(join(folder, n), 'utf8'))) const vectors = await RunAnywhere.embeddings.embed(texts) ``` Batch in chunks rather than passing thousands of documents at once, and persist the vectors so a relaunch does not re-embed everything. ## Where this fits Most people reach for these through [RAG](/electron/rag), which does retrieval, reranking, and generation in one session. Use the namespaces directly for semantic search with no generation, deduplication, or classification by nearest neighbour. ## Models Embedding and reranking models are small, often tens of megabytes rather than gigabytes, so keeping both resident alongside a language model is reasonable on a desktop. # Error Handling Source: https://docs.runanywhere.ai/electron/error-handling SDKException, codes, and recovery Everything the SDK throws is an `SDKException`, wrapping a proto-backed error so the same code and category cross every language binding. ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} import { SDKException } from '@runanywhere/electron' try { const result = await RunAnywhere.llm.generate(prompt) } catch (error) { if (error instanceof SDKException) { console.log(error.code, error.message) } } ``` ## Codes worth handling | Code | Meaning | What to do | | - | - | - | | `notInitialized` | Called before `initialize()` | Initialize first | | `modelNotFound` | No such model | Check the id, or register it | | `networkUnavailable` | Download needs a connection | Retry when online | | `insufficientStorage` | Not enough disk | Free space, or use `storage.deletePlan()` | | `insufficientMemory` | The model will not fit | Unload something first | | `microphonePermissionDenied` | No mic access | Send the user to System Settings | | `timeout` | Took too long | Retry | | `invalidApiKey` | Credentials rejected | Check the key | | `cancelled` | The caller cancelled | Usually ignore | | `invalidState` | Wrong order of operations | Read the message | | `unsupportedCapability` | Not implemented in this build | See below | ## unsupportedCapability ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} // Throws: constrained decoding is not wired in await RunAnywhere.llm.generateStructured(prompt, schema, 'constrained') ``` You will also meet it on `win32-arm64`, where only QHexRT loads and anything needing llama.cpp, ONNX, or Sherpa reports unavailable rather than failing obscurely at first use. ## Errors in streams Preflight failures throw from the call. A failure **during** generation arrives as a `failed` event: ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} for await (const event of RunAnywhere.llm.generateStream(prompt)) { if (event.type === 'failed') { showRetry(event.error) break } if (event.type === 'textDelta') output += event.text } ``` Breaking out yields a `cancelled` terminal event, which is not a failure. This is the same shape as the Web SDK and the opposite of Swift, Kotlin, Flutter, and React Native, where in-flight failures are thrown into the consumer. ## Errors across the context bridge An exception thrown in the main process does not arrive in the renderer as an `SDKException`. Structured clone drops the prototype, so `instanceof` fails there. Send the code and message explicitly rather than the error object: ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} // main catch (error) { if (error instanceof SDKException) { win.webContents.send('sdk-error', { code: error.code, message: error.message }); } } ``` ## Failures that are not the SDK | Symptom | Cause | | - | - | | Native module fails to load | `prebuilds/**` was not `asarUnpack`ed | | "Corrupt model" on Windows ARM64 | A missing `libqnnhtpv81.cat` beside the Hexagon skel | | Window never appears | `ELECTRON_RUN_AS_NODE` is set | | Nothing loads on Linux | No Linux build is published | The second one is worth remembering, because the error names the model rather than the packaging fault that caused it. ## Reporting ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} catch (error) { if (error instanceof SDKException) { telemetry.track('sdk_error', { code: String(error.code) }); } else { throw error; } } ``` Rethrow what you did not recognise. # Image Generation Source: https://docs.runanywhere.ai/electron/images Generate images on-device `RunAnywhere.images` generates an image from a prompt. ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const result = await RunAnywhere.images.generate('A lighthouse at dusk, oil painting', { negativePrompt: 'blurry, text, watermark', width: 512, height: 512, steps: 20, seed: 42, }) ``` ## Options | Field | Default | Meaning | | - | - | - | | `negativePrompt` | none | What to avoid | | `width` | model default | Output width | | `height` | model default | Output height | | `steps` | model default | Denoising steps; more is slower and usually better | | `guidanceScale` | model default | How closely to follow the prompt | | `seed` | random | Fix it to reproduce an image | | `mode` | generate | Generate or inpaint | | `reportPartials` | `false` | Emit intermediate images while streaming | ## Watching it appear With `reportPartials` on, the stream emits intermediate images, so you can show the picture resolving rather than a spinner. ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} for await (const event of RunAnywhere.images.generateStream('A lighthouse at dusk', { steps: 20, reportPartials: true, })) { if (event.type === 'partial') showPreview(event.image) if (event.type === 'completed') showFinal(event.result) } ``` Put the progress on the taskbar or dock icon as well as in the window: ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} win.setProgressBar(fraction) ``` ## Reproducing an image Fix the seed and every other option, and the same prompt produces the same image. Without a fixed seed, it will not. A desktop app that lets people iterate on a prompt should surface the seed, so a result they liked can be recovered. ## Saving ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} import { writeFile } from 'node:fs/promises' import { dialog } from 'electron' const { filePath } = await dialog.showSaveDialog({ defaultPath: 'image.png' }) if (filePath) await writeFile(filePath, result.image) ``` ## Cost Image generation is the heaviest thing in the SDK. A desktop has the headroom a phone does not, but it still takes far longer than a text completion and spins the fans. Run one at a time, show progress, and allow cancellation. ## Availability A diffusion model has to be registered and a backend has to serve it. On `win32-arm64` only QHexRT loads, so check `capabilities()` before offering this. # Installation Source: https://docs.runanywhere.ai/electron/installation Packages, the three-process model, and packaging ## Packages ```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} npm ci ``` ```json package.json theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} { "dependencies": { "@runanywhere/electron": "^0.20.31", "@runanywhere/electron-llamacpp": "^0.20.31", "@runanywhere/electron-neurt": "^0.20.31", "@runanywhere/electron-onnx": "^0.20.31", "@runanywhere/electron-qhexrt": "^0.20.31", "@runanywhere/electron-sherpa": "^0.20.31", "@runanywhere/proto-ts": "^0.20.31" } } ``` Every package carries its own prebuilt native binaries under `node_modules/@runanywhere//prebuilds/-/`. Nothing compiles from source, so `npm ci` is the whole staging step. Use `npm ci` rather than `npm install` so the committed lock file decides the tree. Node 20 or newer, per the package's own `engines` field. ## The three processes This is the part with no equivalent in the other SDKs. Electron splits the SDK across process roles, and each role imports a different subpath. | Process | Import | Role | | - | - | - | | Main | `@runanywhere/electron/main` | Forks the utility host, resolves natives, owns the window | | Preload | `@runanywhere/electron/preload` | Exposes `window.runanywhere` across the context bridge | | Utility host | `@runanywhere/electron/host` | Owns the native addon and runs inference | | Anywhere | `@runanywhere/electron` | `createRunAnywhere`, types, input constructors | | Registration | `@runanywhere/electron/backend` | Backend registration | | Audio | `@runanywhere/electron/audio` | Audio helpers | A renderer cannot `require` a native addon, so renderer code never imports the package directly. It uses `window.runanywhere`, which the preload puts there. ## Two backends ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} import { createRunAnywhere, NativeBackend, RpcBackend } from '@runanywhere/electron' // In-process: the addon is loaded here const RunAnywhere = createRunAnywhere(new NativeBackend()) // Over RPC: talks to the utility host const RunAnywhere = createRunAnywhere(new RpcBackend(send)) ``` `NativeBackend` loads the addon in the current process. `RpcBackend` forwards to the utility host, which is what keeps a heavy model load off the main process. ## Control-plane credentials Optional. Copy `.env.example` to `.env` and fill in `RUNANYWHERE_BASE_URL` and `RUNANYWHERE_API_KEY` to initialize in the production environment, which sends org-scoped telemetry. With both blank the SDK initializes keyless, in development. Either way inference stays on the machine. Real environment variables win over the file. ## Packaging Native artifacts cannot load from inside `app.asar`, so everything under `node_modules/@runanywhere/*/prebuilds/` has to be `asarUnpack`ed. Unpack both by extension (`.node`, `.dylib`, `.dll`, `.so`) and by the whole `prebuilds/**` tree. The tree rule is what covers QHexRT's `libqnnhtpv81.cat`, which is not a loadable image but whose absence makes the Hexagon skel fail signature verification. That failure reads as a corrupt model rather than a packaging fault, which is why it is worth getting right the first time. Unpacking is only half of it. The paths `register()` computes point inside `app.asar`, where Electron's fs shim makes them look real to JavaScript while the OS loader sees nothing. The addon path and each plugin path have to be rewritten to `app.asar.unpacked` before use. ## Compute device CPU. A `--gpu` flag asks for a CUDA build of the addon first and falls back to the CPU prebuild when it is absent. The published packages ship no CUDA prebuild, so that fallback is what happens today unless you supply a CUDA addon yourself. ## Windows installers The published installers carry no Authenticode certificate, so SmartScreen warns on first run. Local packages are unsigned for the same reason. # Introduction Source: https://docs.runanywhere.ai/electron/introduction RunAnywhere Electron SDK for Windows and macOS desktop apps The RunAnywhere Electron SDK runs language, speech, and vision models inside a desktop app. Prompts, documents, and audio stay on the machine, and everything keeps working with the network off. ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} import { createRunAnywhere, NativeBackend } from '@runanywhere/electron' const RunAnywhere = createRunAnywhere(new NativeBackend()) await RunAnywhere.initialize() const result = await RunAnywhere.llm.generate('Name three colours') console.log(result.text) ``` Unlike the other SDKs, the facade is **constructed** rather than imported as a singleton. `createRunAnywhere(backend)` is how you get one. ## Packages | Package | Version | Role | | - | - | - | | `@runanywhere/electron` | 0.20.31 | Core SDK: main-process host, preload, native addon | | `@runanywhere/electron-llamacpp` | 0.20.31 | LlamaCPP backend: LLM and VLM | | `@runanywhere/electron-onnx` | 0.20.31 | ONNX backend: embeddings, segmentation | | `@runanywhere/electron-sherpa` | 0.20.31 | Sherpa backend: STT, TTS, VAD | | `@runanywhere/electron-neurt` | 0.20.31 | Apple Neural Engine backend | | `@runanywhere/electron-qhexrt` | 0.20.31 | QHexRT: Qualcomm Hexagon NPU on Snapdragon X | | `@runanywhere/proto-ts` | 0.20.31 | Generated protobuf types | All seven publish at 0.20.31. Electron is the one platform where QHexRT is **not** pinned back to 0.20.19, because it ships as an npm prebuild rather than through Maven or pub.dev. ## Which platform gets which engines | platform-arch | Engines that load | | - | - | | `darwin-arm64` | llamacpp, onnx, sherpa | | `win32-x64` | llamacpp, onnx, sherpa | | `win32-arm64` | qhexrt only | | linux | no prebuild published; builds from source | The two Windows lanes are mutually exclusive by construction: QAIRT ships no Hexagon stub for x86\_64, and neither ggml nor ONNX Runtime builds for win-arm64. The NPU is the only engine on an ARM64 Windows host, with no CPU fallback behind it. Declaring all the backend packages unconditionally stays safe. A package with no payload for the running platform records a path that does not exist, and the SDK drops non-existent paths before forking the utility host. Only what your platform can run shows up in `capabilities().backends`. ## The namespaces Fourteen modality namespaces: `llm`, `vlm`, `stt`, `tts`, `vad`, `voice`, `rag`, `models`, `embeddings`, `rerank`, `lora`, `images`, `diarization`, `segmentation`. Five platform namespaces: `storage`, `logging`, `secure`, `auth`, `telemetry`. Input constructors also hang off the facade as `audio`, `image`, and `ragDocument`, so renderer code can build the same values main-process code builds. ## What Electron has that nothing else does * **Download control**: `models.pause`, `models.resume`, `models.cancel`, `models.interrupted()`. * **A delete dry-run**: `storage.deletePlan()` tells you what a delete would remove before you do it. * **A live log stream**: `logging.records()`. * **A `cancelled` terminal event** on generation streams. * **The full LoRA catalog surface**, matching Swift rather than the short form the other TypeScript SDKs carry. ## Where to go next Packages, processes, and packaging A working window Models, storage, logging, auth # Chat Source: https://docs.runanywhere.ai/electron/llm/chat Multi-turn conversations with message history Pass an array of messages instead of a string. Same method. ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const reply = await RunAnywhere.llm.generate([ { role: 'system', content: 'You are a concise assistant.' }, { role: 'user', content: 'What is on-device inference?' }, { role: 'assistant', content: 'Running the model on your own hardware.' }, { role: 'user', content: 'Why does that matter?' }, ]) ``` ## How messages are split The array is not sent verbatim: * System turns become `options.systemPrompt`. * The **trailing user turn** becomes the prompt. * Everything between travels as history. A conversation with no user turn throws. ## Streaming a conversation ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} for await (const event of RunAnywhere.llm.generateStream(messages)) { if (event.type === 'textDelta') reply += event.text } ``` ## Keeping a transcript ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} import type { ChatMessage } from '@runanywhere/electron' const messages: ChatMessage[] = [{ role: 'system', content: 'You are a helpful assistant.' }] async function send(text: string): Promise { messages.push({ role: 'user', content: text }) let reply = '' for await (const event of RunAnywhere.llm.generateStream(messages)) { if (event.type === 'textDelta') reply += event.text } messages.push({ role: 'assistant', content: reply }) } ``` ## Persisting across launches A desktop app is expected to remember. Conversations are plain values, so writing them as JSON under the app's user-data directory is enough. Persist the array, not SDK state: models reload from disk on the next launch, and history is all you need to carry. ## Context length Trim before the conversation outgrows the model's window, keeping the system message: ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} if (messages.length > 21) { messages.splice(1, messages.length - 21) } ``` A desktop can afford a larger `contextLength` at load time, which pushes this problem further out but does not remove it. # generate() Source: https://docs.runanywhere.ai/electron/llm/generate One-shot text generation ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const result = await RunAnywhere.llm.generate('Name three colours') console.log(result.text, result.tokensPerSecond) ``` ## Signature One method takes either a prompt or a conversation: ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} generate(input: string | ChatMessage[], options?: LlmOptions): Promise ``` ## With options ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const result = await RunAnywhere.llm.generate('Explain quantum computing', { model: 'qwen3-0.6b', maxOutputTokens: 200, temperature: 0.7, }) ``` Generation auto-loads what it needs, and downloads when `options.model` names a model that is absent. ## LlmOptions | Field | Type | Default | | - | - | - | | `model` | `string` | none; uses the loaded model | | `maxOutputTokens` | `number` | proto default | | `temperature` | `number` | proto default | | `topP` | `number` | proto default | | `systemPrompt` | `string` | none | | `stopSequences` | `string[]` | `[]` | | `reasoning` | `ReasoningOptions` | none | | `tools` | `ToolDefinition[]` | `[]` | | `toolChoice` | `ToolChoice` | `auto` | | `maxToolCalls` | `number` | `5` | A desktop has real headroom, so `maxOutputTokens` can be higher here than on a phone. It is still the biggest lever on how long the user waits. ## From the renderer The same call works through the context bridge: ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const result = await window.runanywhere.llm.generate('Hello') ``` ## Errors Throws `SDKException` when no model is loaded and `options.model` is absent, or when generation fails. See [Error handling](/electron/error-handling). # generateStream() Source: https://docs.runanywhere.ai/electron/llm/stream Stream tokens as they are produced ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} for await (const event of RunAnywhere.llm.generateStream('Write a haiku')) { if (event.type === 'textDelta') process.stdout.write(event.text) } ``` ## Signature ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} generateStream(input: string | ChatMessage[], options?: LlmOptions): AsyncIterableIterator ``` Not a promise. You get the iterator back synchronously and consume it with `for await`. ## The events In order: | Event | Meaning | | - | - | | `started` | Generation began | | `textDelta` | A chunk of output text | | `reasoningDelta` | A chunk of reasoning, for thinking models | | `toolCallAdded` | The model asked to call a tool | | `toolArgumentsDone` | Tool arguments finished streaming | | `usage` | Token counts | | `completed` / `failed` / `cancelled` | Exactly one of these ends the stream | Electron is the only SDK with a `cancelled` terminal event. Elsewhere a cancelled generation simply stops. The SDK never fabricates a successful `completed`. ## Where errors surface Preflight failures throw from the call: no model available, bad options. A failure **during** generation arrives as a `failed` event, so a window can render a retry without a `try`/`catch` around the loop. ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} for await (const event of RunAnywhere.llm.generateStream(prompt)) { if (event.type === 'failed') { showRetry(event.error) break } if (event.type === 'textDelta') output += event.text } ``` ## Streaming into the renderer Generation usually runs in the utility host while the text belongs on screen. Forward the deltas rather than the whole result: ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} for await (const event of RunAnywhere.llm.generateStream(prompt)) { if (event.type === 'textDelta') { win.webContents.send('token', event.text) } } ``` Sending per token across IPC is fine for text this small, but batching on a timer keeps a busy window smoother. ## Cancelling Break out of the loop. That produces a `cancelled` terminal event rather than an error. ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} for await (const event of RunAnywhere.llm.generateStream(prompt)) { if (aborted) break if (event.type === 'textDelta') output += event.text } ``` # System Prompts Source: https://docs.runanywhere.ai/electron/llm/system-prompts Control model behaviour with a system prompt ## Through options ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const result = await RunAnywhere.llm.generate('How do I reverse a string?', { systemPrompt: 'You are a TypeScript expert. Answer with code, then one sentence.', }) ``` ## Through a message A `system` message becomes `options.systemPrompt` automatically: ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const result = await RunAnywhere.llm.generate([ { role: 'system', content: 'You are a TypeScript expert.' }, { role: 'user', content: 'How do I reverse a string?' }, ]) ``` Equivalent. When both are present, the explicit `options.systemPrompt` wins. ## Writing one that works Even with desktop headroom, the models here are small. Short, concrete instructions land; long personas do not. ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} // Works 'You are a terse assistant. Answer in one sentence. No preamble.' // Works less well on a 0.6B model 'You are an extremely helpful, knowledgeable, and friendly AI assistant who always strives to provide the most comprehensive and detailed responses possible…' ``` Naming the exact output shape usually holds: ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} await RunAnywhere.llm.generate(review, { systemPrompt: 'You are a classifier. Reply with exactly one word: positive, negative, or neutral.', temperature: 0.1, }) ``` For a guaranteed shape rather than a requested one, use [structured output](/electron/structured-output). ## Reuse ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} import type { LlmOptions } from '@runanywhere/electron' export const prompts = { summarizer: { systemPrompt: 'Summarize in three bullet points. No introduction.', maxOutputTokens: 150, temperature: 0.3, }, coder: { systemPrompt: 'You are a TypeScript expert. Answer with code first.', temperature: 0.2, }, } satisfies Record ``` Keeping these in the main process rather than the renderer means a settings change does not need a window reload. # LoRA Adapters Source: https://docs.runanywhere.ai/electron/lora Layer an adapter onto the loaded base model A LoRA adapter specialises a base model without swapping it out. The base model stays loaded throughout, so switching behaviour costs far less than switching models. ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} await RunAnywhere.lora.apply('legal-summarizer', 0.8) const result = await RunAnywhere.llm.generate('Summarize this clause: …') await RunAnywhere.lora.remove('legal-summarizer') ``` ## The short form ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} lora.apply(adapterId: string, scale?: number): Promise lora.remove(adapterId?: string): Promise lora.removeAll(): Promise lora.list(): Promise lora.state(): Promise ``` ## Scale `scale` controls how strongly the adapter pulls the base model. Leave it unset for the adapter's own default. Lower values blend the adapter with base behaviour; higher values commit to it. If output degrades after applying an adapter, scale is the first thing to turn down. ## Catalog management Electron carries the full surface, matching Swift rather than the short form the other TypeScript SDKs have. ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} await RunAnywhere.lora.register(entry); await RunAnywhere.lora.registerArtifact(/* … */); await RunAnywhere.lora.download(/* … */); await RunAnywhere.lora.importAdapter(sourcePath); await RunAnywhere.lora.listCatalog(query?); await RunAnywhere.lora.queryCatalog(query); await RunAnywhere.lora.checkCompatibility(config); ``` `importAdapter(sourcePath)` takes a file from disk. On a desktop that is how most people will bring their own adapter, and it is worth wiring to a file picker. ## Check compatibility first ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const verdict = await RunAnywhere.lora.checkCompatibility(config) ``` An adapter is trained against a specific base model. Applying one to a different base either fails or produces nonsense, so check rather than discovering it in the output. ## Switching adapters Adapters stack. Remove before applying another, or two of them fight and the output is worse than either alone. ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} async function useAdapter(id: string): Promise { await RunAnywhere.lora.removeAll() await RunAnywhere.lora.apply(id) } ``` # Models Source: https://docs.runanywhere.ai/electron/models Find, download, pause, resume, load, and remove models Electron has the richest `models` namespace of any SDK. Download control exists only here. ## What is available ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const models = await RunAnywhere.models.list(filter?); const model = await RunAnywhere.models.get('qwen3-0.6b'); await RunAnywhere.models.refresh(options?); ``` ## Downloading ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} for await (const event of RunAnywhere.models.download('qwen3-0.6b')) { if (event.type === 'progress') { win.setProgressBar(event.fraction ?? 0) } } ``` On a desktop, models are the largest thing your app will ever fetch. Put the progress on the taskbar or dock icon with `setProgressBar` as well as in the window. ### Pause, resume, cancel ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} await RunAnywhere.models.pause('qwen3-0.6b') await RunAnywhere.models.resume('qwen3-0.6b') await RunAnywhere.models.cancel('qwen3-0.6b') ``` None of these exist on any other SDK. They are what makes a multi-gigabyte download survive a user closing the laptop lid. ### After a crash ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const stalled = await RunAnywhere.models.interrupted() // string[] for (const id of stalled) { if (await RunAnywhere.models.isResumable(id)) { // offer to continue rather than restart } } ``` Call `interrupted()` on launch. It is the difference between a user losing a 4GB download and not noticing it stopped. ## Loading ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const loaded = await RunAnywhere.models.load('qwen3-0.6b') const tuned = await RunAnywhere.models.load('qwen3-0.6b', { contextLength: 8192, threads: 8, }) ``` | Field | Meaning | | - | - | | `backendPreferences` | Ordered preference; each can be required | | `contextLength` | Context window to allocate | | `threads` | CPU threads | | `forceReload` | Reload even if already resident | | `framework` | Pin to one backend | | `useGpu` | Request GPU execution | A desktop has real headroom, so a larger `contextLength` and higher `threads` are reasonable here in a way they are not on a phone. ### Which backend will serve it ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const capabilities = await RunAnywhere.capabilities() console.log(capabilities.backends) ``` Generated from packaging facts about the linked addon. On `win32-arm64` you will see `qhexrt` alone, with no CPU fallback behind it; on `win32-x64` and `darwin-arm64` you will see llamacpp, onnx, and sherpa. ## Unloading ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} await RunAnywhere.models.unload('qwen3-0.6b') ``` ## Residency policy ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} import { ResidencyPolicy } from '@runanywhere/electron' ``` `ResidencyPolicy`, `ResidencyDecision`, `ResidencySlots`, and `ResidentModel` control which models stay loaded when memory is contended. On a desktop app with a chat model and a voice session in play, this is what stops one evicting the other on every turn. ## Deleting, with a dry run ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const plan = await RunAnywhere.storage.deletePlan() // show the user exactly what would go await RunAnywhere.storage.delete(request) ``` `deletePlan` exists only on Electron. Show it before deleting, rather than deleting and reporting afterwards. ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} await RunAnywhere.models.delete('qwen3-0.6b') await RunAnywhere.models.unregister('qwen3-0.6b') await RunAnywhere.storage.clearCache() await RunAnywhere.storage.cleanTempFiles() ``` ### Will it fit ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const availability = await RunAnywhere.storage.availability(request) ``` Ask before starting a download rather than failing partway. ## Registering your own model ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const info = await RunAnywhere.models.register(registration) ``` ### Gated repositories ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} await RunAnywhere.setHfToken(token) ``` `null` falls back to the environment lookup the core resolves (`HF_TOKEN`, then `$HF_TOKEN_PATH`, then `$HF_HOME/token`, then `~/.cache/huggingface/token`), so `hf auth login` is honoured. An empty string clears the token and disables that fallback. The token lives in the platform secure store and is re-applied on the next `initialize()`. ## Renderer access Everything above works identically through `window.runanywhere` in the renderer, so a settings page can drive downloads without the main process proxying each call: ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} for await (const event of window.runanywhere.models.download(id)) { if (event.type === 'progress') bar.value = event.fraction ?? 0 } ``` # Quick Start Source: https://docs.runanywhere.ai/electron/quick-start A working window ## 1. Construct the facade ```ts main.ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} import { createRunAnywhere, NativeBackend } from '@runanywhere/electron' const RunAnywhere = createRunAnywhere(new NativeBackend()) await RunAnywhere.initialize() ``` `initialize()` is single-phase here: one call, no second services phase to await. ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} interface InitializeOptions { apiKey?: string baseUrl?: string environment?: Environment } ``` ## 2. Generate text ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const result = await RunAnywhere.llm.generate('Explain on-device AI in one sentence', { model: 'qwen3-0.6b', maxOutputTokens: 100, }) console.log(result.text, result.tokensPerSecond) ``` ## 3. Stream it ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} for await (const event of RunAnywhere.llm.generateStream('Write a haiku')) { if (event.type === 'textDelta') process.stdout.write(event.text) } ``` The stream emits `started`, `textDelta` / `reasoningDelta`, `toolCallAdded` / `toolArgumentsDone`, `usage`, then exactly one terminal `completed`, `failed`, or `cancelled`. Electron is the only SDK with a `cancelled` terminal event. ## 4. Reach it from the renderer The preload exposes the same facade as `window.runanywhere`, so page code calls the identical methods without importing a native addon. ```ts renderer.ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const result = await window.runanywhere.llm.generate('Hello') output.textContent = result.text ``` Input constructors come along for the ride, which is what lets renderer code build the same values main-process code builds: ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const image = window.runanywhere.image.bytes(data) const result = await window.runanywhere.vlm.generate(image, 'What is this?') ``` ## 5. Download a model with progress Electron is the only SDK where a download can be paused and resumed. ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} for await (const event of RunAnywhere.models.download('qwen3-0.6b')) { if (event.type === 'progress') bar.value = event.fraction ?? 0 } await RunAnywhere.models.pause('qwen3-0.6b') await RunAnywhere.models.resume('qwen3-0.6b') await RunAnywhere.models.cancel('qwen3-0.6b') const stalled = await RunAnywhere.models.interrupted() // string[] const canResume = await RunAnywhere.models.isResumable('qwen3-0.6b') ``` ## 6. Add speech ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const transcription = await RunAnywhere.stt.transcribe(RunAnywhere.audio.wav(bytes)) const handle = await RunAnywhere.tts.speak('Ready when you are') ``` ## Verify it is really running locally ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const capabilities = await RunAnywhere.capabilities() console.log(capabilities.backends) for (const missing of capabilities.unavailable) { console.log(missing.name, missing.reason) } ``` `capabilities()` is generated from packaging facts about the linked addon, not from namespace presence. On Windows ARM64 you will see `qhexrt` alone; on x64 and macOS you will see llamacpp, onnx, and sherpa. ## Next Models, storage, logging, auth Processes and packaging # RAG Source: https://docs.runanywhere.ai/electron/rag Answer questions over a folder of documents Open a session with an embedding model, add documents, ask questions. ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const session = await RunAnywhere.rag.open(/* embedding model, llm model, config */) await session.ingest(RunAnywhere.ragDocument.file(path)) const answer = await session.query('What is the refund window?') console.log(answer.text) await session.close() ``` ## RagSession ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} session.ingest(document | documents): Promise session.search(query: string, topK?: number): Promise session.query(question: string, options?: RagQueryOptions): Promise session.queryStream(question: string, options?: RagQueryOptions): AsyncIterableIterator session.stats(): Promise session.clear(): Promise session.close(): Promise ``` ## Ingesting from disk A desktop app can point RAG at a real directory, which is the thing it has over the browser. ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} import { readdir } from 'node:fs/promises' import { join } from 'node:path' const files = await readdir(folder) for (const name of files) { if (!name.endsWith('.md') && !name.endsWith('.txt')) continue await session.ingest(RunAnywhere.ragDocument.file(join(folder, name))) } ``` Ingestion chunks, embeds, and indexes. It is the slow part. Do it once, persist the index, and reopen it on the next launch rather than re-ingesting at every start. Show progress: a folder of a few hundred documents takes real time, and a frozen window looks like a crash. ## Retrieval without generation ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const matches = await session.search('refund policy', 5) ``` Useful for a search box over the same index, or for showing sources next to an answer. ## Streaming an answer ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} for await (const event of session.queryStream('What is the refund window?')) { if (event.type === 'token') answer += event.text if (event.type === 'sources') renderSources(event.matches) } ``` Sources usually arrive before the first token, so you can show what the answer draws on while it is still being written. ## Configuration | Field | Meaning | | - | - | | `retrievalTopK` | Chunks retrieved per question | | `chunkSize` | Characters per chunk | | `chunkOverlap` | Overlap between neighbouring chunks | | `similarityThreshold` | Drop matches below this score | | `persistPath` | Where the index lives, for reuse across launches | `persistPath` is what turns a slow first launch into a fast second one. Point it somewhere under the app's user-data directory. Overlap matters more than it looks. Without it, a sentence split across a chunk boundary is retrievable from neither half. ## Two models at once A RAG session holds an embedding model and a language model. Embedding models are small, often tens of megabytes, so the pair is far lighter than two language models. See [residency policy](/electron/models#residency-policy) when several sessions compete. ## Closing ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} await session.close() ``` Close on window close, not only on quit. A session left open holds both models. # Segmentation Source: https://docs.runanywhere.ai/electron/segmentation Find and label regions in an image Segmentation outlines objects in an image and labels them. Where a [vision model](/electron/vlm) describes a picture in words, segmentation gives you the regions themselves: a mask per object, with a class and a confidence. ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const result = await RunAnywhere.segmentation.segment(RunAnywhere.image.file(path), { includeDiagnosticImage: true, }) ``` ## Building the input ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} RunAnywhere.image.file(path) RunAnywhere.image.bytes(data) RunAnywhere.image.rawRgb(data, width, height) ``` Passing a path rather than bytes avoids reading a large image into JavaScript memory only to hand it back to native code. ## Options | Field | Default | Meaning | | - | - | - | | `includeDiagnosticImage` | `false` | Return a rendered overlay alongside the regions | Turn the diagnostic image on while you are building the feature: it shows you what the model actually found, which is much faster than reasoning about mask arrays. Turn it off in production, since rendering it costs time and memory. ## What comes back A `SegmentationResult` holds the regions found. Each carries its class label, a confidence, and the mask itself, so you can draw an outline, crop to the object, or count instances. ## Segmentation or a vision model | You want | Use | | - | - | | A sentence about the picture | [VLM](/electron/vlm) | | "Is there a dog in this" | VLM | | The exact pixels of the dog | Segmentation | | Object counts, or a crop | Segmentation | | Anything you will draw on the image | Segmentation | ## Availability Segmentation runs on the ONNX backend, which does not load on `win32-arm64`. Check `capabilities()` before offering it. # Structured Output Source: https://docs.runanywhere.ai/electron/structured-output Get JSON back instead of prose Asking a model for JSON in a system prompt works most of the time, which is the problem. `generateStructured` validates the result against a schema and can repair it. ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} interface Person { name: string age: number } const result = await RunAnywhere.llm.generateStructured( 'Extract the name and age: Dana is 34.', schema, 'validationOnly', { temperature: 0.1 } ) ``` Electron's `generateStructured` is generic over the parsed type, so the result is typed without a cast. ## Enforcement modes | Mode | Behaviour | | - | - | | `'validationOnly'` | Generate freely, then validate. The default. | | `'repair'` | Validate, then retry once with a repair instruction. | | `'constrained'` | **Throws today.** | Validation-only is right when a retry is cheap. Repair costs a second generation when the first is invalid, which is worth it when the call is user-facing. ## The schema ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const schema = { /* your JSON schema */ } ``` Keep schemas small. A small model handles four flat fields reliably and a deeply nested structure badly. When you need something complex, ask for the pieces separately. Naming fields the way a person would helps: `totalPrice` gets better results than `tp`. ## When to reach for this | Task | Approach | | - | - | | Extracting fields from text | Structured output | | Classifying into fixed labels | Structured output, or a one-word prompt | | Anything you will `JSON.parse` | Structured output | | Prose for a human to read | Plain generation | If you are about to write a regex over a model's reply, you want this instead. ## Temperature Lower it. Structured extraction is not a creative task, and `0.1` to `0.3` produces far more consistent field values than the default. # STT Options Source: https://docs.runanywhere.ai/electron/stt/options Configure transcription ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const transcription = await RunAnywhere.stt.transcribe(audio, { language: 'en', punctuation: true, wordTimestamps: true, diarization: false, translateToEnglish: false, }) ``` ## Fields | Field | Type | Default | Meaning | | - | - | - | - | | `language` | `string` | none, auto-detect | BCP-47 code such as `'en'` | | `punctuation` | `boolean` | proto default | Insert punctuation and casing | | `wordTimestamps` | `boolean` | proto default | Per-word timings | | `diarization` | `boolean` | `false` | Attribute speech to speakers | | `maxSpeakers` | `number` | none | Cap the speaker count | | `translateToEnglish` | `boolean` | `false` | Translate rather than transcribe | ## Language Leave `language` unset to auto-detect. Set it when you know the language: detection costs time and picks wrong on short clips. ## Word timestamps ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} await RunAnywhere.stt.transcribe(audio, { wordTimestamps: true }) ``` Use these for captions, or to seek a player to a spoken word. On a desktop app transcribing long recordings, this is what makes the transcript navigable. ## Diarization `diarization: true` labels speakers inside the transcription. For a standalone "who spoke when" answer, use the [diarization namespace](/electron/diarization). ## Translation `translateToEnglish: true` returns English regardless of source language. ## Long recordings A desktop is the one platform where transcribing an hour of audio is reasonable. Word timestamps and diarization both cost extra time per minute of audio, so turn on only what the interface will actually show. # Streaming STT Source: https://docs.runanywhere.ai/electron/stt/stream Transcribe speech as it arrives Two shapes. Pick by where your audio comes from. ## Push: openStream ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const stream = await RunAnywhere.stt.openStream({ encoding: 'pcmF32Le', sampleRate: 16000, channels: 1, }) // per buffer stream.pushFrame({ samples, sampleCount }) // when the utterance ends stream.flush() stream.finish() await stream.close() ``` ## Pull: transcribeStream ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} for await (const event of RunAnywhere.stt.transcribeStream(audioIterable)) { if (event.type === 'partial') preview = event.text if (event.type === 'final') transcript += event.transcription.text } ``` ## Signatures ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} openStream(format: AudioFormatSpec, options?: SttOptions): Promise transcribeStream(audio: AsyncIterable, options?: SttOptions): AsyncIterableIterator ``` ## Capturing in the renderer, transcribing in the host The microphone lives in the renderer, where `getUserMedia` works. Inference lives in the utility host. Forward frames across the bridge: ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} // renderer const media = await navigator.mediaDevices.getUserMedia({ audio: true }) // … convert to PCM frames, then window.runanywhere.stt.pushFrame(frame) ``` Keep the conversion in the renderer. Sending raw `MediaStream` data across IPC per callback is far more traffic than sending the PCM you actually need. ## Closing Always `flush()`, `finish()`, and `close()`. A stream left open holds the model and the microphone, and on a desktop app that outlives a window close. ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} win.on('closed', () => { void stream.close() }) ``` ## Choosing between them Push suits a live microphone, since the capture API already hands you frames in a callback. Pull suits a source you can express as an async iterable, such as a file read in chunks or audio arriving over the network. # transcribe() Source: https://docs.runanywhere.ai/electron/stt/transcribe Convert speech to text ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const transcription = await RunAnywhere.stt.transcribe(RunAnywhere.audio.wav(bytes)) console.log(transcription.text) ``` ## Signature ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} transcribe(input: AudioInput, options?: SttOptions): Promise ``` ## Building an AudioInput Input constructors hang off the facade as `RunAnywhere.audio`, so renderer code builds the same values main-process code does: ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} RunAnywhere.audio.pcm16(bytes, sampleRate?, channels?) RunAnywhere.audio.float32(samples, sampleRate?, channels?) RunAnywhere.audio.wav(bytes) RunAnywhere.audio.file(path) ``` `file` is the desktop path: point it at a recording on disk with no reading step of your own. ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} import { readFile } from 'node:fs/promises' const transcription = await RunAnywhere.stt.transcribe(RunAnywhere.audio.file(path)) ``` ## With options ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const transcription = await RunAnywhere.stt.transcribe(audio, { language: 'en', punctuation: true, wordTimestamps: true, }) ``` See [STT options](/electron/stt/options). ## Checking readiness ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} const state = await RunAnywhere.stt.state() ``` ## Microphone permission On macOS, recording needs `NSMicrophoneUsageDescription` in the app's Info.plist and the user's consent. Electron surfaces the prompt on first use, and a denial is permanent until the user changes it in System Settings, so handle the refusal rather than retrying. # Tool Calling Source: https://docs.runanywhere.ai/electron/tool-calling Let the model call your functions Register a tool and the function that runs it. The SDK owns the call-and-execute loop and detects the format the model expects. ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} RunAnywhere.llm.tools.register(weatherTool, async (args) => ({ tempC: await fetchTemp(String(args.city)), })) const result = await RunAnywhere.llm.generate('What is the weather in Lisbon?') ``` ## The tools namespace `RunAnywhere.llm.tools` carries `register`, `unregister`, and `list`. ## Defining a tool ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} import type { ToolDefinition } from '@runanywhere/electron' const weatherTool: ToolDefinition = { name: 'get_weather', description: 'Get the current temperature for a city', parameters: [ { name: 'city', type: 'string', description: 'City name', required: true, }, ], } ``` ## Controlling the loop | Field | Default | Meaning | | - | - | - | | `tools` | `[]` | Tools for this call only | | `toolChoice` | `auto` | Whether the model may call a tool | | `maxToolCalls` | `5` | Cap on calls in one exchange | Tools registered through `tools.register` are offered to every generation. Tools passed in `options.tools` apply to that call alone. ## Executors run in the main process This is what Electron has over the browser and the phone. A tool executor here has the full Node API: the filesystem, child processes, the network with no CORS, and any native module you have installed. ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} RunAnywhere.llm.tools.register(readFileTool, async (args) => { const contents = await readFile(String(args.path), 'utf8') return { contents } }) ``` That power cuts both ways. A model deciding which file to read is a model deciding which file to read. Validate the arguments, confine paths to a directory you chose, and never hand a tool a shell string the model composed. ## Watching the calls happen The stream reports tool activity, which is what lets you show "checking the weather…" rather than a stalled cursor: ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} for await (const event of RunAnywhere.llm.generateStream(prompt)) { if (event.type === 'toolCallAdded') status.textContent = `Calling ${event.name}…` if (event.type === 'textDelta') output.textContent += event.text } ``` ## Which models can do this Tool calling needs a model with enough context to hold the tool definitions. The SDK gates on context length rather than a model allowlist, so any model with a large enough window can participate, though instruction-tuned models trained on tool use do it far more reliably. A desktop can afford a larger `contextLength` at load time, which helps here. # Streaming TTS Source: https://docs.runanywhere.ai/electron/tts/stream Start speaking before the whole text is synthesized `synthesizeStream` emits `AudioChunk` values as they are produced, so playback starts on the first chunk instead of the last. ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} for await (const chunk of RunAnywhere.tts.synthesizeStream(longText)) { player.enqueue(chunk) } ``` ## Signature ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} synthesizeStream(text: string, options?: TtsOptions): AsyncIterableIterator ``` ## When it is worth it For a sentence, `speak` is simpler and the latency difference is invisible. For a paragraph, or for a model reply being streamed as it generates, streaming synthesis is what keeps the gap between "the model finished" and "the user hears something" short. ## Pairing it with a streaming completion Speak each sentence as the model finishes it: ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} let sentence = '' for await (const event of RunAnywhere.llm.generateStream(prompt)) { if (event.type !== 'textDelta') continue sentence += event.text if (/[.!?]/.test(event.text)) { await RunAnywhere.tts.speak(sentence) sentence = '' } } if (sentence) await RunAnywhere.tts.speak(sentence) ``` For a full conversation rather than one reply, use the [voice session](/electron/voice-agent), which handles turn-taking and interruption. ## Forwarding chunks to the renderer When the audio graph lives in the window, send chunks as they arrive rather than buffering the whole utterance in the main process: ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} for await (const chunk of RunAnywhere.tts.synthesizeStream(text)) { win.webContents.send('audio-chunk', chunk) } ``` Audio chunks are much larger than text deltas, so batching matters here in a way it does not for tokens. Send each chunk once and let the renderer schedule them against a running clock. ## Stopping ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} await RunAnywhere.tts.stop() ``` # synthesize() Source: https://docs.runanywhere.ai/electron/tts/synthesize Turn text into audio ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} // Get the audio back const audio = await RunAnywhere.tts.synthesize('Hello from RunAnywhere') // Play it const handle = await RunAnywhere.tts.speak('Hello from RunAnywhere') ``` ## Signatures ```ts theme={"theme":{"light":"vitesse-light","dark":"vitesse-black"}} synthesize(text: string, options?: TtsOptions): Promise