Skip to main content

​
Serve the isolation headers first

Without them there is no SharedArrayBuffer and nothing loads. Every other problem is downstream of this one.

​
Register a backend before initializing

autoRegister picks the CPU or WebGPU build for the current browser, so you do not have to detect it yourself.

​
Read capabilities rather than assuming

A namespace existing does not mean a browser engine backs it. images and segmentation have no registered engine today. Check unavailable before putting a feature in the interface.

​
Respect the gesture rules

Three things need a user gesture, and all three fail silently or confusingly without one:
  • RunAnywhere.storage.chooseDirectory()
  • microphone access
  • audio playback
Put them behind a click. Not behind a useEffect, not behind a timer.

​
Pick small models

A browser download is the user’s bandwidth and the user’s disk. A 4GB model is not a reasonable thing to fetch on a page load. Show the size before downloading, and default to the smallest model that does the job.

​
Stream, and batch the DOM writes

Inference runs in a worker, so the main thread stays free. What stalls a page is writing to the DOM on every token. Batch with requestAnimationFrame.

​
Cap output length

maxOutputTokens is the biggest lever on latency.

​
Unload what you are not using

A tab has less headroom than a phone. Holding a language model, a speech model, and a vision model at once is how you get insufficientMemory.

​
Close sessions on pagehide

Voice sessions hold the microphone. A session that survives a navigation keeps the browser’s recording indicator lit.

​
Offer a real directory for repeat visitors

OPFS is the default and is wiped when the user clears site data. For anyone who will return, chooseDirectory() puts the models somewhere durable and turns a repeat 2GB download into nothing.

​
One RAG session at a time

The web index is process-wide. Opening a second session while one is open throws. Close the first.

​
Remember which methods are synchronous

models.list, models.get, models.register, llm.tools.*, and tts.stop are synchronous on Web and async on the other SDKs. Code ported from Swift or React Native will have await in the wrong places.

​
Test in Safari

Safari’s isolation and storage behaviour differ enough from Chromium that “works in Chrome” tells you very little. The isolation service worker also forces a one-time reload there.