Put inference in the utility host
RpcBackend forwards to the utility process; NativeBackend loads the addon in the current
one. Use the host for anything heavy, or a model load blocks the main process and the window
stops responding.
Unpack the natives
Everything undernode_modules/@runanywhere/*/prebuilds/ has to be asarUnpacked, both by
extension and by the whole tree. The tree rule is what covers QHexRT’s libqnnhtpv81.cat,
whose absence reads as a corrupt model rather than a packaging fault.
Read capabilities, do not assume
win32-arm64 only QHexRT loads, with
no CPU fallback, so vision, segmentation, and speech are unavailable there. Check before
putting a feature in a menu.
Offer pause and resume
Download control exists only on Electron. A user closing a laptop lid mid-download should not lose four gigabytes.Show a delete plan
Use residency policy with several models
A chat model and a voice session compete. Without a policy, one evicts the other on every turn and the user waits through a reload each time.Close sessions on window close, not only on quit
Validate tool arguments
Tool executors here have the filesystem, child processes, and the network with no CORS. A model choosing which file to read is a model choosing which file to read. Confine paths to a directory you picked, and never build a shell string from model output.Batch what crosses the bridge
Token deltas are small enough to forward individually. Audio chunks are not. Send audio once per chunk and let the renderer schedule playback against a running clock.Send error codes, not error objects
Structured clone drops the prototype, so anSDKException sent to the renderer arrives as a
plain object and instanceof fails. Forward code and message explicitly.
Size the load for the machine
A desktop can afford a largercontextLength and higher threads than a phone. Both still cost
memory, so read what the machine has rather than hardcoding numbers that work on yours.