RAG

Open a session with an embedding model, add documents, ask questions.

const session = await RunAnywhere.rag.open(/* embedding model, llm model, config */)

await session.ingest(RunAnywhere.ragDocument.file(path))

const answer = await session.query('What is the refund window?')
console.log(answer.text)

await session.close()

RagSession

session.ingest(document | documents): Promise<void>
session.search(query: string, topK?: number): Promise<Match[]>
session.query(question: string, options?: RagQueryOptions): Promise<RagResult>
session.queryStream(question: string, options?: RagQueryOptions): AsyncIterableIterator<RagEvent>
session.stats(): Promise<RagStats>
session.clear(): Promise<void>
session.close(): Promise<void>

Ingesting from disk

A desktop app can point RAG at a real directory, which is the thing it has over the browser.

import { readdir } from 'node:fs/promises'
import { join } from 'node:path'

const files = await readdir(folder)

for (const name of files) {
  if (!name.endsWith('.md') && !name.endsWith('.txt')) continue
  await session.ingest(RunAnywhere.ragDocument.file(join(folder, name)))
}

Ingestion chunks, embeds, and indexes. It is the slow part. Do it once, persist the index, and reopen it on the next launch rather than re-ingesting at every start. Show progress: a folder of a few hundred documents takes real time, and a frozen window looks like a crash.

Retrieval without generation

const matches = await session.search('refund policy', 5)

Useful for a search box over the same index, or for showing sources next to an answer.

Streaming an answer

for await (const event of session.queryStream('What is the refund window?')) {
  if (event.type === 'token') answer += event.text
  if (event.type === 'sources') renderSources(event.matches)
}

Sources usually arrive before the first token, so you can show what the answer draws on while it is still being written.

Configuration

Field	Meaning
`retrievalTopK`	Chunks retrieved per question
`chunkSize`	Characters per chunk
`chunkOverlap`	Overlap between neighbouring chunks
`similarityThreshold`	Drop matches below this score
`persistPath`	Where the index lives, for reuse across launches

persistPath is what turns a slow first launch into a fast second one. Point it somewhere under the app’s user-data directory. Overlap matters more than it looks. Without it, a sentence split across a chunk boundary is retrievable from neither half.

Two models at once

A RAG session holds an embedding model and a language model. Embedding models are small, often tens of megabytes, so the pair is far lighter than two language models. See residency policy when several sessions compete.

Closing

await session.close()

Close on window close, not only on quit. A session left open holds both models.

Getting Started

Swift SDK

Kotlin SDK

React Native SDK

Flutter SDK

Web SDK

Electron SDK

Vibe Coding

RagSession

Ingesting from disk

Retrieval without generation

Streaming an answer

Configuration

Two models at once

Closing

​RagSession

​Ingesting from disk

​Retrieval without generation

​Streaming an answer

​Configuration

​Two models at once

​Closing

RagSession

Ingesting from disk

Retrieval without generation

Streaming an answer

Configuration

Two models at once

Closing