> ## Documentation Index
> Fetch the complete documentation index at: https://docs.runanywhere.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# RAG

> Ingest documents and answer questions from them

`RunAnywhere.rag.open` creates a session that owns one corpus and its index. The session loads the
embedding model and, when you pass one, the language model.

```dart theme={null}
final session = await RunAnywhere.rag.open(
  embeddingModel: const ModelRef('all-minilm-l6-v2'),
  llmModel: const ModelRef('smollm2-360m-q8_0'),
);

await session.ingest(const RagDocument('The sky is blue because of Rayleigh scattering.'));

final answer = await session.query('Why is the sky blue?');
print(answer.answer);
for (final source in answer.sources) {
  print('${source.score}: ${source.text}');
}

await session.close();
```

Leave `llmModel` null for a retrieval-only session. `search` still works; `query` and `queryStream`
throw.

<Warning>
  The Flutter RAG bridge owns a single native session. Opening a second session supersedes the
  first, and calls on the superseded session throw `SDKException`. Close one before opening the
  next.
</Warning>

## open

```dart theme={null}
Future<RagSession> open({
  required ModelRef embeddingModel,
  ModelRef? llmModel,
  RagConfig? config,
});
```

It throws `SDKException` when a model cannot be loaded or the index cannot be created. The RAG backend
registers itself here, so there is no separate wiring step.

## RagConfig

```dart theme={null}
final session = await RunAnywhere.rag.open(
  embeddingModel: const ModelRef('all-minilm-l6-v2'),
  llmModel: const ModelRef('smollm2-360m-q8_0'),
  config: RagConfig(
    topK: 5,
    chunkSize: 512,
    chunkOverlap: 64,
    similarityThreshold: 0.3,
    persistPath: '/path/to/index',
  ),
);
```

| Field                 | Type      | Default | Notes                                         |
| --------------------- | --------- | ------- | --------------------------------------------- |
| `topK`                | `int`     | 5       | Chunks retrieved per query                    |
| `chunkSize`           | `int`     | 512     | Characters per indexed chunk                  |
| `chunkOverlap`        | `int`     | 64      | Characters shared between adjacent chunks     |
| `similarityThreshold` | `double?` | `null`  | Minimum similarity for a chunk to be returned |
| `persistPath`         | `String?` | `null`  | Null keeps the index in memory                |

## The session

```dart theme={null}
final bool generates;                                        // true when an LLM was supplied
Future<void> ingest(RagDocument document);
Future<void> ingestAll(List<RagDocument> documents);
Future<List<Match>> search(String query, {int? topK});
Future<RagResult> query(String question, {LlmOptions? options});
Stream<RagEvent> queryStream(String question, {LlmOptions? options});
Future<RagStats> stats();
Future<void> clear();
Future<void> close();
```

`clear` drops every document but keeps the session open. `close` destroys the index.

## Ingesting

```dart theme={null}
// In-memory text
await session.ingest(const RagDocument('Some text to index.'));

// With metadata echoed back on every match
await session.ingest(RagDocument(
  chapterText,
  metadata: const {'chapter': '3', 'source': 'handbook'},
));

// From disk
await session.ingest(await RagDocument.file('/path/to/notes.txt'));

// Several at once
await session.ingestAll([doc1, doc2, doc3]);
```

`RagDocument.file` reads UTF-8 text and throws when the path does not exist. It sets `sourceUri` for
you. `ingestAll` runs the documents one at a time and stops at the first failure.

## Retrieval without generation

```dart theme={null}
final matches = await session.search('Rayleigh scattering', topK: 3);

for (final match in matches) {
  print('${match.score.toStringAsFixed(3)}  ${match.text}');
  print(match.metadata);
}
```

`Match` carries `text`, `score`, and `metadata`. `topK` overrides the session config for one call.

## Answering

```dart theme={null}
final result = await session.query(
  'Why is the sky blue?',
  options: LlmOptions(maxOutputTokens: 200, temperature: 0.3),
);

print(result.answer);
print('${result.metrics.outputTokens} tokens');
```

`RagResult` carries `answer`, `sources` (the `Match` list the answer was grounded in), and `metrics`,
a full `GenerationResult` for the generation half.

## Streaming an answer

```dart theme={null}
await for (final event in session.queryStream('Why is the sky blue?')) {
  switch (event) {
    case RagRetrieved(:final matches):
      setState(() => _sources = matches);
    case RagToken(:final text, :final kind):
      if (kind == TokenKind.text) setState(() => _answer += text);
    case RagCompleted(:final result):
      setState(() => _answer = result.answer);
  }
}
```

Retrieval finishes first, so `RagRetrieved` arrives before any token. `RagEvent` is sealed.

## Statistics

```dart theme={null}
final stats = await session.stats();
print('${stats.documentCount} documents');
print('${stats.chunkCount} chunks');
print('${stats.indexSizeBytes} bytes');
```

## Models

RAG needs an embedding model, and an embedding model needs its vocabulary next to the weights, which
means a multi-file registration.

```dart theme={null}
await RunAnywhere.models.register(
  ModelRegistration.multiFile(
    id: 'all-minilm-l6-v2',
    name: 'All MiniLM L6 v2 (Embedding)',
    files: [
      ModelFileDescriptor(
        filename: 'model.onnx',
        url: 'https://huggingface.co/Xenova/all-MiniLM-L6-v2/resolve/main/onnx/model.onnx',
        isRequired: true,
      ),
      ModelFileDescriptor(
        filename: 'vocab.txt',
        url: 'https://huggingface.co/Xenova/all-MiniLM-L6-v2/resolve/main/vocab.txt',
        isRequired: true,
      ),
    ],
    framework: InferenceFramework.INFERENCE_FRAMEWORK_ONNX,
    category: ModelCategory.MODEL_CATEGORY_EMBEDDING,
    memoryRequirementBytes: 90000000,
  ),
);
```

Register the ONNX backend with `await Onnx.register()` before opening a session.

## Complete example

```dart theme={null}
class RagViewModel extends ChangeNotifier {
  RagSession? _session;
  String answer = '';
  List<Match> sources = const [];

  Future<void> load(String documentText) async {
    await _session?.close();
    final session = await RunAnywhere.rag.open(
      embeddingModel: const ModelRef('all-minilm-l6-v2'),
      llmModel: const ModelRef('smollm2-360m-q8_0'),
    );
    _session = session;
    await session.ingest(RagDocument(documentText));
    notifyListeners();
  }

  Future<void> ask(String question) async {
    final session = _session;
    if (session == null) return;

    answer = '';
    sources = const [];
    notifyListeners();

    await for (final event in session.queryStream(question)) {
      switch (event) {
        case RagRetrieved(:final matches):
          sources = matches;
        case RagToken(:final text, :final kind):
          if (kind == TokenKind.text) answer += text;
        case RagCompleted(:final result):
          answer = result.answer;
          sources = result.sources;
      }
      notifyListeners();
    }
  }

  @override
  void dispose() {
    _session?.close();
    super.dispose();
  }
}
```

## See also

<CardGroup cols={2}>
  <Card title="Embeddings" icon="vector-square" href="/flutter/embeddings">
    Vectors and reranking
  </Card>

  <Card title="Models" icon="box" href="/flutter/models">
    Multi-file registration
  </Card>
</CardGroup>