> ## Documentation Index
> Fetch the complete documentation index at: https://docs.runanywhere.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# OpenAI-compatible API

> Models, chat completions, streaming, and errors

## Base URL and authentication

```text theme={null}
https://inference.runanywhere.ai/v1
Authorization: Bearer sk-runa-…
```

Cloud keys are separate from device SDK keys. Never log a key or include one in
client-side analytics. The console's playground receives the endpoint and
model from your signed-in session; production applications should keep the
key server-side.

## Models and pricing

Models are billed per million tokens against prepaid credits. Production Wally
uses zero data retention for prompt and completion
bodies. The current models and their
rates are on
[console.runanywhere.ai](https://console.runanywhere.ai), and
[refund terms](https://runanywhere.ai/legal/refunds) cover how unused credits come back.

Entitlement is per environment, so a key that works in development can get a `403` in
production for the same model id. Treat the `/models` response for your own key as the
authority on what you can call.

## List models

```bash theme={null}
curl https://inference.runanywhere.ai/v1/models \
  -H "Authorization: Bearer $RUNA_CLOUD_KEY"
```

The response follows the OpenAI list shape. Treat the returned IDs as the
authority; do not hardcode a model that is absent from the current response.

## Chat completions

The endpoint is OpenAI-compatible, so the official OpenAI clients work by pointing `base_url`
at it.

For Python, install the client with `python3 -m pip install openai`. Set
`RUNA_CLOUD_KEY` in your environment to your cloud API key, then replace
`<model-id>` with an ID returned by `/models` before running the example.

<CodeGroup>
  ```bash cURL theme={null}
  curl -N https://inference.runanywhere.ai/v1/chat/completions \
    -H "Authorization: Bearer $RUNA_CLOUD_KEY" \
    -H 'Content-Type: application/json' \
    -d '{
      "model": "<model-id>",
      "stream": true,
      "messages": [{"role": "user", "content": "Explain edge AI in one sentence."}]
    }'
  ```

  ```python Python theme={null}
  import os

  from openai import OpenAI

  client = OpenAI(
      base_url="https://inference.runanywhere.ai/v1",
      api_key=os.environ["RUNA_CLOUD_KEY"],
  )

  stream = client.chat.completions.create(
      model="<model-id>",
      messages=[{"role": "user", "content": "Explain edge AI in one sentence."}],
      stream=True,
  )
  for chunk in stream:
      # Usage-only chunks have no choices.
      if chunk.choices:
          print(chunk.choices[0].delta.content or "", end="", flush=True)
  ```

  ```javascript JavaScript theme={null}
  import OpenAI from 'openai'

  const client = new OpenAI({
    baseURL: 'https://inference.runanywhere.ai/v1',
    apiKey: process.env.RUNA_CLOUD_KEY,
  })

  const stream = await client.chat.completions.create({
    model: '<model-id>',
    messages: [{ role: 'user', content: 'Explain edge AI in one sentence.' }],
    stream: true,
  })
  for await (const chunk of stream) process.stdout.write(chunk.choices[0]?.delta?.content ?? '')
  ```
</CodeGroup>

### Body parameters

<ParamField body="model" type="string" required>
  The model id to call. The `/models` response for your key is the authority on what is valid.
</ParamField>

<ParamField body="messages" type="array" required>
  The conversation so far, in OpenAI's `{(role, content)}` shape.
</ParamField>

<ParamField body="stream" type="boolean" default="false">
  Send the reply as SSE chunks instead of one response.
</ParamField>

<ParamField body="temperature" type="number">
  Sampling temperature.
</ParamField>

<ParamField body="max_tokens" type="integer">
  Cap on tokens generated for this reply.
</ParamField>

<ParamField body="tools" type="array">
  Tool definitions the model may call.
</ParamField>

<ParamField body="tool_choice" type="string | object">
  Which tool the model must use, if any.
</ParamField>

<ParamField body="stream_options" type="object">
  Streaming extras, such as asking for a usage chunk.
</ParamField>

The service may reject fields outside the current contract. Streaming sends JSON chunks after
`data: `, optionally includes a usage chunk, and terminates with `data: [DONE]`.

## Errors and request IDs

Errors use the following safe shape; prompts, completions, and upstream bodies
are not echoed:

```json theme={null}
{
  "error": {
    "message": "The request could not be admitted.",
    "type": "capacity",
    "code": null,
    "param": null
  }
}
```

Error responses carry `x-request-id` for support and usage correlation. The contract does not
declare that header on successful responses, so do not depend on it there.
Common statuses are `400` (invalid request), `401` (unknown key), `403`
(not entitled), `429` (rate or capacity), `502` (engine failure), `503` (not
ready), and `504` (upstream timeout). Capacity `429` responses may include
`Retry-After`.