# AI gateway

The AI gateway lets your app call AI models with no provider account, no API key of its own and no separate bill. It speaks OpenAI's chat-completions format and Anthropic's Messages format, so the official SDKs work unmodified. AI is prepaid: your organization buys AI credits on the Billing page, and each call is paid from them at its model's price (see [Paying for AI](#paying-for-ai)).

```js
// The whole integration: no base URL, no API key.
const client = new OpenAI();

const completion = await client.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{ role: "user", content: "Summarise this ticket" }]
});
```

This works because attaching the gateway to an app sets `OPENAI_BASE_URL` and `OPENAI_API_KEY` in its environment, and the OpenAI SDK reads both on its own.

## Your data leaves Canada

Your apps and data are hosted in Canada, but the models behind the gateway are run by third-party providers (OpenAI, Anthropic and Google) in the United States. Prompts and content your app sends through the gateway go to those providers.

External AI is off by default. Before anyone in your organization can use the gateway, an owner or admin must choose **Acknowledge & enable external AI** on the notice the console shows when you first attach AI. The acknowledgement applies to the whole organization and is recorded with who made it and when. Until then, gateway requests are refused with `external_ai_not_acknowledged`. See [Security](https://nimbusland.ca/security) for the list of providers.

## Attaching AI to an app

Open the app's **Resources** tab, choose **Add dependency** and pick **AI gateway**. The dialog lets you rename the two variables before attaching (or keep the names the SDK expects). Attaching:

1. creates a gateway key that belongs to that app,
2. sets the two variables on the app, and restarts it so they take effect.

| Variable (default name) | Value |
|---|---|
| `OPENAI_BASE_URL` | The gateway's address, reachable from your app |
| `OPENAI_API_KEY` | The app's gateway key, stored as a [secret](secret-env-vars.md) |

Both are managed by the platform: you can see them in the app's environment variables but can't edit or delete them there, from the CLI or from the API. If your app already has its own `OPENAI_API_KEY`, the dialog points out the clash and offers to replace it; or rename the gateway's variables (for example with a `NIMBUS_AI_` prefix) to keep both. To change the names later, choose **Edit** on the dependency.

- **Rotate key** replaces the app's key; the old one stops working immediately.
- **Detach** revokes the key, removes the variables and restarts the app.

An app has one AI gateway attachment. It is one kind of [dependency](dependencies.md).

### Using the Anthropic SDK from your app

Attaching sets only the OpenAI-style pair. To use the Anthropic SDK in a deployed app, give it the injected base URL without its trailing `/v1`, and the injected key:

```python
import os
from anthropic import Anthropic

client = Anthropic(
    base_url=os.environ["OPENAI_BASE_URL"].removesuffix("/v1"),
    api_key=os.environ["OPENAI_API_KEY"],
)
```

## Using the gateway from your own machine

To develop against the gateway from a laptop, CI or anywhere outside a deployed app, use a **developer key**: a gateway key that belongs to your organization rather than to an app.

Open **Intelligence → API keys**. The page shows the gateway's public address under **Gateway endpoint**. Enter a label and choose **Mint key**; the key is shown once. You need the **Manage intelligence** permission.

```sh
export OPENAI_BASE_URL=https://nimbusland.ca/ai/v1
export OPENAI_API_KEY=nak_...      # your developer key
```

Unmodified OpenAI SDK code then works exactly as it does when deployed.

- **Mint one key per person or system**, so each can be revoked on its own. Revoking takes effect immediately. The page shows each key's label, creator and spend this period.
- **Developer keys are not tied to an app.** Their usage doesn't appear in any app's figures, and app spend caps don't apply to them. Your organization's spend cap does.
- **Each key is rate limited.** Over the limit, requests get `429 rate_limit_exceeded` with a `retry-after` header.

Don't copy a deployed app's key for local work: it shares the app's spend cap and usage, and rotating it after a leak also rotates production.

### Claude Code and the Anthropic SDK

The gateway serves Anthropic's Messages API, so Claude Code and the Anthropic SDKs work against it. Point them at the gateway address **without** `/v1` (they add `/v1/messages` themselves) and use a developer key:

```sh
export ANTHROPIC_BASE_URL=https://nimbusland.ca/ai
export ANTHROPIC_AUTH_TOKEN=nak_...   # a developer key
unset ANTHROPIC_API_KEY               # so your own Anthropic key isn't used instead
claude
```

```python
from anthropic import Anthropic

client = Anthropic(base_url="https://nimbusland.ca/ai", api_key="nak_...")
client.messages.create(
    model="claude-sonnet-5-5",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Summarise this ticket"}],
)
```

Name a model by its gateway id (from `GET /ai/v1/models`) or by Anthropic's own model id, including dated ids such as `claude-haiku-4-5-20251001`. If Claude Code's default model isn't available, set `ANTHROPIC_MODEL` to one that is.

The gateway bills only what it can meter, so these are refused with `400 invalid_request_error`:

- **Server tools**: any `tools[]` entry with a `type`, such as web search, web fetch or code execution. In Claude Code, start with `--disallowedTools=WebSearch` so the agent doesn't try web search. Your own (client-side) tools work, including Claude Code's WebFetch.
- **MCP servers** in the request (`mcp_servers`). MCP servers Claude Code runs itself are unaffected.
- **Fast mode** (`speed` other than `"standard"`).
- **A chosen inference region** (`inference_geo`).
- **Service tiers** other than `"standard_only"`.

The Batches API is not available. Prompt caching works.

## Endpoints

All endpoints are under `https://nimbusland.ca/ai/v1` (from a deployed app, use `OPENAI_BASE_URL`). Authenticate with `Authorization: Bearer <key>` or `x-api-key: <key>`, using an app key or a developer key.

| Endpoint | Format |
|---|---|
| `POST /ai/v1/chat/completions` | OpenAI chat completions, streaming supported |
| `POST /ai/v1/messages` | Anthropic Messages, streaming supported |
| `POST /ai/v1/messages/count_tokens` | Anthropic token counting; not billed |
| `GET /ai/v1/models` | The available models, in OpenAI's model-list format |

### Chat completions

Supported: `model`, `messages`, `temperature`, `max_tokens` / `max_completion_tokens`, `top_p`, `top_k`, `stop`, `frequency_penalty`, `presence_penalty`, `seed`, `user`, `tools`, `tool_choice`, `stream`. `response_format` works with OpenAI models only. `stream_options` is accepted with `include_usage`; every stream's last chunk carries usage anyway.

A request that sets neither `max_tokens` nor `max_completion_tokens` is sent to the model with a limit of 8,192 output tokens, the same figure that is reserved from your credits for it. Set your own limit if you need longer or shorter answers.

If a streamed answer fails part-way, the stream ends with an error chunk and `data: [DONE]`, and you pay only for what was generated before the failure.

These are refused with an `unsupported_parameter` error rather than silently ignored: `n` greater than 1, `logit_bias`, `logprobs`, `top_logprobs`, `parallel_tool_calls`, `service_tier`, `store`, `metadata`, `modalities`, `audio`, `prediction`, `functions`, `function_call`.

### Models

`GET /ai/v1/models` lists the models you can use. Each entry adds a `display_name` and a `residency` field saying where inference runs (`us`, `ca` or `eu`), so you can choose models against a data-residency requirement:

```json
{
  "object": "list",
  "data": [
    { "id": "gpt-4o-mini", "object": "model", "owned_by": "nimbus",
      "display_name": "GPT-4o mini", "residency": "us" }
  ]
}
```

## Paying for AI

AI is paid for up front with AI credits, not on your monthly invoice.

- **Buying credits**: an owner, or anyone with the **Manage billing** permission, buys credits under **Billing** in **AI credits**: $10, $25, $50 or $100, or any whole-dollar amount from $10 to $500. The card on file is charged at once, with sales tax added at purchase, and the receipt appears under **Invoices** as "AI credits". If your bank asks you to confirm the payment, the page links to the payment page, and the credits are added once it goes through. A declined card adds nothing and charges nothing. If the payment provider can't be reached, nothing is charged and you can try again; that never counts as a decline. If a purchase is refunded or disputed, the credits it added are taken back, which can leave the balance below zero until you buy more.
- **Spending credits**: each call is paid from the balance at its model's price in Canadian dollars. The Billing page shows the balance, AI used this period and recent purchases.
- **Purchase limits**: at most 3 purchases and $500 a day (a rolling 24 hours), or $100 a day while the organization is new. After 3 declined payments in 24 hours, buying credits is locked for 24 hours; contact support if a lock is a mistake.
- **Knowledge**: indexing a knowledge source and the embeddings behind knowledge search are paid from AI credits like any other AI call. At zero credits, indexing pauses and says why, and continues once you buy more.
- **Running out**: while the balance is zero or below, AI requests are refused with `ai_credits_exhausted` until more credits are bought. Buying credits makes AI available again within seconds, with the same keys.
- **Reserved for requests in progress**: before a request is sent to the model, the most it could cost is set aside from your credits: its prompt plus its `max_tokens` (8,192 output tokens if it sets none), at the model's price. When it finishes, only what it actually cost is taken and the rest is released. A request that could cost more than your balance is refused with the same `ai_credits_exhausted` code and a message giving both amounts, so set `max_tokens` to what you need or buy more credits. A request your balance would cover, but that doesn't fit next to what your other requests in progress have reserved, gets `429 rate_limit_exceeded` with a `retry-after` header instead: retry it once they finish (agents such as OpenCode and Claude Code do this on their own). The same applies to an agent task whose other requests have reserved the rest of its budget. **Billing** shows the amount reserved while requests are running. Rarely, a request costs a little more than was set aside, and the balance goes slightly below zero; the next purchase covers that first.
- **Not on your bill**: AI isn't on the monthly invoice, doesn't use your plan's included usage and doesn't count toward your spending budget, so a reached budget never stops AI.
- **Low-balance alerts**: billing managers get an email when the balance falls below $5 and again when it runs out.

The app's AI dialog and the developer keys page say so when your organization is out of credits.

## Spend caps

Every organization has a monthly AI spend cap, so a runaway app or a leaked key can only spend so much of your credits.

- **Organization cap**: set it under **Billing** in **Monthly AI limit** (needs the **Manage billing** permission). Without your own setting, a platform default applies. It covers every key, app and developer alike.
- **App cap**: optional, set in **App spend cap ($/month)** on the app's AI gateway dependency. It applies on its own, so one app can't use up the rest of the organization's budget.

Caps are in Canadian dollars, as billed: a $50 cap stops requests once $50 of credits have been spent on AI this period, whatever the provider's own price for it was. A cap applies even with credits left: it limits how fast credits can be spent.

The dependency also shows the app's usage this period: tokens, cost and requests. Spend can run slightly past a cap for a few seconds before requests are refused.

## Errors

The chat-completions endpoint returns errors in OpenAI's shape, so SDK error handling works:

```json
{"error": {"message": "...", "type": "...", "code": "...", "param": null}}
```

| Status | Code | Meaning |
|---|---|---|
| 400 | `unsupported_parameter` | A parameter the gateway doesn't support |
| 401 | `invalid_api_key` | Missing, unknown or revoked key |
| 403 | `external_ai_not_acknowledged` | External AI hasn't been enabled for your organization |
| 403 | `billing_not_active` | Your organization's billing isn't active |
| 403 | `app_suspended` | AI is turned off for this app |
| 404 | `model_not_found` | Not an available model |
| 429 | `insufficient_quota` | A spend cap was reached |
| 429 | `ai_credits_exhausted` (type `insufficient_quota`) | Your organization is out of AI credits; buy credits on the Billing page |
| 429 | `rate_limit_exceeded` | The key is over its rate limit (see `retry-after`), your other requests in progress have reserved the credits this one needs (retry after `retry-after`), or the provider is rate limiting |
| 502 | `upstream_error` | The model provider failed |
| 503 | `backend_unavailable` | The model isn't usable right now |
| 504 | `upstream_timeout` | The model provider didn't answer in time |

The Messages endpoints use Anthropic's error shape (`{"type": "error", "error": {"type": "...", "message": "..."}}`) so the Anthropic SDK and Claude Code handle them:

| Status | `type` | Meaning |
|---|---|---|
| 400 | `invalid_request_error` | A refused feature (see above) or a malformed request |
| 401 | `authentication_error` | Missing, unknown or revoked key |
| 403 | `permission_error` | External AI not enabled, billing not active, app suspended, or a spend cap reached ("AI spend cap reached …") |
| 403 | `permission_error` | Your organization is out of AI credits ("This organization is out of AI credits …") |
| 404 | `not_found_error` | Not an available Anthropic model |
| 429 | `rate_limit_error` | The key is over its rate limit, or your other requests in progress have reserved the credits this one needs (see `retry-after`) |

On the Messages endpoints a spend cap or running out of credits is a 403, not a 429, so clients such as Claude Code stop instead of retrying. Other provider errors are passed through with the provider's own status and body.