AI gateway

The AI gateway lets your app call AI models with no provider account, no API key of its own and no separate bill. It speaks OpenAI's chat-completions format and Anthropic's Messages format, so the official SDKs work unmodified. AI is prepaid: your organization buys AI credits on the Billing page, and each call is paid from them at its model's price (see Paying for AI).

// The whole integration: no base URL, no API key.
const client = new OpenAI();

const completion = await client.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{ role: "user", content: "Summarise this ticket" }]
});

This works because attaching the gateway to an app sets OPENAI_BASE_URL and OPENAI_API_KEY in its environment, and the OpenAI SDK reads both on its own.

Your data leaves Canada#

Your apps and data are hosted in Canada, but the models behind the gateway are run by third-party providers (OpenAI, Anthropic and Google) in the United States. Prompts and content your app sends through the gateway go to those providers.

External AI is off by default. Before anyone in your organization can use the gateway, an owner or admin must choose Acknowledge & enable external AI on the notice the console shows when you first attach AI. The acknowledgement applies to the whole organization and is recorded with who made it and when. Until then, gateway requests are refused with external_ai_not_acknowledged. See Security for the list of providers.

Attaching AI to an app#

Open the app's Resources tab, choose Add dependency and pick AI gateway. The dialog lets you rename the two variables before attaching (or keep the names the SDK expects). Attaching:

1. creates a gateway key that belongs to that app,
2. sets the two variables on the app, and restarts it so they take effect.

Both are managed by the platform: you can see them in the app's environment variables but can't edit or delete them there, from the CLI or from the API. If your app already has its own OPENAI_API_KEY, the dialog points out the clash and offers to replace it; or rename the gateway's variables (for example with a NIMBUS_AI_ prefix) to keep both. To change the names later, choose Edit on the dependency.

• Rotate key replaces the app's key; the old one stops working immediately.
• Detach revokes the key, removes the variables and restarts the app.

An app has one AI gateway attachment. It is one kind of dependency.

Using the Anthropic SDK from your app#

Attaching sets only the OpenAI-style pair. To use the Anthropic SDK in a deployed app, give it the injected base URL without its trailing /v1, and the injected key:

import os
from anthropic import Anthropic

client = Anthropic(
    base_url=os.environ["OPENAI_BASE_URL"].removesuffix("/v1"),
    api_key=os.environ["OPENAI_API_KEY"],
)

Using the gateway from your own machine#

To develop against the gateway from a laptop, CI or anywhere outside a deployed app, use a developer key: a gateway key that belongs to your organization rather than to an app.

Open Intelligence → API keys. The page shows the gateway's public address under Gateway endpoint. Enter a label and choose Mint key; the key is shown once. You need the Manage intelligence permission.

export OPENAI_BASE_URL=https://nimbusland.ca/ai/v1
export OPENAI_API_KEY=nak_...      # your developer key

Unmodified OpenAI SDK code then works exactly as it does when deployed.

• Mint one key per person or system, so each can be revoked on its own. Revoking takes effect immediately. The page shows each key's label, creator and spend this period.
• Developer keys are not tied to an app. Their usage doesn't appear in any app's figures, and app spend caps don't apply to them. Your organization's spend cap does.
• Each key is rate limited. Over the limit, requests get 429 rate_limit_exceeded with a retry-after header.

Don't copy a deployed app's key for local work: it shares the app's spend cap and usage, and rotating it after a leak also rotates production.

Claude Code and the Anthropic SDK#

The gateway serves Anthropic's Messages API, so Claude Code and the Anthropic SDKs work against it. Point them at the gateway address without /v1 (they add /v1/messages themselves) and use a developer key:

export ANTHROPIC_BASE_URL=https://nimbusland.ca/ai
export ANTHROPIC_AUTH_TOKEN=nak_...   # a developer key
unset ANTHROPIC_API_KEY               # so your own Anthropic key isn't used instead
claude

from anthropic import Anthropic

client = Anthropic(base_url="https://nimbusland.ca/ai", api_key="nak_...")
client.messages.create(
    model="claude-sonnet-5-5",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Summarise this ticket"}],
)

Name a model by its gateway id (from GET /ai/v1/models) or by Anthropic's own model id, including dated ids such as claude-haiku-4-5-20251001. If Claude Code's default model isn't available, set ANTHROPIC_MODEL to one that is.

The gateway bills only what it can meter, so these are refused with 400 invalid_request_error:

• Server tools: any tools[] entry with a type, such as web search, web fetch or code execution. In Claude Code, start with --disallowedTools=WebSearch so the agent doesn't try web search. Your own (client-side) tools work, including Claude Code's WebFetch.
• MCP servers in the request (mcp_servers). MCP servers Claude Code runs itself are unaffected.
• Fast mode (speed other than "standard").
• A chosen inference region (inference_geo).
• Service tiers other than "standard_only".

The Batches API is not available. Prompt caching works.

Endpoints#

All endpoints are under https://nimbusland.ca/ai/v1 (from a deployed app, use OPENAI_BASE_URL). Authenticate with Authorization: Bearer <key> or x-api-key: <key>, using an app key or a developer key.

Chat completions#

Supported: model, messages, temperature, max_tokens / max_completion_tokens, top_p, top_k, stop, frequency_penalty, presence_penalty, seed, user, tools, tool_choice, stream. response_format works with OpenAI models only. stream_options is accepted with include_usage; every stream's last chunk carries usage anyway.

A request that sets neither max_tokens nor max_completion_tokens is sent to the model with a limit of 8,192 output tokens, the same figure that is reserved from your credits for it. Set your own limit if you need longer or shorter answers.

If a streamed answer fails part-way, the stream ends with an error chunk and data: [DONE], and you pay only for what was generated before the failure.

These are refused with an unsupported_parameter error rather than silently ignored: n greater than 1, logit_bias, logprobs, top_logprobs, parallel_tool_calls, service_tier, store, metadata, modalities, audio, prediction, functions, function_call.

Models#

GET /ai/v1/models lists the models you can use. Each entry adds a display_name and a residency field saying where inference runs (us, ca or eu), so you can choose models against a data-residency requirement:

{
  "object": "list",
  "data": [
    { "id": "gpt-4o-mini", "object": "model", "owned_by": "nimbus",
      "display_name": "GPT-4o mini", "residency": "us" }
  ]
}

Paying for AI#

AI is paid for up front with AI credits, not on your monthly invoice.

• Buying credits: an owner, or anyone with the Manage billing permission, buys credits under Billing in AI credits: $10, $25, $50 or $100, or any whole-dollar amount from $10 to $500. The card on file is charged at once, with sales tax added at purchase, and the receipt appears under Invoices as "AI credits". If your bank asks you to confirm the payment, the page links to the payment page, and the credits are added once it goes through. A declined card adds nothing and charges nothing. If the payment provider can't be reached, nothing is charged and you can try again; that never counts as a decline. If a purchase is refunded or disputed, the credits it added are taken back, which can leave the balance below zero until you buy more.
• Spending credits: each call is paid from the balance at its model's price in Canadian dollars. The Billing page shows the balance, AI used this period and recent purchases.
• Purchase limits: at most 3 purchases and $500 a day (a rolling 24 hours), or $100 a day while the organization is new. After 3 declined payments in 24 hours, buying credits is locked for 24 hours; contact support if a lock is a mistake.
• Knowledge: indexing a knowledge source and the embeddings behind knowledge search are paid from AI credits like any other AI call. At zero credits, indexing pauses and says why, and continues once you buy more.
• Running out: while the balance is zero or below, AI requests are refused with ai_credits_exhausted until more credits are bought. Buying credits makes AI available again within seconds, with the same keys.
• Reserved for requests in progress: before a request is sent to the model, the most it could cost is set aside from your credits: its prompt plus its max_tokens (8,192 output tokens if it sets none), at the model's price. When it finishes, only what it actually cost is taken and the rest is released. A request that could cost more than your balance is refused with the same ai_credits_exhausted code and a message giving both amounts, so set max_tokens to what you need or buy more credits. A request your balance would cover, but that doesn't fit next to what your other requests in progress have reserved, gets 429 rate_limit_exceeded with a retry-after header instead: retry it once they finish (agents such as OpenCode and Claude Code do this on their own). The same applies to an agent task whose other requests have reserved the rest of its budget. Billing shows the amount reserved while requests are running. Rarely, a request costs a little more than was set aside, and the balance goes slightly below zero; the next purchase covers that first.
• Not on your bill: AI isn't on the monthly invoice, doesn't use your plan's included usage and doesn't count toward your spending budget, so a reached budget never stops AI.
• Low-balance alerts: billing managers get an email when the balance falls below $5 and again when it runs out.

The app's AI dialog and the developer keys page say so when your organization is out of credits.

Spend caps#

Every organization has a monthly AI spend cap, so a runaway app or a leaked key can only spend so much of your credits.

• Organization cap: set it under Billing in Monthly AI limit (needs the Manage billing permission). Without your own setting, a platform default applies. It covers every key, app and developer alike.
• App cap: optional, set in App spend cap ($/month) on the app's AI gateway dependency. It applies on its own, so one app can't use up the rest of the organization's budget.

Caps are in Canadian dollars, as billed: a $50 cap stops requests once $50 of credits have been spent on AI this period, whatever the provider's own price for it was. A cap applies even with credits left: it limits how fast credits can be spent.

The dependency also shows the app's usage this period: tokens, cost and requests. Spend can run slightly past a cap for a few seconds before requests are refused.

Errors#

The chat-completions endpoint returns errors in OpenAI's shape, so SDK error handling works:

{"error": {"message": "...", "type": "...", "code": "...", "param": null}}

The Messages endpoints use Anthropic's error shape ({"type": "error", "error": {"type": "...", "message": "..."}}) so the Anthropic SDK and Claude Code handle them:

On the Messages endpoints a spend cap or running out of credits is a 403, not a 429, so clients such as Claude Code stop instead of retrying. Other provider errors are passed through with the provider's own status and body.