OpenAI-compatible API

At a glance

Two changes: your base URL and your API key.
Supported: chat, completions, embeddings, models and rerank, with streaming.
Different: the error format and a few platform-managed fields, listed below.

Endpoints

Each Tenant has one base URL for all of its models. Your console shows it next to each deployment, with ready-to-copy snippets.

Endpoint	Supported
`POST /chat/completions`	Yes, including streaming
`POST /completions`	Yes, including streaming
`POST /embeddings`	Yes, for embedding models
`GET /models`	Yes: lists your ready deployments and adapters
`POST /rerank`	Yes, for reranking models (Cohere- and Jina-style request shape)

Not available: audio, images, files, the Batch API and the OpenAI fine-tuning API. Fine-tuning has its own rackAI API and console pages.

Using your existing SDK

Point the OpenAI Python SDK, or any OpenAI-compatible client, at your Tenant's base URL and use a rackAI API key:

from openai import OpenAI

client = OpenAI(
    base_url="<your Tenant's base URL from the console>",
    api_key="<your rackAI API key>",
)

response = client.chat.completions.create(
    model="<your deployment name>",
    messages=[{"role": "user", "content": "Hello"}],
    stream=True,
)

The console also gives curl and Vercel AI SDK examples.

Choosing a model or adapter

Set the model field to:

a deployment name, to call that deployment's base model;
a fine-tuned adapter's name, or <deployment>/<adapter>, to call an adapter;
a route name, if you've set up a route that splits traffic between deployments.

Responses return the name you used.

Authentication

Send your API key as a bearer token. Each key belongs to one Tenant and is inference-only by default. Create and revoke keys in the console.

Where rackAI differs from OpenAI

Request parameters pass through to the model server. Support for tool calling, structured outputs (response_format) and log probabilities depends on the model and how its deployment is configured.
Platform-managed fields. rackAI sets model routing, cache isolation and usage reporting itself. Streaming responses always include usage in their final chunk.
Error format. Errors raised by rackAI itself (authentication, permissions, unknown models, a model still starting, request too large) use the standard HTTP problem format (application/problem+json) with a code and a requestId, not OpenAI's error object.
Limits. Request bodies are limited to 4 MiB. Streams that go quiet for two minutes are closed, and requests are capped at 30 minutes. A deployment that is still starting answers 503 with Retry-After.