AI Model APIs

Chat

Chat supports three vendor protocols. The model determines the format: Anthropic models use Anthropic Messages, Google models use the native Gemini format, and other LLMs use the OpenAI-compatible format.

Choose by model vendor

FormatEndpointApplies to
OpenAI Chat CompletionsPOST /v1/chat/completionsLLMs from vendors other than Anthropic and Google
Anthropic MessagesPOST /v1/messagesClaude models from Anthropic
Gemini generateContentPOST /v1beta/models/{model}:generateContentGemini models from Google

All three formats share one API keyThe API key and billing account are shared, but the path, auth header and request body are determined by the model vendor. The formats are not interchangeable. Get model IDs from GET /v1/models or the Model Catalog.

OpenAI format

POST https://api.tokgate.io/v1/chat/completions

For LLMs from vendors other than Anthropic and Google. Supports streaming and non-streaming responses and follows OpenAI /v1/chat/completions.

Request parameters

ParameterTypeRequiredDescription
modelstringRequiredThe model ID.
messagesarray<object>RequiredThe conversation messages, each with a role (system / user / assistant / tool) and content. For vision models, content can be an array mixing text and image_url.
temperaturenumberOptionalSampling temperature, 0 ~ 2, default 1.
top_pnumberOptionalNucleus sampling, 0 ~ 1, default 1. Adjust either this or temperature, not both.
nintegerOptionalNumber of candidates to generate, default 1.
streambooleanOptionalWhether to stream the response, default false.
stream_optionsobjectOptionalStreaming options, e.g. {"include_usage": true} attaches usage stats to the final chunk.
stopstring / arrayOptionalStop sequences — a string or an array of strings.
max_tokensintegerOptionalMaximum number of tokens to generate.
max_completion_tokensintegerOptionalMaximum completion tokens. In the current gateway it behaves differently from max_tokens; reasoning tokens may also consume this limit and reduce visible output, so test each target model separately.
presence_penaltynumberOptionalPresence penalty, -2 ~ 2, default 0.
frequency_penaltynumberOptionalFrequency penalty, -2 ~ 2, default 0.
logit_biasobjectOptionalSampling bias for the specified tokens.
toolsarray<object>OptionalList of tool (function) definitions the model may call.
tool_choicestring / objectOptionalTool selection mode: none / auto / required, or an object naming a specific function.
response_formatobjectOptionalStructured output, e.g. {"type": "json_object"} or {"type": "json_schema", ...}.
seedintegerOptionalRandom seed for better reproducibility (exact identity not guaranteed).
reasoning_effortstringOptionalReasoning effort: low / medium / high; only effective on reasoning-capable models.
userstringOptionalEnd-user identifier, useful for abuse investigation.

Parameters depend on the modelDifferent models support different parameter sets. When you pass a parameter the model does not support, the gateway either ignores it or returns 400. Model capabilities (vision, tool calling, JSON mode, streaming) are shown as capability tags in the Model Catalog.

Request example

curl -X POST https://api.tokgate.io/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-***" \
  -d '{
    "model": "deepseek-v4-pro",
    "messages": [
      {"role": "system", "content": "You are a rigorous technical assistant"},
      {"role": "user", "content": "Explain what nucleus sampling is"}
    ],
    "temperature": 0.7,
    "max_tokens": 1024
  }'
from openai import OpenAI

client = OpenAI(
    api_key="sk-***",
    base_url="https://api.tokgate.io/v1",
)

resp = client.chat.completions.create(
    model="deepseek-v4-pro",
    messages=[
        {"role": "system", "content": "You are a rigorous technical assistant"},
        {"role": "user", "content": "Explain what nucleus sampling is"},
    ],
    temperature=0.7,
    max_tokens=1024,
)
print(resp.choices[0].message.content)
print(resp.usage)
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.TOKGATE_API_KEY,
  baseURL: "https://api.tokgate.io/v1",
});

const resp = await client.chat.completions.create({
  model: "deepseek-v4-pro",
  messages: [
    { role: "system", content: "You are a rigorous technical assistant" },
    { role: "user", content: "Explain what nucleus sampling is" },
  ],
  temperature: 0.7,
  max_tokens: 1024,
});
console.log(resp.choices[0].message.content);

Response structure

{
  "id": "chatcmpl-xxxxxxxx",
  "object": "chat.completion",
  "created": 1779348818,
  "model": "deepseek-v4-pro",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Nucleus sampling (top-p) is...",
        "reasoning_content": null,
        "tool_calls": null
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 42,
    "completion_tokens": 186,
    "total_tokens": 228,
    "prompt_tokens_details": { "cached_tokens": 0 },
    "completion_tokens_details": { "reasoning_tokens": 0 }
  },
  "system_fingerprint": null
}
FieldDescription
choices[].finish_reasonWhy generation stopped: stop (finished naturally) / length (hit the length limit) / tool_calls (requesting a tool call) / content_filter (content blocked).
choices[].message.tool_callsTools the model requests to call, with id, function.name and function.arguments (a JSON string).
choices[].message.reasoning_contentThe reasoning trace returned by reasoning models; null for regular models.
usageToken usage of this call, for reconciliation. Cache hits are reported in prompt_tokens_details.cached_tokens.

Streaming

Set stream: true and the server returns chunks via Server-Sent Events, ending with data: [DONE]. Ideal for typewriter-style chat UIs.

stream = client.chat.completions.create(
    model="deepseek-v4-pro",
    messages=[{"role": "user", "content": "Tell a short story about overseas expansion"}],
    stream=True,
    stream_options={"include_usage": True},
)
for chunk in stream:
    if chunk.choices:
        print(chunk.choices[0].delta.content or "", end="", flush=True)
const stream = await client.chat.completions.create({
  model: "deepseek-v4-pro",
  messages: [{ role: "user", content: "Tell a short story about overseas expansion" }],
  stream: true,
});
for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}
data: {"id":"chatcmpl-xxx","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null}]}

data: {"id":"chatcmpl-xxx","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"Once"},"finish_reason":null}]}

data: {"id":"chatcmpl-xxx","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]

Tool calling

Declare functions in tools and the model returns tool_calls when needed. After you execute the function, append the result to messages as a role: "tool" message and request again; the model then produces the final answer.

{
  "model": "deepseek-v4-pro",
  "messages": [{"role": "user", "content": "What's the weather in Shenzhen right now?"}],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get the real-time weather of a given city",
        "parameters": {
          "type": "object",
          "properties": {
            "city": {"type": "string", "description": "City name, e.g. Shenzhen"}
          },
          "required": ["city"]
        }
      }
    }
  ],
  "tool_choice": "auto"
}

Anthropic format

POST https://api.tokgate.io/v1/messages

For Claude models from Anthropic only. Key differences between Anthropic Messages and OpenAI: system is a top-level field; max_tokens is required; response content is a block array; usage fields are input_tokens / output_tokens.

Request headers

HeaderRequiredDescription
x-api-keyRequiredYour API key. You can also use Authorization: Bearer <key> instead.
anthropic-versionRequiredProtocol version, fixed at 2023-06-01. The official SDK sets it automatically.
Content-TypeRequiredapplication/json

Request parameters

ParameterTypeRequiredDescription
modelstringRequiredThe model ID.
messagesarray<object>RequiredConversation messages; role can only be user or assistant and must alternate. content can be a string or an array of content blocks (text / image / tool_use / tool_result).
max_tokensintegerRequiredMaximum number of tokens to generate. Required in the Anthropic format.
systemstring / arrayOptionalSystem prompt, a top-level field. Do not put it into messages.
temperaturenumberOptionalSampling temperature, 0 ~ 1.
top_pnumberOptionalNucleus sampling, 0 ~ 1.
top_kintegerOptionalSample only from the K most likely tokens.
streambooleanOptionalWhether to stream the response, default false.
stop_sequencesarray<string>OptionalCustom stop sequences. When hit, stop_reason is stop_sequence.
toolsarray<object>OptionalTool definitions with name / description / input_schema.
tool_choiceobjectOptionalE.g. {"type": "auto"}, {"type": "any"}, {"type": "tool", "name": "..."}.
thinkingobjectOptionalExtended thinking, e.g. {"type": "enabled", "budget_tokens": 4096}; only effective on models that support it.
metadataobjectOptionalExtra information, e.g. {"user_id": "..."}.

Request example

curl -X POST https://api.tokgate.io/v1/messages \
  -H "Content-Type: application/json" \
  -H "x-api-key: sk-***" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
    "model": "claude-sonnet-4.6",
    "max_tokens": 1024,
    "system": "You are a rigorous technical assistant",
    "messages": [
      {"role": "user", "content": "Explain what nucleus sampling is"}
    ]
  }'
from anthropic import Anthropic

client = Anthropic(
    api_key="sk-***",
    base_url="https://api.tokgate.io",
)

msg = client.messages.create(
    model="claude-sonnet-4.6",
    max_tokens=1024,
    system="You are a rigorous technical assistant",
    messages=[{"role": "user", "content": "Explain what nucleus sampling is"}],
)
print(msg.content[0].text)
print(msg.usage)
import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic({
  apiKey: process.env.TOKGATE_API_KEY,
  baseURL: "https://api.tokgate.io",
});

const msg = await client.messages.create({
  model: "claude-sonnet-4.6",
  max_tokens: 1024,
  system: "You are a rigorous technical assistant",
  messages: [{ role: "user", content: "Explain what nucleus sampling is" }],
});
console.log(msg.content[0].text);

base_url without /v1Anthropic SDK: appends /v1/messages automatically, set https://api.tokgate.ioOpenAI SDK: set https://api.tokgate.io/v1

Response structure

{
  "id": "msg_xxxxxxxx",
  "type": "message",
  "role": "assistant",
  "model": "claude-sonnet-4-6",
  "content": [
    { "type": "text", "text": "Nucleus sampling (top-p) is..." }
  ],
  "stop_reason": "end_turn",
  "stop_sequence": null,
  "usage": {
    "input_tokens": 38,
    "output_tokens": 174,
    "cache_creation_input_tokens": 0,
    "cache_read_input_tokens": 0
  }
}
FieldDescription
content[]Array of content blocks. type is text (text), thinking (reasoning trace) or tool_use (requesting a tool call, with id / name / input).
stop_reasonend_turn (finished naturally) / max_tokens (hit the limit) / stop_sequence (hit a stop sequence) / tool_use (waiting for a tool result).
usage.input_tokens / output_tokensInput and output token counts, corresponding to prompt_tokens / completion_tokens in the OpenAI format.

Streaming events

Set stream: true and the server returns an SSE event stream with named event: types. Unlike OpenAI's single chunk shape, each event type needs separate handling.

event: message_start
data: {"type":"message_start","message":{"id":"msg_xxx","role":"assistant","content":[],"usage":{"input_tokens":38,"output_tokens":0}}}

event: content_block_start
data: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"nucleus"}}

event: content_block_stop
data: {"type":"content_block_stop","index":0}

event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":"end_turn"},"usage":{"output_tokens":174}}

event: message_stop
data: {"type":"message_stop"}
EventMeaning
message_startMessage start, carrying initial metadata and the input token count.
content_block_start / content_block_delta / content_block_stopStart / delta / stop of a content block. Text increments arrive in delta.text.
message_deltaMessage-level delta, carrying the final stop_reason and the output token count.
message_stopEnd of the stream.
pingHeartbeat; safe to ignore.

Field mapping vs the OpenAI format

PurposeOpenAI formatAnthropic format
System promptmessages[0] with role: "system"top-level system
Max outputmax_tokens (optional)max_tokens (required)
Stop sequencesstopstop_sequences
Answer textchoices[0].message.contentcontent[0].text
Stop reasonfinish_reason (stop)stop_reason (end_turn)
Input usageusage.prompt_tokensusage.input_tokens
Output usageusage.completion_tokensusage.output_tokens
Tool definition parametersfunction.parametersinput_schema
Feeding back tool resultsa role: "tool" messagea tool_result block inside a role: "user" message

Native Gemini format

POST https://api.tokgate.io/v1beta/models/{model}:generateContent

For Google Gemini models only. Put the model ID in the URL path and use Gemini-native contents / parts, not OpenAI messages.

Request headers

HeaderRequiredDescription
AuthorizationRequiredUse Bearer sk-***. Do not send x-goog-api-key — that header causes a 401.
Content-TypeRequiredapplication/json

Request parameters

ParameterTypeRequiredDescription
contentsarray<object>RequiredConversation content. Each item contains role and parts; text goes in parts[].text.
systemInstructionobjectOptionalSystem instructions using the parts content structure.
generationConfigobjectOptionalGeneration settings such as temperature, topP and maxOutputTokens.
toolsarray<object>OptionalGemini-native tool declarations such as functionDeclarations.

Request example

curl -X POST "https://api.tokgate.io/v1beta/models/YOUR_GEMINI_MODEL_ID:generateContent" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-***" \
  -d '{
    "systemInstruction": {"parts": [{"text": "You are a rigorous technical assistant"}]},
    "contents": [{"role": "user", "parts": [{"text": "Explain nucleus sampling"}]}],
    "generationConfig": {"temperature": 0.7, "maxOutputTokens": 1024}
  }'

Response structure

Answer text is in candidates[0].content.parts[].text, the stop reason in finishReason, and token usage in usageMetadata.

Streaming

POST https://api.tokgate.io/v1beta/models/{model}:streamGenerateContent?alt=sse

Use streamGenerateContent with the same request body. Add alt=sse to receive Gemini-native response chunks via Server-Sent Events.

Error handling

For status-code semantics and error body structures, see API Reference · Error codes. Frequent chat-API issues:

  • 400 with messages-related errors: the Anthropic format requires alternating user / assistant messages and does not accept a role: "system" message.
  • 400 for missing max_tokens: this field is required in the Anthropic format.
  • 404 model_not_found: the model ID is misspelled, or the model is outside the key's whitelist (the latter usually returns 403).
  • Streaming interrupted: check whether a middle layer (Nginx, CDN, corporate proxy) has response buffering enabled; buffering must be off for SSE to pass through in real time.