Chat
Chat supports three vendor protocols. The model determines the format: Anthropic models use Anthropic Messages, Google models use the native Gemini format, and other LLMs use the OpenAI-compatible format.
Choose by model vendor
| Format | Endpoint | Applies to |
|---|---|---|
| OpenAI Chat Completions | POST /v1/chat/completions | LLMs from vendors other than Anthropic and Google |
| Anthropic Messages | POST /v1/messages | Claude models from Anthropic |
| Gemini generateContent | POST /v1beta/models/{model}:generateContent | Gemini models from Google |
All three formats share one API keyThe API key and billing account are shared, but the path, auth header and request body are determined by the model vendor. The formats are not interchangeable. Get model IDs from GET /v1/models or the Model Catalog.
OpenAI format
For LLMs from vendors other than Anthropic and Google. Supports streaming and non-streaming responses and follows OpenAI /v1/chat/completions.
Request parameters
| Parameter | Type | Required | Description |
|---|---|---|---|
model | string | Required | The model ID. |
messages | array<object> | Required | The conversation messages, each with a role (system / user / assistant / tool) and content. For vision models, content can be an array mixing text and image_url. |
temperature | number | Optional | Sampling temperature, 0 ~ 2, default 1. |
top_p | number | Optional | Nucleus sampling, 0 ~ 1, default 1. Adjust either this or temperature, not both. |
n | integer | Optional | Number of candidates to generate, default 1. |
stream | boolean | Optional | Whether to stream the response, default false. |
stream_options | object | Optional | Streaming options, e.g. {"include_usage": true} attaches usage stats to the final chunk. |
stop | string / array | Optional | Stop sequences — a string or an array of strings. |
max_tokens | integer | Optional | Maximum number of tokens to generate. |
max_completion_tokens | integer | Optional | Maximum completion tokens. In the current gateway it behaves differently from max_tokens; reasoning tokens may also consume this limit and reduce visible output, so test each target model separately. |
presence_penalty | number | Optional | Presence penalty, -2 ~ 2, default 0. |
frequency_penalty | number | Optional | Frequency penalty, -2 ~ 2, default 0. |
logit_bias | object | Optional | Sampling bias for the specified tokens. |
tools | array<object> | Optional | List of tool (function) definitions the model may call. |
tool_choice | string / object | Optional | Tool selection mode: none / auto / required, or an object naming a specific function. |
response_format | object | Optional | Structured output, e.g. {"type": "json_object"} or {"type": "json_schema", ...}. |
seed | integer | Optional | Random seed for better reproducibility (exact identity not guaranteed). |
reasoning_effort | string | Optional | Reasoning effort: low / medium / high; only effective on reasoning-capable models. |
user | string | Optional | End-user identifier, useful for abuse investigation. |
Parameters depend on the modelDifferent models support different parameter sets. When you pass a parameter the model does not support, the gateway either ignores it or returns 400. Model capabilities (vision, tool calling, JSON mode, streaming) are shown as capability tags in the Model Catalog.
Request example
curl -X POST https://api.tokgate.io/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer sk-***" \
-d '{
"model": "deepseek-v4-pro",
"messages": [
{"role": "system", "content": "You are a rigorous technical assistant"},
{"role": "user", "content": "Explain what nucleus sampling is"}
],
"temperature": 0.7,
"max_tokens": 1024
}'
from openai import OpenAI
client = OpenAI(
api_key="sk-***",
base_url="https://api.tokgate.io/v1",
)
resp = client.chat.completions.create(
model="deepseek-v4-pro",
messages=[
{"role": "system", "content": "You are a rigorous technical assistant"},
{"role": "user", "content": "Explain what nucleus sampling is"},
],
temperature=0.7,
max_tokens=1024,
)
print(resp.choices[0].message.content)
print(resp.usage)
import OpenAI from "openai";
const client = new OpenAI({
apiKey: process.env.TOKGATE_API_KEY,
baseURL: "https://api.tokgate.io/v1",
});
const resp = await client.chat.completions.create({
model: "deepseek-v4-pro",
messages: [
{ role: "system", content: "You are a rigorous technical assistant" },
{ role: "user", content: "Explain what nucleus sampling is" },
],
temperature: 0.7,
max_tokens: 1024,
});
console.log(resp.choices[0].message.content);
Response structure
{
"id": "chatcmpl-xxxxxxxx",
"object": "chat.completion",
"created": 1779348818,
"model": "deepseek-v4-pro",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "Nucleus sampling (top-p) is...",
"reasoning_content": null,
"tool_calls": null
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 42,
"completion_tokens": 186,
"total_tokens": 228,
"prompt_tokens_details": { "cached_tokens": 0 },
"completion_tokens_details": { "reasoning_tokens": 0 }
},
"system_fingerprint": null
}
| Field | Description |
|---|---|
choices[].finish_reason | Why generation stopped: stop (finished naturally) / length (hit the length limit) / tool_calls (requesting a tool call) / content_filter (content blocked). |
choices[].message.tool_calls | Tools the model requests to call, with id, function.name and function.arguments (a JSON string). |
choices[].message.reasoning_content | The reasoning trace returned by reasoning models; null for regular models. |
usage | Token usage of this call, for reconciliation. Cache hits are reported in prompt_tokens_details.cached_tokens. |
Streaming
Set stream: true and the server returns chunks via Server-Sent Events, ending with data: [DONE]. Ideal for typewriter-style chat UIs.
stream = client.chat.completions.create(
model="deepseek-v4-pro",
messages=[{"role": "user", "content": "Tell a short story about overseas expansion"}],
stream=True,
stream_options={"include_usage": True},
)
for chunk in stream:
if chunk.choices:
print(chunk.choices[0].delta.content or "", end="", flush=True)
const stream = await client.chat.completions.create({
model: "deepseek-v4-pro",
messages: [{ role: "user", content: "Tell a short story about overseas expansion" }],
stream: true,
});
for await (const chunk of stream) {
process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}
data: {"id":"chatcmpl-xxx","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null}]}
data: {"id":"chatcmpl-xxx","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"Once"},"finish_reason":null}]}
data: {"id":"chatcmpl-xxx","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
data: [DONE]
Tool calling
Declare functions in tools and the model returns tool_calls when needed. After you execute the function, append the result to messages as a role: "tool" message and request again; the model then produces the final answer.
{
"model": "deepseek-v4-pro",
"messages": [{"role": "user", "content": "What's the weather in Shenzhen right now?"}],
"tools": [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the real-time weather of a given city",
"parameters": {
"type": "object",
"properties": {
"city": {"type": "string", "description": "City name, e.g. Shenzhen"}
},
"required": ["city"]
}
}
}
],
"tool_choice": "auto"
}
Anthropic format
For Claude models from Anthropic only. Key differences between Anthropic Messages and OpenAI: system is a top-level field; max_tokens is required; response content is a block array; usage fields are input_tokens / output_tokens.
Request headers
| Header | Required | Description |
|---|---|---|
x-api-key | Required | Your API key. You can also use Authorization: Bearer <key> instead. |
anthropic-version | Required | Protocol version, fixed at 2023-06-01. The official SDK sets it automatically. |
Content-Type | Required | application/json |
Request parameters
| Parameter | Type | Required | Description |
|---|---|---|---|
model | string | Required | The model ID. |
messages | array<object> | Required | Conversation messages; role can only be user or assistant and must alternate. content can be a string or an array of content blocks (text / image / tool_use / tool_result). |
max_tokens | integer | Required | Maximum number of tokens to generate. Required in the Anthropic format. |
system | string / array | Optional | System prompt, a top-level field. Do not put it into messages. |
temperature | number | Optional | Sampling temperature, 0 ~ 1. |
top_p | number | Optional | Nucleus sampling, 0 ~ 1. |
top_k | integer | Optional | Sample only from the K most likely tokens. |
stream | boolean | Optional | Whether to stream the response, default false. |
stop_sequences | array<string> | Optional | Custom stop sequences. When hit, stop_reason is stop_sequence. |
tools | array<object> | Optional | Tool definitions with name / description / input_schema. |
tool_choice | object | Optional | E.g. {"type": "auto"}, {"type": "any"}, {"type": "tool", "name": "..."}. |
thinking | object | Optional | Extended thinking, e.g. {"type": "enabled", "budget_tokens": 4096}; only effective on models that support it. |
metadata | object | Optional | Extra information, e.g. {"user_id": "..."}. |
Request example
curl -X POST https://api.tokgate.io/v1/messages \
-H "Content-Type: application/json" \
-H "x-api-key: sk-***" \
-H "anthropic-version: 2023-06-01" \
-d '{
"model": "claude-sonnet-4.6",
"max_tokens": 1024,
"system": "You are a rigorous technical assistant",
"messages": [
{"role": "user", "content": "Explain what nucleus sampling is"}
]
}'
from anthropic import Anthropic
client = Anthropic(
api_key="sk-***",
base_url="https://api.tokgate.io",
)
msg = client.messages.create(
model="claude-sonnet-4.6",
max_tokens=1024,
system="You are a rigorous technical assistant",
messages=[{"role": "user", "content": "Explain what nucleus sampling is"}],
)
print(msg.content[0].text)
print(msg.usage)
import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic({
apiKey: process.env.TOKGATE_API_KEY,
baseURL: "https://api.tokgate.io",
});
const msg = await client.messages.create({
model: "claude-sonnet-4.6",
max_tokens: 1024,
system: "You are a rigorous technical assistant",
messages: [{ role: "user", content: "Explain what nucleus sampling is" }],
});
console.log(msg.content[0].text);
base_url without /v1Anthropic SDK: appends /v1/messages automatically, set https://api.tokgate.ioOpenAI SDK: set https://api.tokgate.io/v1
Response structure
{
"id": "msg_xxxxxxxx",
"type": "message",
"role": "assistant",
"model": "claude-sonnet-4-6",
"content": [
{ "type": "text", "text": "Nucleus sampling (top-p) is..." }
],
"stop_reason": "end_turn",
"stop_sequence": null,
"usage": {
"input_tokens": 38,
"output_tokens": 174,
"cache_creation_input_tokens": 0,
"cache_read_input_tokens": 0
}
}
| Field | Description |
|---|---|
content[] | Array of content blocks. type is text (text), thinking (reasoning trace) or tool_use (requesting a tool call, with id / name / input). |
stop_reason | end_turn (finished naturally) / max_tokens (hit the limit) / stop_sequence (hit a stop sequence) / tool_use (waiting for a tool result). |
usage.input_tokens / output_tokens | Input and output token counts, corresponding to prompt_tokens / completion_tokens in the OpenAI format. |
Streaming events
Set stream: true and the server returns an SSE event stream with named event: types. Unlike OpenAI's single chunk shape, each event type needs separate handling.
event: message_start
data: {"type":"message_start","message":{"id":"msg_xxx","role":"assistant","content":[],"usage":{"input_tokens":38,"output_tokens":0}}}
event: content_block_start
data: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"nucleus"}}
event: content_block_stop
data: {"type":"content_block_stop","index":0}
event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":"end_turn"},"usage":{"output_tokens":174}}
event: message_stop
data: {"type":"message_stop"}
| Event | Meaning |
|---|---|
message_start | Message start, carrying initial metadata and the input token count. |
content_block_start / content_block_delta / content_block_stop | Start / delta / stop of a content block. Text increments arrive in delta.text. |
message_delta | Message-level delta, carrying the final stop_reason and the output token count. |
message_stop | End of the stream. |
ping | Heartbeat; safe to ignore. |
Field mapping vs the OpenAI format
| Purpose | OpenAI format | Anthropic format |
|---|---|---|
| System prompt | messages[0] with role: "system" | top-level system |
| Max output | max_tokens (optional) | max_tokens (required) |
| Stop sequences | stop | stop_sequences |
| Answer text | choices[0].message.content | content[0].text |
| Stop reason | finish_reason (stop) | stop_reason (end_turn) |
| Input usage | usage.prompt_tokens | usage.input_tokens |
| Output usage | usage.completion_tokens | usage.output_tokens |
| Tool definition parameters | function.parameters | input_schema |
| Feeding back tool results | a role: "tool" message | a tool_result block inside a role: "user" message |
Native Gemini format
For Google Gemini models only. Put the model ID in the URL path and use Gemini-native contents / parts, not OpenAI messages.
Request headers
| Header | Required | Description |
|---|---|---|
Authorization | Required | Use Bearer sk-***. Do not send x-goog-api-key — that header causes a 401. |
Content-Type | Required | application/json |
Request parameters
| Parameter | Type | Required | Description |
|---|---|---|---|
contents | array<object> | Required | Conversation content. Each item contains role and parts; text goes in parts[].text. |
systemInstruction | object | Optional | System instructions using the parts content structure. |
generationConfig | object | Optional | Generation settings such as temperature, topP and maxOutputTokens. |
tools | array<object> | Optional | Gemini-native tool declarations such as functionDeclarations. |
Request example
curl -X POST "https://api.tokgate.io/v1beta/models/YOUR_GEMINI_MODEL_ID:generateContent" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer sk-***" \
-d '{
"systemInstruction": {"parts": [{"text": "You are a rigorous technical assistant"}]},
"contents": [{"role": "user", "parts": [{"text": "Explain nucleus sampling"}]}],
"generationConfig": {"temperature": 0.7, "maxOutputTokens": 1024}
}'
Response structure
Answer text is in candidates[0].content.parts[].text, the stop reason in finishReason, and token usage in usageMetadata.
Streaming
Use streamGenerateContent with the same request body. Add alt=sse to receive Gemini-native response chunks via Server-Sent Events.
Error handling
For status-code semantics and error body structures, see API Reference · Error codes. Frequent chat-API issues:
400withmessages-related errors: the Anthropic format requires alternatinguser/assistantmessages and does not accept arole: "system"message.400for missingmax_tokens: this field is required in the Anthropic format.404 model_not_found: the model ID is misspelled, or the model is outside the key's whitelist (the latter usually returns403).- Streaming interrupted: check whether a middle layer (Nginx, CDN, corporate proxy) has response buffering enabled; buffering must be off for SSE to pass through in real time.