Chat Completions API Complete Reference and Guide

You've inherited a chatbot that works in development, but production is exposing the parts tutorials skip. Requests time out under load, conversation history grows until costs and truncation become unpredictable, search and file features don't behave like newer examples suggest, and someone has asked whether the application should move to the Responses API.

The Chat Completions API is still a practical interface for message-based generation, especially when your application manages its own state and needs a broadly adopted request format. It's also no longer the obvious answer for every new OpenAI integration. The right decision depends on how much your system values simple message control over native tools, managed chaining, and a newer response model.

This guide treats Chat Completions as a production API, not just a first request. You'll find the endpoint and message model, parameter decisions, response handling, stored completions, reliability patterns, rate-limit implications, implementation examples, and a decision framework for staying put or migrating safely.

Table of Contents

• Introduction to the Chat Completions API
• Core Endpoint and Message Structure Explained
  • Roles and conversation state
  • Storing a completion
• Request Parameters and Configuration Options
  • Core inputs and instruction control
  • Sampling and output controls
  • Delivery and operational flags
• Response Schema and Stored Completions Workflow
  • Reading choices safely
  • Stored completion operations
• Error Codes Rate Limits and Reliability Handling
  • Match the response to the failure
  • Rate limits are part of architecture
• Chat Completions Versus Responses Migration Guide
  • A safer migration path
• SDK Usage and Code Examples for Implementation
  • Python
  • Node.js
  • curl and streaming
  • Tool calls and history management
• Quick Reference Glossary and Cross Linked Resources
  • Endpoint and field lookup
  • Production readiness checklist

Introduction to the Chat Completions API

OpenAI introduced Chat Completions in March 2023, and on April 24, 2024, the company said the interface represented 97% of its API GPT usage. That rapid adoption made the message-based format the default developer path for OpenAI models, replacing older prompt-only patterns with a structure designed for multi-turn conversations. See OpenAI's GPT-4 API general availability announcement for that platform history.

The core idea is straightforward: your application sends an ordered list of messages, each with a role and content, and the model returns one or more generated choices. The API doesn't automatically understand your application's conversation unless you provide the relevant history, so your code controls what the model sees, what gets discarded, and how context is persisted.

That design works well for several common workloads:

• Customer support chat: Keep a stable instruction layer, add user messages, and return assistant replies.
• Classification and extraction: Use explicit system or developer instructions, then send the item to classify.
• Content generation: Provide a reusable instruction and task-specific user content.
• Tool-enabled workflows: Let the model request a function, execute it in your application, and send the result back as a message.
• Provider portability: Chat-style message formats are widely supported by SDKs, gateways, and model platforms.

It's less attractive when your application depends on built-in web search, native file workflows, server-managed turn chaining, or complex agent loops. OpenAI recommends Responses for new projects, while continuing to support Chat Completions, so the practical question isn't whether the older endpoint suddenly stopped working. It's whether its deliberately manual architecture still matches your application.

Production perspective: Chat Completions is easiest to operate when your team owns the conversation state, tool loop, validation, and retry policy. That control is an advantage until you're rebuilding capabilities the newer API already provides.

Use the sections below as a reference rather than a linear course. The endpoint section gives you the request shape, the parameter section helps tune behavior, the response section covers parsing and storage, and the migration section addresses the decision many teams now face.

Core Endpoint and Message Structure Explained

The API accepts an HTTP POST request at /v1/chat/completions. OpenAI's Chat Completions API reference describes it as a structured conversation endpoint that receives a message list and returns a model-generated reply. Authentication uses your OpenAI API key through the standard SDK or an Authorization header.

A minimal request has three operational decisions: the model, the messages, and whether you want a streamed response.

{
  "model": "gpt-4o-mini",
  "messages": [
    {
      "role": "user",
      "content": "Summarize the purpose of an API gateway."
    }
  ]
}

The message array is ordered. Earlier messages establish context, while the latest user message usually supplies the immediate task. Roles matter because the model follows an instruction hierarchy. OpenAI's chat completion prompting guidance emphasizes separating high-priority instructions from user content instead of putting every instruction into one undifferentiated prompt.

Roles and conversation state

A system message establishes broad behavior, such as tone, output constraints, or domain boundaries. A developer message is intended for application instructions and should remain separate from user-controlled content where the model and API support that role. A user message carries the request or data. An assistant message represents prior model output, and a tool message carries the result of a tool call initiated by the model.

A multi-turn request might look like this:

{
  "model": "gpt-4o-mini",
  "messages": [
    {
      "role": "system",
      "content": "Answer with concise implementation guidance."
    },
    {
      "role": "user",
      "content": "What does idempotency mean?"
    },
    {
      "role": "assistant",
      "content": "Idempotency means repeating the same operation produces the same intended result."
    },
    {
      "role": "user",
      "content": "Give me an example for payment retries."
    }
  ]
}

Chat Completions is effectively stateless from your application's perspective. Your service must decide which messages to retain, summarize, redact, or remove. Every retained message contributes to context consumption, so blindly appending history creates both cost pressure and truncation risk.

Storing a completion

The current reference supports store=true. Stored items can then be retrieved, listed, and updated through the operations documented by OpenAI. Storage is useful when your team needs post-hoc inspection, audit trails, evaluation data, or metadata-driven lifecycle management.

Don't treat storage as a replacement for your own application database. Keep business identifiers, authorization rules, retention policy, and user-visible records in systems you control. Use the API's stored-completion workflow when inspecting the model interaction itself is valuable.

Request Parameters and Configuration Options

Most Chat Completions bugs come from treating parameters as decoration. They change output shape, variability, latency, token consumption, and operational behavior. Keep stable instructions in role-separated messages, pass user data separately, and make configuration explicit in application code rather than relying on assumptions about defaults.

Parameter	Category	Purpose and Effect
model	Core input	Selects the model that processes the request.
messages	Core input	Supplies ordered role-content pairs that form the conversation context.
temperature	Sampling	Adjusts output variability. Lower values favor more consistent responses, while higher values permit more variation.
top_p	Sampling	Limits token selection to a probability mass. Tune it carefully rather than changing it alongside temperature without a reason.
max_tokens	Output control	Places a ceiling on generated output where supported. It helps prevent unexpectedly long replies.
stream	Delivery	Returns incremental chunks instead of waiting for the complete response.
tools	Orchestration	Describes functions or other supported tool interactions that the model may request.
tool_choice	Orchestration	Controls whether tool use is automatic, required, or constrained to a selected tool where supported.
response_format	Output control	Requests a particular output structure where supported by the selected model and endpoint.
store	Operations	Requests that the completion be stored for later retrieval and lifecycle operations.
metadata	Operations	Attaches application metadata where supported, useful for inspection and filtering workflows.

Core inputs and instruction control

model and messages are mandatory decisions. The model determines capabilities and compatibility, while the message list determines what the model can use as context. Keep system or developer instructions stable. Don't splice raw user text into those high-priority instructions unless you've intentionally validated and escaped the content.

messages also carries tool results. A typical function loop is:

1. Send the conversation and available tool definitions.
2. Detect a tool call in the assistant response.
3. Validate the requested function and arguments.
4. Execute the function in your application.
5. Append the tool result.
6. Send the updated message list for the final answer.

Your server, not the model, should authorize actions and validate arguments.

Sampling and output controls

Temperature and top_p both affect selection behavior, but changing both at once makes debugging harder. For classification, extraction, and policy-sensitive responses, use conservative sampling and validate the output. For creative drafting, allow more variation, then impose a separate length and format boundary.

max_tokens protects downstream systems from unbounded output, but it doesn't guarantee a complete answer. Inspect the finish reason and treat truncation as a recoverable application state, not as a successful final response.

Delivery and operational flags

Streaming improves perceived responsiveness because the client can render partial output while generation continues. It complicates error handling, moderation, cancellation, and persistence because the final answer arrives across chunks rather than as one object.

store should be enabled deliberately. Store requests that support debugging, evaluation, or audit requirements, and avoid retaining sensitive content without a clear data policy. metadata can connect a stored completion to a tenant, workflow, or evaluation run, but it shouldn't contain secrets or unrestricted personal data.

Implementation rule: Trim or summarize old history before the request reaches the endpoint. The model can't distinguish useful context from obsolete context unless your application does that work.

Response Schema and Stored Completions Workflow

A non-streaming Chat Completions response normally gives your application an object containing request metadata, one or more choices, and usage information. The practical parsing path is usually choices[0].message.content, but production code should inspect the entire choice before treating the text as complete.

A representative shape looks like this:

{
  "id": "chatcmpl_example",
  "object": "chat.completion",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "An API gateway centralizes access, policy, and routing for backend services."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 0,
    "completion_tokens": 0,
    "total_tokens": 0
  }
}

The values above are placeholders, not measurements. Your code should read the returned usage object rather than estimating consumption from the visible text. That distinction matters because the prompt includes message history and structured tool data, not just the latest user sentence.

Reading choices safely

choices is an array because the API can return multiple candidates where the selected model and request configuration support that behavior. Don't assume every response contains usable text. A choice may finish because it reached a limit, requested a tool, or encountered another stopping condition.

Use the finish_reason to decide what happens next:

• Normal completion: Persist or display the assistant content after validation.
• Length limit: Mark the result incomplete and either ask the model to continue or revise the request.
• Tool request: Route the call through an allowlisted server-side function.
• Missing content: Treat the response as a protocol failure and log the response metadata for diagnosis.

If you stream, the response is delivered as chunks. Accumulate content deltas, capture tool-call fragments carefully, and wait for the terminal event before marking the generation complete. A client that writes each chunk directly to a database can leave partial records when the connection closes.

Stored completion operations

With store=true, OpenAI's current reference supports later retrieval, listing, and update operations for stored completions. That creates a useful inspection path for teams running prompt evaluations or investigating why a production response was accepted.

A stored-completion workflow should include:

1. Application identity: Attach a tenant, request, or workflow identifier through your own data model or supported metadata.
2. Retention policy: Decide how long model interactions remain available.
3. Access control: Restrict inspection to authorized operators and services.
4. Evaluation status: Record whether a completion passed validation, required repair, or triggered a fallback.
5. Usage review: Save returned usage fields for cost and capacity analysis.

Stored completions complement, rather than replace, application logging. Keep secrets out of prompts and redact sensitive data before persistence. If your compliance requirements require a complete record, store the request and response in an approved system with the same controls as any other customer data.

Error Codes Rate Limits and Reliability Handling

A reliable client separates permanent configuration failures from transient service failures. Retrying an invalid API key won't repair the request, while failing to retry a temporary overload can turn a recoverable incident into a user-visible outage.

Match the response to the failure

• 401 Unauthorized: Check the API key, environment selection, and server-side credential injection. Don't retry indefinitely.
• 429 Too Many Requests: Apply exponential backoff with jitter, reduce concurrency, and inspect both request and token ceilings.
• 500 Internal Server Error: Treat it as potentially transient. Retry within a bounded budget and record the request identifier.
• 503 Service Unavailable: Delay and retry later. If the user is waiting synchronously, return a controlled fallback instead of holding the connection open without limit.
• 400-level validation errors: Log the structured error, fix the request construction, and avoid automatic retries unless the request changes.
• Timeouts and disconnects: Use idempotency-aware application logic so a retry doesn't duplicate an external side effect.

The correct retry policy depends on whether the request only generates text or also triggers an action. A failed generation can usually be retried. A tool call that creates an order or sends an email needs a durable operation identifier and server-side deduplication.

Rate limits are part of architecture

OpenAI's published table ranges from 500 requests per minute and 30,000 tokens per minute at Tier 1 to 15,000 requests per minute and 40,000,000 tokens per minute at Tier 5. Those figures come from the published OpenAI rate-limit guidance, and they show why a client can fail even when its request count looks modest. Large prompts and retained histories consume token capacity quickly.

Use a queue or concurrency limiter rather than allowing every incoming web request to call the model immediately. Track request rate, token rate, latency, retries, and rejection counts separately. A service can remain under its request ceiling while exceeding its token-per-minute limit.

For practical planning, distinguish message limits from provider quota and endpoint rate limits. Writingmate's message limits documentation is a useful separate reference when a product layer sits between your application and an underlying model provider.

Reliability pattern: Backoff belongs at the boundary of the model client. Individual features shouldn't each invent their own retry loop, because competing retries can multiply load during an incident.

A fallback model or provider can preserve continuity, but only if the fallback supports the required schema, tool behavior, and output validation. Don't switch blindly when the request includes structured output or a function call.

Chat Completions Versus Responses Migration Guide

A production chat application can run reliably on Chat Completions for years, then hit a requirement that changes the calculation. Built-in tools, response chaining, or newer reasoning workflows may justify Responses, but an endpoint swap alone does not deliver those benefits. The migration can affect parsers, state handling, tool loops, tests, dashboards, and incident runbooks.

OpenAI recommends Responses for new projects while continuing to support Chat Completions. Its migration guidance calls out concrete differences, including the removal of the n parameter for multiple generations, a changed response shape, and capabilities that Chat Completions does not provide natively.

Start by examining the workload, not the endpoint name. Stay on Chat Completions if the service mainly generates text, application-managed history is already tested, and your existing tool loop meets product requirements. Its familiar message list and broad compatibility can be more valuable than access to a newer surface. Replacing a stable pipeline only to rebuild the same orchestration layer creates migration risk without much operational gain.

Responses becomes the stronger choice when built-in tools or response chaining are part of the product design. The older interface was not built around the newer reasoning and coding workflows described in OpenAI's migration material. OpenAI also identifies a Codex deprecation path with full removal targeted for early 2026. Teams dependent on that workflow should plan the change as active engineering work.

The hidden cost is usually the adapter, not the HTTP request. Code that reads choices[0].message, depends on parallel generations, or assumes Chat Completions-specific message roles needs explicit review. Tool calls also require comparison of orchestration, validation, retries, and stored state. A workflow that depends on n must be redesigned rather than translated mechanically.

A safer migration path

Create an internal interface first. A domain object such as GenerationResult can normalize text, tool requests, usage, finish state, and provider identifiers. Keep business logic against that object, then add a Responses implementation beside the existing Chat Completions adapter.

Run both implementations against recorded, redacted requests. Compare output validation, tool behavior, latency, error handling, and cost telemetry before changing traffic. Release with a controlled percentage or an operator-selected path, and retain the ability to return to the established client while mismatches are investigated.

For interoperability planning, this OpenAI-compatible API overview frames the trade-off between a portable chat interface and provider-specific features. Staged migration is often the practical choice, especially when compatibility, existing observability, or message-based tests still carry operational value.

SDK Usage and Code Examples for Implementation

Chat Completions became popular because the request model is easy to express in mainstream SDKs. OpenAI's announcement that the interface represented 97% of API GPT usage by April 24, 2024 reflects that developer adoption, as documented in the original platform announcement. The implementation details still matter more than the popularity.

Python

from openai import OpenAI

client = OpenAI()

messages = [
    {
        "role": "system",
        "content": "Answer with concise, technically accurate guidance."
    },
    {
        "role": "user",
        "content": "Explain request idempotency."
    }
]

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=messages,
    temperature=0.2,
    max_tokens=300
)

choice = response.choices[0]
if choice.finish_reason == "stop":
    print(choice.message.content)
else:
    raise RuntimeError(f"Incomplete response: {choice.finish_reason}")

Keep the model client in one service module. That's where you should centralize timeouts, retry classification, structured logging, and model fallback. Feature code should pass a task and receive a validated result, not manage provider-specific transport rules.

Node.js

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY
});

const response = await client.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [
    {
      role: "system",
      content: "Return concise technical explanations."
    },
    {
      role: "user",
      content: "Explain request idempotency."
    }
  ],
  temperature: 0.2,
  max_tokens: 300
});

const choice = response.choices?.[0];

if (!choice || choice.finish_reason !== "stop") {
  throw new Error("The model did not return a complete answer");
}

console.log(choice.message.content);

Optional chaining prevents a parser crash, but it doesn't make the result valid. Validate the content and finish state before handing the response to a customer, database, or downstream action.

curl and streaming

curl  \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [
      {
        "role": "user",
        "content": "List three checks for an API health endpoint."
      }
    ],
    "stream": true
  }'

Streaming clients need an accumulator. Append text deltas in memory or to a controlled response channel, detect the terminal event, and persist only after the stream finishes. If the connection drops, mark the generation incomplete instead of storing a partial answer.

Tool calls and history management

Tool definitions belong in the request, but execution belongs in your server. Validate function names, argument types, authorization, and side effects. Send the tool result back as the appropriate message, then request the final assistant response.

For long conversations, keep a rolling summary plus recent turns rather than preserving every message forever. The customer-support chatbot implementation guide provides a practical application context for combining history, instruction design, and response handling.

Quick Reference Glossary and Cross Linked Resources

Use this glossary when reviewing a design or debugging a request:

• Chat Completions: OpenAI's message-based endpoint for generating replies from ordered conversation messages.
• Message role: A label such as system, developer, user, assistant, or tool that helps establish instruction priority and conversation meaning.
• Context: The instructions, history, and data included in the current request.
• Completion: The model-generated response returned for a request.
• Choice: One candidate response in the returned choices collection.
• Finish reason: The server's indication of why generation stopped, such as normal completion, a limit, or a tool request.
• Usage: Returned token accounting for the prompt, completion, and total request.
• Streaming: Incremental delivery of generated output rather than one complete response.
• Tool call: A model request for your application to execute a declared function.
• Stored completion: A completion saved through the API's storage option for later retrieval, listing, or update operations.
• Responses API: OpenAI's newer API surface for projects that need built-in tools, different response semantics, or response chaining.
• previous_response_id: A Responses capability for chaining turns without reconstructing the same Chat Completions message history.
• Rate limit: A provider-enforced ceiling on request volume or token throughput.

Endpoint and field lookup

Need	Chat Completions reference
Create a response	POST /v1/chat/completions
Supply conversation context	messages
Select a model	model
Stream output	stream
Permit tool interaction	tools
Request storage	store=true
Inspect candidates	choices
Read generated text	choices[0].message.content
Check completion state	finish_reason
Track consumption	usage

Production readiness checklist

Before launch, verify each item:

• Prompt boundaries: System or developer instructions are separate from user-controlled content.
• History policy: Your service summarizes, trims, or expires old context.
• Output validation: The application checks finish state and validates structured output before use.
• Tool security: Functions are allowlisted, arguments are validated, and side effects are idempotent.
• Retry discipline: The client retries transient failures with bounded exponential backoff and jitter.
• Quota monitoring: Request and token throughput are tracked independently against your assigned tier.
• Streaming recovery: Partial streams are marked incomplete and aren't presented as finished responses.
• Storage governance: Stored completions have retention, access, and redaction rules.
• Migration boundary: Provider-specific response parsing is isolated behind an internal adapter.
• Feature fit: Search, files, and agent requirements are checked against the endpoint's current support rather than copied from an old tutorial.

Search deserves its own caution. OpenAI's current tool documentation says Chat Completions supports specialized search models for web search, while those legacy paths lack Responses capabilities such as domain filters, complete source lists, live-access control, and token-budget control. The documentation also lists shutdown dates for gpt-4o-search-preview and gpt-4o-mini-search-preview on 2026-07-23, so search-dependent systems should review that surface directly instead of assuming compatibility.

File handling has also become a practical fault line. Community reports from September 2025 describe Chat Completions file-input support as unavailable, which can force client-side extraction or a different API path. Treat that as an implementation constraint to verify in your own account and model configuration, not as a reason to assume every file workflow is portable.

The historical pricing path also explains why many early examples look different from current designs. The first Chat Completions release bundled GPT-3.5 Turbo at $0.002 per 1,000 tokens, while GPT-4 initially used $0.03 per 1,000 prompt tokens and $0.06 per 1,000 completion tokens, with limits of 40,000 tokens per minute and 200 requests per minute. OpenAI later announced GPT-4 Turbo with a 128K-token context window, priced at $0.01 per 1,000 input tokens and $0.03 per 1,000 output tokens, at DevDay in November 2023. These historical details are documented in the GPT-3.5 Turbo reference, and they're useful context, but current projects should verify live model pricing and limits before budgeting.

The simplest architecture is still often the most durable: own your state, normalize responses, validate outputs, and keep provider-specific code behind one boundary. Migrate when native capabilities remove more work than the migration creates.

---

Writingmate provides an OpenAI-compatible API for applications that need a Chat Completions-style integration, alongside a workspace for multi-model chat, web research, file analysis, media generation, and agents. If you're evaluating a portable model layer or need fallback options around provider availability, visit Writingmate and compare its developer workflow with your existing Chat Completions client.