OpenAI Batch API

Last updated Jul 17, 2026

Run **large offline jobs** for embeddings or chat completions at lower cost — submit JSONL, poll status, download results within ~24h.

#When to use

Indexing thousands of document chunks for RAG

Nightly re-embedding after doc updates

Bulk eval or classification where latency is not interactive

Skip batch for user-facing chat — use synchronous endpoints or streaming instead.

POST https://api.openai.com/v1/batches
GET  https://api.openai.com/v1/batches/{batch_id}

Each JSONL line is one sub-request (embeddings or chat completions).

{
  "input_file_id": "file-abc123",
  "endpoint": "/v1/embeddings",
  "completion_window": "24h"
}

JSONL line example:

{"custom_id": "chunk-001", "method": "POST", "url": "/v1/embeddings", "body": {"model": "text-embedding-3-small", "input": "Refund policy text…"}}

from openai import OpenAI
client = OpenAI()

| Pitfall | Fix |

|---------|-----|

| Mixing models in one batch | One model per batch job |

| Losing chunk ↔ vector mapping | Stable `custom_id` per row |

| Polling too aggressively | Back off; batch is async by design |

| Secrets in JSONL uploads | Redact PII before upload |