Batch is a different workflow
A batch job submits many independent requests and processes results later. It suits offline labeling, document extraction, evaluation and scheduled content processing. It is usually a poor fit for an interactive chat response that a person expects immediately.
Check the model and category
OpenAI's Batch API documentation describes a 50% discount relative to standard synchronous API rates for eligible models and workloads. Other providers have their own batch rules. Check the current model, endpoint, turnaround window and pricing page before applying a blanket discount to all usage. Tool or non-text charges may follow different rules.
Example with a failure allowance
If a standard text job would cost $100 and all requests qualify for a 50% batch discount, the ideal batch charge is $50. If 5% require a full-price retry, add roughly $5, giving about $55 before other costs. The exact amount depends on which requests failed and their token mix; track this from usage records.
Operational checklist
Store a stable request ID for each item, validate outputs, and retry only failed items. Watch deadline and rate limits. If faster answers unlock revenue or prevent user abandonment, include that cost when deciding between synchronous and batch. Estimate standard request cost before applying the provider's documented discount.