How do I find out which API calls are costing the most?

Your provider's usage dashboard is the starting point. OpenAI's dashboard shows token usage by model and time period. You can also add logging to your application that records input and output token counts per endpoint, then aggregate to find your most expensive call types. This is more granular than the provider dashboard and helps you pinpoint which features are driving costs.

Does setting max_tokens actually save money?

Setting max_tokens caps the output length, but you are only charged for tokens actually generated — not the max_tokens limit itself. However, setting a realistic max_tokens value prevents runaway responses from edge cases where the model produces unexpectedly long outputs, which can spike costs on individual calls. It is a good safety measure even if it does not reduce your average-case cost.

How much can prompt caching actually save?

For applications with a large, static system prompt sent on every request, prompt caching can reduce effective input costs by 60 to 90 percent on the cached portion. If your system prompt is 2,000 tokens and you make 500,000 monthly requests, that is 1 billion cached tokens per month. At Anthropic's cache read rate versus standard input rate, the savings can be thousands of dollars monthly.

Should I use GPT-4o-mini for everything to save money?

Not unconditionally. GPT-4o-mini is excellent for simple, well-defined tasks with clear right or wrong answers. For complex reasoning, nuanced judgment, or tasks where output quality directly affects user satisfaction, the cost of poor outputs — user churn, support tickets, manual review — often exceeds the savings from a cheaper model. Test quality on your specific tasks before committing to a cost-reduction switch.

My API bill doubled this month but traffic only grew 20%. What happened?

Common causes include a new feature launch that uses a more expensive model, a change to a system prompt that increased token count, a bug that is triggering retry loops, or a new conversation flow that maintains longer history. Start by comparing your average tokens per request this month versus last month in your logging system. A significant increase in average input tokens usually points to prompt growth or history accumulation.

Why Your LLM Bill Is Higher Than Expected — And How to Fix It

Your System Prompt Is Costing More Than You Think

The system prompt is the silent cost driver that most developers underestimate. Every single API call you make includes the full system prompt in the input token count, even if it never changes. A detailed system prompt with persona instructions, guardrails, output formatting rules, and example responses can easily reach 1,500 to 3,000 tokens. At GPT-4o's input rate of $2.50 per million tokens, a 2,000-token system prompt adds $0.005 to every request. That seems trivial, but at 200,000 monthly requests it contributes $1,000 to your bill — from a prompt that has not changed once. The fix is two-pronged: first, audit your system prompt and cut anything redundant; second, enable prompt caching so repeated prefixes are billed at the lower cached rate rather than full price on every call.

Conversation History Accumulates Silently

Chat applications that maintain conversation context are particularly vulnerable to token accumulation. Every turn you append to the history array increases the input token count for every subsequent request in that session. A user who sends 20 messages in a conversation, each averaging 80 tokens with 150-token replies, has accumulated 4,600 tokens of history by the final message. That entire history gets sent as input on message 21. If your application never truncates or summarizes conversation history, long sessions become exponentially expensive. The practical fix is to implement a sliding window that keeps only the last N turns, or to periodically summarize older portions of the conversation into a compact paragraph that replaces the raw message history. Both techniques can cut conversation-related input costs by 50 to 80 percent.

Debug Logging and Development Traffic

Many teams forget to disable verbose logging and testing traffic before they ship. During development it is common to make exploratory API calls with generous max_tokens settings, test edge cases repeatedly, and log full request and response payloads for debugging. If your staging environment shares an API key with production, or if you forget to switch your development setup to a mock provider, real billing costs accumulate from traffic that produces no user value. Even after launch, some teams leave request logging enabled at a level that triggers redundant API calls for analytics or auditing. A regular audit of your API usage dashboard, filtered by endpoint and time of day, can surface unexpected traffic sources that have no business justification.

Not Using Caching for Repeated Inputs

If your application sends the same or similar inputs repeatedly — product catalogues, knowledge base articles, document context — and you are not using prompt caching, you are paying full price for tokens the model has already processed. Both OpenAI and Anthropic offer prompt caching mechanisms that charge significantly less for cache hits on repeated prompt prefixes. Anthropic's prompt caching for Claude charges $0.30 per million cache write tokens and $3.75 per million cache read tokens, compared to $3.00 per million for standard input on Claude Sonnet 4. OpenAI automatically applies caching at 50 percent of standard input rates for eligible prompts. Implementing caching for static or slow-changing content in your prompts is often the single highest-leverage optimization available. Read more in the prompt caching guide.

Using the Wrong Model Tier for Simple Tasks

One of the most expensive habits in AI development is using a flagship model for tasks that do not require it. If you are running a GPT-4o or Claude Sonnet 4 pipeline to do simple intent classification, sentiment analysis, or short-text summarization, you are spending roughly 10 to 20 times more than necessary. GPT-4o-mini at $0.15 per million input tokens and Claude Haiku at $0.80 per million input tokens handle these tasks accurately at a fraction of the cost. The typical production architecture that optimizes for cost has a lightweight model handling the majority of routine requests and routing complex cases to a more capable model only when needed. Auditing your request logs to identify which call types actually require the premium model is a straightforward way to find where you can safely downgrade.

Output Tokens You Are Generating but Not Using

Output tokens are always more expensive than input tokens, and generating output that your application discards or ignores is pure waste. This happens more often than developers realize. Some prompts are written in a way that encourages the model to reason out loud before giving the answer, producing verbose chain-of-thought text that never surfaces in the UI. Some applications request JSON with fields that are parsed but not displayed. Some pipelines request long explanations when a one-sentence answer would suffice. Reviewing your prompt structure and using explicit output constraints — setting a lower max_tokens value, specifying response length in the prompt, or requesting structured formats with only the fields you need — can meaningfully reduce your output token spend without degrading user-facing quality.