When you use a large language model (LLM) through an API, the provider generally charges for the text the model processes and generates. The API measures that text in tokens. A token can represent a whole word, part of a word, or punctuation. Providers commonly quote separate prices for input tokens and output tokens, often per million tokens.
Your total bill depends on more than the rate per token. It also depends on the model and request length. Repeated conversation history adds to the total, as do features such as caching or batch processing. To estimate cost, count the tokens sent and received, then apply the rates for the model and API service you plan to use.
How token charges add up
Input tokens count all text the API sends to the model: the user’s message, system instructions, and any conversation history included in the request. Output tokens count the model’s response. A token is roughly three-quarters of an English word, although the exact count depends on the text and the provider’s tokenizer.
Use this calculation for a basic estimate:
Cost = (input tokens / 1,000,000 × input price per million) + (output tokens / 1,000,000 × output price per million)
For example, a request that sends a long document and asks for a short answer has many input tokens and relatively few output tokens. A request that asks the model to draft a long report has a larger output-token count. The provider’s pricing page lists the rates to use; OpenAI’s API pricing page, for example, separates rates by model and token type.
Output tokens generally cost more per token than input tokens, but the difference depends on the provider and model. Check both rates rather than estimating the entire request with a single blended price.
What changes the cost per request
Your model choice has a large effect on cost. Providers charge different rates across their model lineups, so two models that answer the same prompt can produce different bills. The lowest per-token rate does not always produce the lowest cost per completed task: if a model gives an unusable answer and the request must be retried, the extra calls add expense. Test models on the same representative tasks. Compare answer quality and retry rates as well as price.
Long prompts also cost more because the API processes more input. A large system prompt is billed again when it is sent with each request. In a multi-turn conversation, the client often resends earlier messages so the model has the context it needs. That repeated history adds input tokens over the course of the conversation. Remove irrelevant instructions and old messages when the model no longer needs them.
Some reasoning models also produce internal reasoning tokens that count as output for billing, increasing the charge beyond the visible answer alone. The exact billing treatment depends on the provider and model, so check the model’s pricing details rather than assuming every generated token appears in the response.
Ways to lower the bill
Start by matching the model to the task. A lower-cost model may handle simple classification or extraction, while a more capable model may be needed for difficult analysis. Test both against real examples and measure successful task completion, including retries. If you route easier requests to a cheaper model, base the decision on those results rather than on token price alone.
Caching and batch processing can reduce costs when the request pattern fits the provider’s features. Prompt caching gives a lower rate for eligible input that repeats across requests; batch processing handles eligible requests asynchronously, often at a discount. Providers set different eligibility rules, rates, and tradeoffs, so verify the details before building them into a cost estimate. Finout’s pricing overview describes these options and their potential savings.
Track actual token use alongside request counts. A short user message can still trigger a large bill if the request includes a long system prompt, conversation history, or a lengthy generated response. Also account for agent workflows, where a sequence of model calls can multiply usage, and for fallback requests that switch to another model when a preferred endpoint is unavailable or rate-limited.
Estimate and monitor spending
To forecast spending, measure typical input and output token counts for representative requests. Then apply the current rates for the exact model and API channel. A provider’s direct API and a cloud marketplace that offers the same model can have different rates, so check the service that will actually bill you.
Prices and model lineups change. Treat estimates as snapshots: confirm rates before launch and review them when you change models, prompts, or request patterns. The CloudZero pricing comparison discusses how model choice and usage details affect API costs, while provider pricing pages remain the place to verify current charges.