A founder predicts a support assistant will cost $100 a month by multiplying 100,000 conversations by the input-token price. Then the bill arrives: generated tokens have their own rate, some requests use tools, and caching does not apply to every token. The multiplication was fine; the unit being multiplied was incomplete.
This guide gives you a reproducible estimate based on four measurements per request: uncached input, cached input, output, and additional charges. The worked example is hypothetical. Replace its values with your API logs and the provider's current rates before making a product decision.
Measure one representative workflow first
Choose a specific feature, such as a bot answering order questions. Record 50–100 real requests or internal test requests using the configuration you intend to ship. Do not estimate tokens by counting characters; use the API's usage fields. Include short and long conversations, and record percentiles as well as averages. A small number of lengthy chats can dominate the bill.
SoyJB's AI API comparator helps shortlist models and simulate input and output at list prices. The tool itself says its simple simulation excludes caching and batch discounts. A production budget also needs the charges your workflow actually incurs: cache reads and writes, tool calls, search, storage, or execution, depending on the provider.
Keep the cost categories separate
For one request: (uncached_input × input_rate + cached_input × cache_rate + output × output_rate) / 1_000_000 + extras. Multiply that by monthly calls. Rates are US dollars per million tokens. extras is the measured or separately estimated additional cost per call.
Do not double-count cache hits. OpenAI's usage documentation says cached tokens are included in the input-token total. For this formula, subtract cached input from total input to get uncached input. Check the equivalent breakdown for your provider. Cache writes may also cost money: OpenAI and Anthropic list cache-read and cache-write categories separately. Include applicable writes in extras.
The following example uses made-up rates to demonstrate the arithmetic: $1 per million uncached input tokens, $0.10 per million cached input tokens, and $5 per million output tokens. They are not a quote for any particular model.
from decimal import Decimal as D
def estimate(calls, uncached_in, cached_in, output,
rate_in, rate_cached, rate_out, extras_per_call=0):
million = D(1_000_000)
per_call = (
D(str(uncached_in)) * D(str(rate_in)) / million
+ D(str(cached_in)) * D(str(rate_cached)) / million
+ D(str(output)) * D(str(rate_out)) / million
+ D(str(extras_per_call))
)
return per_call, per_call * D(str(calls))
per_call, monthly = estimate(
calls=100_000, uncached_in=1_000, cached_in=500,
output=200, rate_in=1, rate_cached=0.10,
rate_out=5, extras_per_call=0.0002,
)
print(f"Per call: ${per_call:.6f}")
print(f"Monthly: ${monthly:.2f}")It prints Per call: $0.002250 and Monthly: $225.00. I ran this calculation locally with the synthetic inputs. Of the monthly total, $100 is uncached input, $5 is cached reads, $100 is output, and $20 is additional charges. If the 500 cached tokens were billed as ordinary input, the estimate would rise to $270. The hypothetical saving is $45, but only if the requests actually hit the cache and you account for any cache-write cost.
In the example, extras_per_call=0.0002 is just a placeholder. Replace it with your measured average for tool calls and other per-request fees. If a charge is monthly rather than per call, such as storage, add it to the monthly total instead of spreading it blindly across requests. Track failed or retried calls too: they may consume tokens even when the user never sees a successful answer. A cost dashboard should show both dollars per completed task and dollars per raw API call.
Turn the estimate into a model decision
- Run the same set of questions through two or three candidate models. Record token usage, answer quality, response time, and tool-call frequency. A cheaper token can still produce a costlier task if it generates more output or needs retries.
- Use the current official rates from OpenAI, Anthropic, or Google Gemini. Confirm the processing tier, modality, and region; these may change the rate.
- Estimate low, typical, and high scenarios. Vary calls and output length to fit your product. Add room for retries and traffic spikes. Compare the cost with the value each conversation creates, not only with a rival API.
This is an operating budget, not an exact invoice. Actual charges depend on measured usage, your contract, service tier, discounts, caching, tools, and rate changes. Reconcile the estimate with the provider's bill during the first week, then adjust before scaling.