Hidden reasoning tokens: what you actually pay for when a model 'thinks'
aieducation
Reasoning tokens are invisible but billed. Here's how they work, why bills swing wildly, and how to control them.
When a reasoning model answers, it writes private notes to itself before producing the visible reply. Those notes are tokens — and they count on your bill.
What a reasoning token actually is
Every word in the hidden scratchpad is one full forward pass through the model, the same as a visible output token. You pay the output rate for text you never see. On frontier models, hidden thinking can be 5–30x larger than the final answer.
Why bills swing so hard
Three amplifiers stack:
- Effort level: moving from low to max can multiply thinking tokens 50x+.
- Task difficulty: ambiguous prompts trigger longer internal deliberation.
- Model size: large models charge more per token and often think longer.
A single prompt can cost a twentieth of a cent on a small model at low effort and twenty cents on a large model at max — the same question, 400x apart.
What you can measure
Most APIs now return a `reasoning_tokens` or equivalent field alongside input and output counts. Log all three per request. Without that split, cost attribution is guesswork.
How to control them
- Cap thinking tokens per request where the API allows.
- Route by task class — don't send "summarize this email" to a reasoning model.
- Prompt tightly: vague prompts invite longer internal chains.
- Prefer newer model versions, which often reach the same answer with fewer hidden tokens.
- Evaluate on your own tasks before upgrading effort — accuracy gains often plateau well below max.
Where this fits in model choice
Thinking cost is now a first-class selection criterion alongside accuracy and latency. Teams building this discipline into real workflows can talk to our AI consulting team, or see the step-by-step guide to choosing effort level.