How to pick an AI model's thinking effort: a step-by-step guide
aieducation
A practical workflow for choosing low, medium, high, or max reasoning effort based on task complexity, latency budget, and token cost.
Most teams default to the biggest model on max effort and overpay by 10x. The better move is to match the thinking dial to the job. Here's a repeatable workflow.
Step 1: Classify the task
Sort each request into one of three buckets:
- Reflex — rewrite, summarize, extract, classify. No chain of logic needed.
- Analytical — compare options, spot risks in a contract, debug a short script.
- Deliberative — multi-step planning, legal reasoning, math-heavy analysis.
Step 2: Set a latency budget
User-facing chat usually needs an answer in under 5 seconds. Batch jobs can wait minutes. If a human will not read the output within 30 seconds, high or max effort is on the table.
Step 3: Estimate the token bill
Hidden reasoning tokens are billed as output. A rough guide:
- Low: ~500 tokens per answer
- Medium: ~3,000 tokens
- High: ~10,000 tokens
- Max: 30,000+ tokens
Multiply by request volume and output rate before you commit.
Step 4: Run a 20-example bake-off
Pick 20 real requests. Run them at low, medium, and high. Grade the answers blind. In practice, most teams find the smallest setting that passes 90% of cases — then only escalate specific request types.
Step 5: Route dynamically
Use a cheap classifier to tag incoming requests and send reflex work to a small model, analytical work to medium effort, and deliberative work to high. Our step-by-step enterprise model selection guide walks through the routing pattern in detail.
For help wiring this into production workflows, see Silverberry's AI Consulting Services.