How to pick an AI model's thinking effort: a step-by-step guide
aieducation
A practical workflow for choosing low, medium, high, or max reasoning effort — matched to task risk, latency budget, and cost per call.
Thinking effort is a dial, not a default. Most teams either leave it on "max" and burn budget, or stick with "low" and miss errors that matter. Here's a repeatable way to pick the right setting per task.
Step 1 — Classify the task by stakes
Ask one question: what happens if the answer is wrong?
- Cosmetic (typo fix, tone edit) → low effort.
- Operational (summaries, routing, drafting) → medium.
- Decision-grade (financial reconciliation, compliance checks) → high.
- Irreversible or regulated (clinical notes, legal clauses) → max, with human review.
Step 2 — Set a latency budget
User-facing chat tolerates ~5 seconds. Background jobs tolerate minutes. Match the dial:
- Low ≈ 5s, Medium ≈ 15s, High ≈ 45s, Max ≈ 1–3 minutes.
If your UX can't absorb the wait, either drop a level or move the task async.
Step 3 — Price the call before you ship
Run 20 representative inputs at two effort levels. Record input + hidden thinking + output tokens. Multiply by model pricing. If high effort costs 10x medium but only lifts accuracy a few points on your eval, medium wins.
Step 4 — Anchor to an eval, not a vibe
Build a small gold-set of 30–50 real tasks with known correct answers. Score each effort level. Pick the lowest setting that clears your accuracy bar.
Step 5 — Re-check when models update
New model versions often make lower effort levels viable for the same task. Re-run the eval quarterly.
For the pricing mechanics behind step 3, see hidden reasoning tokens and what you actually pay for. To operationalize this across teams, our AI consulting team can help.