Home › Guides › Reasoning models
Tech Explained · 2026What Are Reasoning Models in 2026? How Test-Time Compute Works, Reasoning vs Standard LLMs and Real Token Costs
Reasoning models are language models that spend extra tokens working through a problem before answering, instead of replying in one pass. Those hidden thinking tokens are billed at the normal output rate, and the Claude API will not accept a thinking budget below 1,024 tokens. You buy accuracy on hard tasks with money and latency.
- Thinking is a billing event, not a feature flag. The thinking tokens a model generates are charged as output tokens even when the API returns only a summary of them, so your invoice will not match what you see on screen.
- The floor is 1,024 tokens. Claude's platform docs, checked on 31 October 2026, say a budget_tokens value below 1,024 is rejected outright, so there is no such thing as a cheap sprinkle of reasoning.
- Manual budgets are on the way out. The same docs mark thinking.type "enabled" with budget_tokens as deprecated on the Claude 4.6 models in favour of adaptive thinking, where the model decides per request whether to think at all.
- Most production traffic does not need it. Classification, extraction, routing and formatting get almost nothing from extra deliberation and pay full price for it.
- The real skill is routing. Deciding which requests deserve deliberation, and proving the lift with an eval, is what separates an AI engineer from someone who turned a parameter on.
- Measure cost per correct answer. Per-token price is the wrong unit once a single request can emit thousands of invisible tokens.
Your support-triage agent worked beautifully in the demo, and now the finance team wants to know why last month's API bill tripled while request volume stayed flat. Nothing in your logs looks wrong. The prompts are the same length, the responses are the same length, and the model is the same one you shipped with. What changed is that someone turned on thinking, and the model is now writing two thousand tokens of private deliberation before every two-line reply. This guide explains what reasoning models do, what they cost, where the extra tokens actually buy you something, and where they quietly do not.
What reasoning models are, and what changed in 2026
A standard language model produces its answer token by token in a single forward pass. Ask it a multi-step question and it has to commit to the first words of the answer before it has worked out the last ones. A reasoning model is trained to do something different: generate a long private working-out first, inspect it, and only then write the answer you see. That working-out is not a prompt trick. The model has been trained with reinforcement on whether the final answer was right, which teaches it to backtrack, check arithmetic and notice when its own first approach was wrong.
The vocabulary differs by vendor, and the differences matter when you read a docs page. Anthropic calls it extended thinking and returns thinking blocks. OpenAI exposes a reasoning_effort setting with discrete levels. Google ships a thinking variant of its Gemini line. Open-weight families such as DeepSeek R1 and the Qwen reasoning checkpoints emit their deliberation inline between delimiters, which is why you sometimes see raw scratchpad text leak into a chat UI that was not built for it.
The shift in 2026 is that the knob is disappearing. Anthropic's platform docs, read on 31 October 2026, describe manual extended thinking as deprecated on the Claude 4.6 models and point to adaptive thinking instead, where the model judges per request whether deliberation is warranted and how much. Requests using the old parameter still work. But the direction is clear: vendors would rather decide this for you than let every team guess a budget, because most teams guess badly in one of two directions.
Where thinking tokens enter the request
The scratchpad never reaches your user, and it is still on your bill.
Flow reflects Anthropic's extended thinking and interleaved thinking documentation, checked 31 October 2026.
How test-time compute works, step by step
Test-time compute is the plain description of what you are buying: compute spent at inference, after training is finished. For a decade the lever that made models smarter was training scale. Reasoning models add a second lever, and it is one you control per request.
The four mechanisms underneath the parameter
First, longer chains. The model generates more intermediate steps before committing, which gives it room to notice a wrong turn. Second, self-verification, where the model re-derives a result a second way and compares. Third, sampling and selection, generating several candidate solutions and picking the one that survives a check, which is where cost climbs fastest. Fourth, interleaving with tools: the model thinks, calls a tool, then thinks again about what came back. Anthropic's docs require the interleaved-thinking-2025-05-14 beta header for that behaviour on Claude Opus 4.5, Sonnet 4.5 and earlier Claude 4 models, and note Haiku 4.5 does not support it.
That fourth mechanism is the one that matters for agents, and it is the one people underestimate. In a single-shot chat, thinking happens once. In an eight-turn agent loop, it happens eight times, and your cost per task grows with the number of turns rather than the size of the answer. Building and costing those loops is the spine of 360DT's live AI Engineer course, where you instrument a tool-using agent and watch the token meter move as you change the loop.
Three numbers worth remembering
Two are documented facts, one is an industry estimate. Treat them differently.
Budget floor from Anthropic's platform docs and pricing from published per-million rates, both checked 31 October 2026; the inference share is a published industry estimate, not a measurement of your workload.
Reasoning models vs standard LLMs: 11 differences that matter in production
The marketing framing is that reasoning models are smarter. The engineering framing is that they shift your failure modes. Here is the comparison that actually drives decisions.
| Criterion | Standard LLM | Reasoning model |
|---|---|---|
| Tokens per request | Roughly the length of the answer | Answer plus a hidden scratchpad, often several thousand tokens |
| Billing | Visible output tokens only | Thinking tokens billed at the output rate, even when summarised |
| First-token latency | Typically under a second | Seconds to tens of seconds, because thinking comes first |
| Streaming experience | Text appears immediately | Long silence, then a block, unless you stream thinking |
| Best-fit tasks | Extraction, classification, rewriting, summarisation | Multi-step math, debugging, planning, constraint satisfaction |
| Behaviour on easy tasks | Correct and cheap | Correct and expensive, sometimes overthought |
| Prompt style that works | Few-shot examples, explicit step instructions | State the goal and the constraints, do not script the steps |
| Chain-of-thought prompting | Often helps | Largely redundant, and can fight the trained process |
| Caching benefit | Large, prefixes are stable | Smaller, because the expensive part is freshly generated output |
| Debuggability | You see everything the model produced | You may see only a summary of the reasoning |
| Cost control lever | Model choice and prompt length | Effort level, budget, and above all request routing |
Two rows deserve emphasis. The prompt-style row trips up experienced teams: the careful "think step by step, first do X, then do Y" scaffolding you built for a 2024 model is at best wasted tokens on a reasoning model and at worst a constraint that stops it exploring. Rewrite those prompts to state the goal and the acceptance criteria, then get out of the way.
The caching row is the one your finance team will care about. Prompt caching cuts the cost of a long stable system prompt, which is input. Thinking tokens are output, and they are generated fresh every time. If you already read our guide on how prompt caching works, the thing to carry across is that caching and reasoning pull on different halves of the invoice.
Also read: What Is Agentic RAG in 2026? for how the same loop economics play out in retrieval.
What reasoning models cost: thinking tokens and real per-request numbers
Published per-million rates let you do the arithmetic yourself, and you should, because the answer is usually not what people expect. Rates below are the per-million input and output prices listed for Anthropic's current line as checked on 31 October 2026: Haiku 5.5 at 0.10 dollars and 0.50 dollars for prompts up to 100,000 tokens, Sonnet 5.5 at 2 dollars and 10 dollars, Opus 5.5 at 4 dollars and 20 dollars.
| Scenario (per 1,000 requests) | Tokens billed as output | Haiku 5.5 | Sonnet 5.5 | Opus 5.5 |
|---|---|---|---|---|
| Short answer, no thinking | 300 | $0.15 | $3.00 | $6.00 |
| Same answer, 1,024 thinking tokens | 1,324 | $0.66 | $13.24 | $26.48 |
| Same answer, 4,000 thinking tokens | 4,300 | $2.15 | $43.00 | $86.00 |
| Agent task, 6 turns, 2,000 thinking each | 12,600 | $6.30 | $126.00 | $252.00 |
Read the second row again. Turning on the smallest legal thinking budget multiplies the output side of a short-answer workload by roughly four and a half times. That is the floor, not the ceiling.
The multiplier that decides your architecture
Not the per-token price. The volume of tokens one request can emit.
Output tokens billed for a 300-token answer once the minimum 1,024-token thinking budget is attached, on the same model and the same prompt.
Arithmetic from the documented 1,024-token minimum budget, checked 31 October 2026. Independent 2026 preprints studying effort tiers report far larger spreads at high effort, in the range of several times to more than ten times, with latency rising faster than cost.
Effort levels, extended thinking and the honest caveat
A 2026 arXiv preprint on reasoning effort as an API contract reports that pushing effort to its highest tier lifted competition-math pass rates by roughly 18 to 22 points on frontier models, while inflating fees several-fold and latency by considerably more. Those are preprint figures on math benchmarks, not a promise about your ticket classifier. The pattern that replicates is the shape, not the magnitude: the lift is real and large on genuinely hard multi-step problems, and close to zero on everything else.
Here is the part the vendor blogs skip. On a task your users actually send, the accuracy gain from maximum reasoning is often smaller than the gain you would get from fixing your retrieval, your schema or your prompt. Reasoning is the expensive lever. Reach for it after you have pulled the cheap ones, not before.
Where reasoning models earn their keep, and where they just burn money
Take a two-person data platform team at a mid-size Indian insurer, running roughly 40,000 LLM requests a month across three features: a document classifier for incoming claim emails, a text-to-SQL assistant for the ops team, and a reconciliation agent that chases mismatches between the policy system and the payment gateway. One budget, three very different answers. This is an illustrative scenario, not a customer story.
| Feature | Monthly volume | Does reasoning help? | What we would ship |
|---|---|---|---|
| Claim email classifier | 30,000 requests | No. The label set is fixed and the signal is in the first 200 words | Smallest model, no thinking, a tight eval set of 300 labelled emails |
| Text-to-SQL assistant | 8,000 requests | Only on joins across more than three tables | No thinking by default, escalate on schema complexity or a failed dry run |
| Reconciliation agent | 2,000 tasks | Yes. Multi-step, tool-using, and a wrong answer costs real money | Mid-tier model with thinking on, hard cap on turns, human review above a threshold |
Spend follows that table, not the request count. The 2,000 reconciliation tasks can easily cost more than the 30,000 classifications, and that is the correct outcome: deliberation is going where a mistake is expensive. Getting to that allocation is a routing problem, and routing is a design decision you can defend in an architecture review, which is exactly the muscle the CCAR-F architect foundations track is built around.
The escalation pattern that works
Run the cheap path first. Attach a verifier: a SQL dry run, a JSON schema validation, a unit test, a second model asked only "is this answer consistent with the evidence". When the verifier fails, retry once with thinking enabled. Most teams find the escalation rate settles in the low single-digit percentages, which means you get reasoning-model accuracy on the hard tail at close to cheap-model cost. You cannot build that without evals, and our walkthrough of how to ship an eval pipeline is the prerequisite, not the follow-up.
- You set max_tokens and forgot it includes thinking. The model spends its allowance deliberating and gets truncated mid-thought, so you pay full price for no answer at all. Budget them separately and leave headroom.
- Your timeouts were tuned for a one-second model. A 30-second client timeout in front of a high-effort request fails after you have been billed for every thinking token.
- Someone turned thinking on globally in a config PR. This is the single most common cause of a tripled bill with flat traffic. Make the setting per-route and log it on every call.
- You kept your old chain-of-thought prompt. Now you are paying for instructed reasoning and trained reasoning at the same time, and the two do not always agree.
- You are storing raw thinking blocks in logs you did not classify. Scratchpads can restate customer data verbatim. Treat them as the same data class as the input.
Are reasoning models worth learning in 2026, and what do you actually learn?
Yes, but not as a topic. Nobody is hiring for "knows about reasoning models". What gets you shortlisted is the adjacent engineering: cost accounting per feature, eval design, routing logic, and the judgement to say no to deliberation on 90 percent of traffic. That is an AI engineering skill set, and it overlaps heavily with what agent work demands anyway. If you are weighing the certification route, the full certifications overview lays out which credential maps to which job family.
A six-week plan to go from reading about this to shipping it
Each stage ends in an artefact you can show an interviewer.
Instrument before you optimise
Log model, effort setting, input tokens, output tokens and latency on every call in one small app. Produce a cost-per-feature table for a week of real traffic.
Build the eval set first
Collect 200 to 300 real requests with known-correct answers. Score your current setup. You now have a baseline that makes every later claim checkable.
Run the A/B that matters
Same prompts, same model, thinking off then on at the minimum budget. Report accuracy delta against cost delta and cost per correct answer, not raw scores.
Write a verifier
Pick one feature and add a deterministic check: schema validation, a SQL dry run, a unit test. Measure how often the cheap path fails it.
Ship escalation routing
Cheap path by default, retry with thinking on verifier failure, hard cap on retries and agent turns. Record the escalation rate as a dashboard metric.
Write the one-page decision memo
Which routes deliberate, what it costs, what accuracy it bought, what you would cut first under budget pressure. This document is the portfolio piece.
Sequence reflects the documented cost behaviour of thinking tokens as checked 31 October 2026; week lengths are a suggested cadence, not a benchmark.
If you want that sequence with a trainer watching your code rather than alone on a weekend, the live Generative AI Developer programme covers the build side and the MLOps Engineer course covers the instrumentation and deployment side. For engineers who sit in front of customers and have to defend these cost decisions to a client's architecture board, that conversation is the core of the Forward Deployed Engineer programme. Developers who want the API-level depth first, including how thinking blocks round-trip through tool use, usually start with the CCDV-F developer prep course.
Learn to decide when a model should think, and prove it with numbers
Build agents that plan, use tools and act, certified on both Microsoft Copilot Studio and Claude Code, the two stacks real job postings name. 100+ hours live over 16 weeks, with RAG, agent loops and token-cost instrumentation built into the labs.
Explore the course
Limitations, and who should not bother
Reasoning does not fix wrong facts. A model that deliberates for 4,000 tokens on a question whose answer is not in its context or its training will produce a beautifully argued wrong answer, and the argument will make it more convincing, not less. If your failure mode is hallucinated specifics, retrieval is your fix and deliberation is an expensive distraction.
It also does not help much on anything latency-sensitive and user-facing. An autocomplete, a search suggestion, an in-chat typeahead: these live or die on sub-second response, and thinking is incompatible with that by construction. Teams running high-volume, low-stakes classification should skip reasoning entirely and spend the saved budget on a better eval set.
And there is a governance edge that gets missed. Summarised reasoning means you cannot always reconstruct how a decision was reached, which is awkward if an auditor asks. If you work under DPDP obligations or an internal model-risk policy, decide early whether you need to retain reasoning traces, and if you do, classify them as sensitive, because they often quote the input verbatim.
The verdict: what we would actually do
If you own a production feature today, leave thinking off by default, add one deterministic verifier, and escalate to a reasoning model only when that verifier fails. You will pay a few percent of your traffic at the expensive rate and get most of the accuracy benefit, and you will be able to show your finance team a number instead of a shrug. The trade-off you are accepting is that a small slice of hard requests takes several seconds longer and occasionally slips through on the cheap path; that is almost always a better deal than paying four times over for deliberation on work that never needed it.
If you are learning rather than shipping, the valuable artefact is not a demo that thinks. It is the one-page memo from week six: which routes deliberate, what it cost, what accuracy it bought. Build that once and you will interview better than someone who has read every model card. If you want to build it with a trainer watching your loop and your token meter, the live AI Engineer course is where that work happens, 100+ hours over 16 weeks, with a free demo class if you want to see the teaching style first.
Related guides
- How Do Large Language Models Work? Start here if the words tokens and attention are doing a lot of unexamined work in your head.
- What Are Small Language Models in 2026? The other half of the routing decision, for the traffic that should never reach a big model.
- What Is Context Engineering in 2026? The cheap lever to pull before you pay for deliberation.
- AI Engineer Interview Questions 2026 Because cost-routing decisions are exactly what senior interviewers probe.
- AI Engineer Salary in India 2026 What the skills in this guide are currently advertised to be worth.
- CCAR-F Certification Cost and Value in 2026 If you want the architect credential behind the routing decisions above.
Frequently asked questions
What are reasoning models in simple terms?
Reasoning models are language models trained to write a long private working-out before they answer, then use it to produce the final reply. The deliberation is generated at inference time, billed as output tokens, and usually hidden from the user. On hard multi-step problems it raises accuracy; on simple ones it mostly raises cost and latency.
Are thinking tokens charged, and how do I see them?
Yes. Anthropic's extended thinking documentation, checked 31 October 2026, states that you are billed for the full thinking tokens even when the response contains only a summary, so your billed output count will not match what you can read. Use the usage fields in the API response rather than counting characters in the reply.
What is test-time compute versus training compute?
Training compute is spent once, by the model provider, to produce the weights. Test-time compute is spent by you, on every request, to let the model deliberate longer. The practical difference is that test-time compute is a per-request budget decision you can route, cap and A/B test.
Is a reasoning model better than a standard LLM for RAG?
Not by default. If retrieval returns the right passages, a standard model usually answers correctly and cheaply. Reasoning helps when the answer requires combining several retrieved facts, resolving a contradiction between sources, or deciding what to search for next, which is the agentic retrieval case rather than plain question answering.
What is the minimum thinking budget I can set?
On the Claude API the documented minimum for budget_tokens is 1,024, and smaller values are rejected. Note that the platform docs mark manual budgets as deprecated on the Claude 4.6 models in favour of adaptive thinking, where the model decides per request, so new builds should plan for the adaptive behaviour.
Do I still need chain-of-thought prompting with a reasoning model?
Generally no. Instructing the model to think step by step duplicates a process it has already been trained to run, adds tokens, and can constrain how it explores. Write the goal, the constraints and the acceptance criteria instead, and keep your few-shot examples for the cheaper non-reasoning models where they still pay off.
How do I stop a reasoning model from tripling my bill?
Make the setting per-route rather than global, log the effort level on every call, cap agent turns, and route by escalation: run the cheap path first and only retry with thinking when a deterministic verifier fails. Track cost per correct answer so a saving that quietly destroys accuracy shows up immediately.
Which 360DT course covers this in practice?
The AI Engineer course is the closest fit: 100+ hours live over 16 weeks, Saturdays and Sundays from 8 to 11 PM IST, covering RAG, agent loops and tool use with the token accounting attached. Developers wanting API-level depth often pair it with the CCDV-F prep course, and architects with CCAR-F.
About this guide. 360 Digital Transformation is an Anthropic Partner and a Microsoft Authorized Training Partner. Other certification bodies, vendors and employers named here are not affiliated with us. Product features and pricing change often; figures cited were checked on 31 October 2026.
Python
Claude API
FAISS
ChromaDB
LangChain
LangGraph
CrewAI
AutoGen (AG2)
