Blog/What is a reasoning model?
Definition

What is a reasoning model?

By Samuel Seidel · Published September 9, 2026

A reasoning model is an LLM trained to generate an extended sequence of intermediate steps, often called a chain of thought, before it produces a final answer. Instead of answering at once, it works through the problem in tokens the way a person might on scratch paper: restating the question, trying an approach, checking it, sometimes backtracking.

Those intermediate tokens cost inference time and memory. On multi-step logic, math and code, the extra work reliably improves accuracy over a model that answers directly. You pay more compute per answer and get a better answer on the problems where it counts.

How this differs from a model that just answers directly

A standard instruction-tuned model, given a hard multi-step problem, tends to produce a plausible-looking answer in one pass, and on anything requiring several dependent steps that answer is often wrong even when it reads fluently. A reasoning model fixes this through training, not a different architecture. Reinforcement learning on problems with checkable answers (math proofs, code that passes tests or doesn't) rewards chains of reasoning that reach correct results, and teaches the model to spend more tokens on a problem before answering. Same transformer, same kind of forward pass, different learned behavior about when to stop and answer.

What this costs at inference time

Every reasoning token the model generates is a token it has to decode, one at a time, the same as any other output token. A response that used to be a few hundred tokens can become several thousand once reasoning is included, and each of those tokens still costs a full decode step and adds to the KV cache the request is holding. On memory-bandwidth-bound hardware, where decode speed is set by memory bandwidth divided by active parameter bytes (the benchmarks page works through this), more output tokens per request means proportionally more decode time and more cache held per request.

That ties reasoning models to the throughput questions in the batching in LLM inference post: the more tokens one request spends thinking, the more it competes with other concurrent requests for decode capacity.

When it's worth the cost

Reasoning is not universally better; on simple factual lookups or short-form writing, the extra tokens mostly add latency without adding accuracy. It earns its cost on tasks with real multi-step structure: debugging, math, planning, anything where a wrong intermediate assumption compounds into a wrong final answer. The models page lists which models fit on a Spark, and the evaluating a model before deploying guide covers how to test whether the accuracy gain is worth the latency and cache cost on your workload.

Related pages

First top-up: pay $20, get $30 in credit

Measure what the extra reasoning tokens cost you.

Run a reasoning model and a direct-answer model on one dedicated Spark, from $0.79/hour.

Create account