What is a token (in LLMs)?
A token is the basic unit of text a language model reads and generates. Before a model sees any text, a tokenizer splits it into a sequence of tokens, each mapped to a number, and the model works entirely in that numeric sequence rather than on raw characters or words.
Tokens are usually pieces of words
Most modern tokenizers use a method called byte-pair encoding or a close variant, which builds a vocabulary of common substrings from a large body of text. Frequent short words end up as single tokens; longer or rarer words get split into two or more pieces. The word "token" itself is one token in many vocabularies; "tokenization" might split into "token" and "ization". Spaces, punctuation and parts of numbers count as tokens too. For English, one token is roughly four characters, or about three-quarters of a word; the exact ratio varies by tokenizer and language.
Why models work on tokens instead of raw text
A fixed, finite vocabulary of tokens (typically tens of thousands of entries) lets a model represent any input text as a bounded sequence of numbers, which is what the neural network operates on. Character-level input would make sequences much longer and harder to model; whole-word input would need an enormous vocabulary and would still fail on words never seen in training. Sub-word tokenization is the standard compromise.
Why tokens matter for cost and context
Two things about running an LLM scale directly with token count. First, cost: nearly every hosted inference API bills by tokens processed, since token count tracks the compute work involved. (GPU rental bills by time instead; the token count then decides how much work an hour buys.) Second, context window: a model can only attend to a fixed number of tokens at once (the "context length" advertised for a given model), and that budget covers the prompt, any documents fed in, and the reply combined. Long documents or long conversation history all draw from the same token budget, which is why techniques like retrieval-augmented generation exist to fetch only the relevant text instead of stuffing everything into context.
Where token processing actually happens
Every token in a request passes through the model's layers in GPU memory, and every token of the reply is generated one at a time, each depending on all the tokens before it. That's why GPU choice matters for LLM workloads: more memory fits a longer context, and more memory bandwidth generates tokens faster. On a DGX Spark, 128 GB of unified memory holds a large model's weights plus a substantial token cache, and that cache is what limits how long a conversation or document a self-hosted model can handle.
Where GPUwerk fits
GPUwerk rents dedicated Spark hardware for running your own models and bills by time, not by token: $0.79/hour for one Spark or $1.79/hour for a linked pair, billed per minute, however many tokens you push through it. For heavy, sustained token volume that can work out cheaper than per-token API pricing; for light or occasional use, per-token pricing usually wins. See private LLM hosting for how that comparison plays out in practice.