Inference vs training: what's the difference?
Training is the process of producing a model's weights by repeatedly adjusting them against a dataset until the model's outputs improve on some measure. Inference is the process of using an already-trained model's fixed weights to produce an output for one specific input. Training happens once, or periodically for updates; inference happens every time the model is used. Most self-hosted deployments only do inference, plus the occasional small fine-tune.
Two different jobs, two different hardware profiles
Training a model, from scratch or by fine-tuning, means running it forward, measuring how wrong the output was, and pushing weight updates backward through the network. That cycle repeats across a dataset, often for days or weeks on many GPUs at once. It needs memory for the gradients and optimizer state as well as the weights, several times the footprint of the weights alone.
Inference runs the model forward once per request and holds only the weights and the current context. So a machine that suits inference doesn't automatically suit training the same model: the memory and interconnect demands differ.
Why this distinction matters when choosing hardware
A GPU or cluster spec sheet that looks impressive for one phase can be the wrong shape for the other. Training benefits from many GPUs linked by fast interconnects so gradients can be synchronized across them; a single machine, however capable, hits a ceiling that a multi-node cluster doesn't. Inference for a single model generally needs no cross-node synchronization, since each request is served independently. Buying training-grade infrastructure for inference-only work wastes money, and so does the reverse.
A Spark is for inference and small fine-tunes
A GPUwerk Spark, with 128 GB of unified memory in a single node, is built for inference and for fine-tuning models that fit in that memory, at $0.79/hour, or $1.79/hour for a two-node cluster. It is not a substitute for a large multi-GPU training cluster if the job is pretraining a model from scratch or full-parameter fine-tuning at frontier scale; that work needs far more aggregate memory and interconnect bandwidth than a one- or two-node Spark setup provides. If your work is deploying and serving an existing model, or adapting one with a parameter-efficient method, a Spark is sized for that. If you are training a new model from the ground up, it isn't the right tool, and I'd rather say so than sell it as one.
A rough way to tell which phase you're looking at
If a workload's memory usage and compute time scale with a fixed dataset that gets processed repeatedly over many passes, and the process ends with a saved set of weights, that's training. If a workload's memory usage scales with the size of a single input and its compute time scales with the length of one response, and nothing about the model's underlying weights changes afterward, that's inference. Most day-to-day use of a deployed model (chat interfaces, API calls, batch scoring of existing data) is inference. Training is run occasionally to produce or update a model.
What is fine-tuning, in this framing
Fine-tuning sits closer to the training side of this split, since it does update weights, but at a much smaller scale than training from scratch. See what is fine-tuning for the mechanics, and fine-tuning vs RAG for how it compares against leaving weights untouched entirely.