Inference vs training: what's the difference?
Training is the process of producing a model's weights by repeatedly adjusting them against a dataset until the model's outputs improve on some measure. Inference is the process of using an already-trained model's fixed weights to produce an output for one specific input. Training happens once (or periodically, for updates); inference happens every time the model is actually used.
Two different jobs, two different hardware profiles
Training a model from scratch, or fine-tuning it, means running the model forward, measuring how wrong its output was, and pushing weight updates backward through the network, a cycle repeated across a dataset, often for days or weeks on many GPUs at once. That requires storing not just the weights but the gradients and optimizer state associated with them, which multiplies the memory footprint several times over compared to the weights alone. Inference, by contrast, runs the model forward exactly once per request and needs to hold only the weights and the current context in memory. A machine well-suited to inference isn't automatically well-suited to training the same model, because the memory and interconnect demands are different.
Why this distinction matters when choosing hardware
A GPU or cluster spec sheet that looks impressive for one phase can be the wrong shape for the other. Training benefits from many GPUs linked by fast interconnects so gradients can be synchronized across them; a single machine, however capable, hits a ceiling that a multi-node cluster doesn't. Inference for a single model generally doesn't need that cross-node synchronization at all, since each request is served independently. Buying training-grade infrastructure to run inference-only workloads, or the reverse, both waste money against what the job actually needs.
Where a Spark fits, honestly
A GPUwerk Spark, with 128GB of unified memory in a single node, is built for inference and for fine-tuning models that fit comfortably in that memory, at $0.79/hour, or $1.79/hour for a two-node cluster. It is not a substitute for a large multi-GPU training cluster if the job is pretraining a model from scratch or full-parameter fine-tuning at frontier scale; that work needs far more aggregate memory and interconnect bandwidth than a one- or two-node Spark setup provides. If your work is deploying and serving an existing model, or adapting one with a parameter-efficient method, a Spark is sized for that. If it's training a new model from the ground up, it isn't the right tool, and we'd say so rather than sell it as one.
A rough way to tell which phase you're looking at
If a workload's memory usage and compute time scale with a fixed dataset that gets processed repeatedly over many passes, and the process ends with a saved set of weights, that's training. If a workload's memory usage scales with the size of a single input and its compute time scales with the length of one response, and nothing about the model's underlying weights changes afterward, that's inference. Most day-to-day usage of a deployed model, chat interfaces, API calls, batch scoring of existing data, is inference; training is comparatively rare, run occasionally to produce or update a model rather than continuously to use one.
What is fine-tuning, in this framing
Fine-tuning sits closer to the training side of this split, since it does update weights, but at a much smaller scale than training from scratch. See what is fine-tuning for the mechanics, and fine-tuning vs RAG for how it compares against leaving weights untouched entirely.