Read the passage below, then write a short technical summary of it for an engineer who has never run a model locally.

Running a large language model on your own computer is mostly a question of memory. The model is a file of weights, often tens of gigabytes, and every one of those weights has to be read for every token the model writes. That is why generation speed tracks memory bandwidth far more closely than it tracks raw compute. A graphics card with fast video memory can stream the weights quickly, so it writes tokens quickly. A laptop that shares slower system memory between the processor and the graphics chip writes them more slowly, even when its processor is new.

Before the model writes anything, it has to read your prompt. This first phase is called prefill, or prompt processing. Here the work is different: all the prompt tokens can be handled together, in large batches, so the limit is compute rather than bandwidth. That is why prompt processing is usually many times faster than generation, and why a long prompt and a short prompt give very different prompt speeds on the same machine. A benchmark that does not fix the prompt length cannot be compared with another one.

Quantization changes the picture again. A model stored with four bits per weight is roughly a quarter of the size of the same model stored with sixteen bits, so it fits in less memory and streams faster. The cost is a small loss of quality, which grows as the bit count falls. Most people running models at home use four-bit or five-bit files because they are the best trade between speed, memory and quality.

Heat and power matter too. A laptop on battery often lowers its clock speeds to save energy, and a machine that has been running hard for several minutes may slow down as it warms up. The first run after loading a model can also be slower, because caches are cold. Careful benchmarks plug the machine in, close other heavy programs, do one warm-up run, and only then record results.

Finally, the software matters. The same model on the same hardware can run at different speeds in different programs, or in the same program with different settings, such as flash attention or the type used for the key-value cache. A fair comparison records these settings alongside the numbers, so that a reader can tell whether a difference comes from the hardware or from the setup.