Temperature

What is Temperature?

Temperature is a setting that controls how predictable or varied a language model selects tokens. The scores computed by the model for possible continuations remain unchanged. Temperature only changes how strongly these scores influence the resulting probability distribution and therefore the selection of the actual token.

How it works

For each token, the model first computes scores for possible continuations—these scores are not affected by temperature. Only when these scores are converted into a probability distribution and a token is selected does temperature take effect. At a low temperature, the most probable token is chosen almost every time, making the output appear deterministic. At a higher temperature, less probable tokens also gain a realistic chance of being selected.

Why the same scores lead to different answers

The scores computed by the model themselves do not change with temperature. Instead, temperature changes the shape of the resulting probability distribution and therefore the selection rule. This is exactly why the same prompt can produce a different—but still plausible—answer each time it is executed with a higher temperature.

An analogy: Just as a person who has not yet “warmed up” to someone tends to react cautiously, a model with a low temperature almost always chooses the safest and most obvious answer. A higher temperature allows the model to consider more unusual possibilities. Technically speaking, a low temperature causes the model to choose the most likely continuation almost every time, while increasing the temperature gives less probable alternatives a greater chance of being selected.

Why Temperature has practical implications

• Consistency: A low temperature produces almost identical answers for repeated requests, making it suitable for reproducible tasks.
• Creativity: A higher temperature is useful for types of writing where variation is desired.
• Coherence limit: Very high values increasingly allow unlikely tokens and can broaden the output until it becomes incoherent.

Typical example

Very low temperature values produce repetitive, less lively texts because the same most probable tokens are selected almost every time. Very high values, on the other hand, can cause the output to lose its overall thread.

Further reading

Fundamentals
→ Prompts: Why Wording Changes the Answer

Related terms
→ Token
→ Inference