Deterministic

What does deterministic mean?

Deterministic describes a computation that produces the same output for identical input. In practice, for language models this depends primarily on the inference configuration—especially the temperature setting used to select tokens from the computed probability distribution—not solely on the model architecture.

How it works

A language model computes a probability distribution over possible continuations for every token—this computation itself always proceeds in the same way. The output becomes deterministic only through the selection rule: At a very low temperature, the most probable token is chosen almost every time, causing repeated requests to produce nearly identical answers.

Why determinism is a matter of configuration

At higher temperatures, token selection becomes non-deterministic. Less probable tokens also gain a realistic chance of being selected, allowing the same prompt to produce a different, yet still plausible, answer each time it is executed. For tasks that require reproducible results—such as automated testing or data evaluation—a setting that is as deterministic as possible is therefore important.

Even at the lowest temperature setting, a language model is not necessarily 100 percent deterministic. Depending on the inference system, rounding effects during parallel computation can lead to small differences. In this context, “deterministic” describes a very strong tendency rather than an absolute guarantee.

Why determinism has practical implications

• Reproducibility: Tests, evaluations, or comparisons between prompt variants benefit from settings that are as deterministic as possible.
• Creativity vs. consistency: Non-deterministic behavior is desirable for generating varied text but can be a disadvantage when consistent formatting is required.
• No absolute guarantee: Even at the lowest temperature setting, identical output cannot be guaranteed with complete certainty.

Typical example

With a temperature close to zero, the same prompt produces almost the same answer every time it is executed. At higher temperatures, repeated executions of the same prompt can produce different, yet still plausible, responses.

Further reading

Fundamentals
→ Prompts: Why Wording Changes the Answer

Related terms
→ Temperature
→ Inference