Fine-Tuning

What is Fine-Tuning?

Fine-Tuning further adapts a pretrained Foundation Model using a much smaller, specialized set of examples. Instead of billions of general text examples as used during pretraining, thousands of carefully selected examples are used—the fundamental knowledge acquired during pretraining is intended to remain largely intact.

How it works

Fine-Tuning works technically in the same way as the original training: the model predicts tokens, and deviations from the desired responses lead to small parameter adjustments. The difference lies in the scale and selection of the data—instead of broad Internet text, carefully selected examples for the desired task or style are used.

Why Fine-Tuning avoids complete retraining

Training a Foundation Model completely from scratch would be uneconomical for most use cases. Fine-Tuning makes it possible to reuse the existing language understanding while shifting the model’s response behavior toward a specific direction. This enables specialized models with significantly less computing effort than complete retraining.

Fine-Tuning does not replace well-written prompts and is not a suitable way to “teach” a model current facts that change frequently—for that purpose, approaches such as retrieval are generally more appropriate. Fine-Tuning primarily changes behavior and style rather than the model’s knowledge.

Why Fine-Tuning has practical implications

• Cost: Significantly less expensive than complete retraining because the existing language understanding is reused.
• Specialization: Enables models tailored to a particular tone, domain, or response format.
• Formatting bias: Uniformly structured Fine-Tuning data can reduce the flexibility with which a model responds to different prompt formats.

Typical example

A Foundation Model is trained with example conversations to follow instructions and respond in a particular style. The result is often referred to as a Chat Model or Instruct Model, in contrast to the pure Foundation Model.

Further reading

Fundamentals

→ How a Language Model Works

Related terms

→ Training
→ Parameter