Reference · Glossary
Pre-training
Last updated
The expensive first stage where an LLM reads massive text to learn general language patterns — before anyone fine-tunes it for a product.
#When to use
Explaining why base models know grammar and trivia but not your company's private docs.
#When not to
Day-to-day chatting uses **inference**, not pre-training — you are not re-training the model each time you send a prompt.
#Example
GPT-style models are pre-trained on broad internet text, then aligned (RLHF) and optionally fine-tuned for assistants.