Reference · Glossary
Token
Last updated
A **token** is a small chunk of text — often a word, part of a word, or punctuation — that the model reads and writes one piece at a time.
#When to use
Estimating cost, fitting prompts into a context window, and debugging why long inputs get truncated or rejected.
#When not to
Don't assume one token equals one word — "ChatGPT" might be one token while "unhappiness" splits into several.
#Example
Rough rule of thumb: ~750 words ≈ 1,000 tokens in English. A 128K context window holds roughly 100 pages of plain text.