First, text becomes tokens
A model receives a sequence of tokens rather than words as we know them. A token can be a word, part of a word, punctuation or another fragment. The exact split depends on the tokenizer.
Think of the next-word suggestions on a phone keyboard. A language model also estimates possible continuations, but uses much more complex patterns and context.
Several possibilities, not just one
For the next token, the model computes a probability distribution. How a token is selected from that distribution affects the result. The selected token is then added to the context, and the process repeats.
An educational simulation with illustrative probabilities. No real model runs here; each word is treated as a single token for simplicity.
What temperature changes
All else being equal, a low temperature strengthens the advantage of the most likely options. A higher temperature flattens the distribution, allowing more varied choices. It does not add knowledge or guarantee a creative or correct answer.
Even a grammatically convincing continuation can be factually wrong. Text probability and factual truth are different things.
KEEP EXPLORINGMake sense of another idea↗