A glossary for curious minds
Sixteen concepts that make AI news easier to follow. Select a term to reveal its explanation.
Token
A piece of data a model processes as a unit. In text, it can be a word, part of a word or punctuation. The exact split depends on the tokenizer.
Explore the idea ↗Context
The information available to a model while it processes a request. The context window limits how many tokens it can work with in a single call.
Explore the idea ↗Attention
A mechanism for combining representations of available tokens using computed weights. It should not be equated with human attention.
Explore the idea ↗Model weights
Numerical parameters that change during training. They define transformations within the network. Access to weights and permission to use them are separate questions.
Explore the idea ↗Temperature
A parameter that changes the probability distribution before the next token is selected. Higher temperature generally increases variety, but does not add factual knowledge.
Explore the idea ↗Hallucination
Confidently phrased but incorrect or unsupported information in a model response. Fluent writing alone is not evidence of truth.
Explore the idea ↗Multimodality
The ability to work with several types of data, such as text, images and sound. Available capabilities depend on both the model and the application.
Explore the idea ↗Diffusion
A family of generative approaches in which a model learns to reverse a noising process. Specific implementations and computational methods vary.
Explore the idea ↗Embedding
A vector representation used to compare items such as texts. Similarity can help retrieval, but does not establish truth.
Explore the idea ↗RAG
Retrieval-augmented generation: finding relevant material and supplying it as context for a model response.
Explore the idea ↗Agent
A system that uses a model to choose actions, call tools and respond to their results. Autonomy depends on its permissions.
Explore the idea ↗Pretraining
The initial large-scale training stage that develops a model’s parameters from data, before later adaptation.
Explore the idea ↗Fine-tuning
Additional training that adapts model parameters to examples or objectives. It differs from adding instructions to a prompt.
Explore the idea ↗Quantization
Representing numerical values with fewer bits to reduce storage or memory, with method-dependent quality and performance trade-offs.
Explore the idea ↗Inference-time computation
Work spent on a particular request, potentially including intermediate steps, alternative solutions and checks. More work does not guarantee correctness.
Explore the idea ↗Benchmark
A defined evaluation task or collection of tasks. Scores should be interpreted with their test conditions and limitations.
Explore the idea ↗