Complex words. Clear meanings.

A glossary for curious minds

Sixteen concepts that make AI news easier to follow. Select a term to reveal its explanation.

Token

A piece of data a model processes as a unit. In text, it can be a word, part of a word or punctuation. The exact split depends on the tokenizer.

Explore the idea ↗
Context

The information available to a model while it processes a request. The context window limits how many tokens it can work with in a single call.

Explore the idea ↗
Attention

A mechanism for combining representations of available tokens using computed weights. It should not be equated with human attention.

Explore the idea ↗
Model weights

Numerical parameters that change during training. They define transformations within the network. Access to weights and permission to use them are separate questions.

Explore the idea ↗
Temperature

A parameter that changes the probability distribution before the next token is selected. Higher temperature generally increases variety, but does not add factual knowledge.

Explore the idea ↗
Hallucination

Confidently phrased but incorrect or unsupported information in a model response. Fluent writing alone is not evidence of truth.

Explore the idea ↗
Multimodality

The ability to work with several types of data, such as text, images and sound. Available capabilities depend on both the model and the application.

Explore the idea ↗
Diffusion

A family of generative approaches in which a model learns to reverse a noising process. Specific implementations and computational methods vary.

Explore the idea ↗
Embedding

A vector representation used to compare items such as texts. Similarity can help retrieval, but does not establish truth.

Explore the idea ↗
RAG

Retrieval-augmented generation: finding relevant material and supplying it as context for a model response.

Explore the idea ↗
Agent

A system that uses a model to choose actions, call tools and respond to their results. Autonomy depends on its permissions.

Explore the idea ↗
Pretraining

The initial large-scale training stage that develops a model’s parameters from data, before later adaptation.

Explore the idea ↗
Fine-tuning

Additional training that adapts model parameters to examples or objectives. It differs from adding instructions to a prompt.

Explore the idea ↗
Quantization

Representing numerical values with fewer bits to reduce storage or memory, with method-dependent quality and performance trade-offs.

Explore the idea ↗
Inference-time computation

Work spent on a particular request, potentially including intermediate steps, alternative solutions and checks. More work does not guarantee correctness.

Explore the idea ↗
Benchmark

A defined evaluation task or collection of tasks. Scores should be interpreted with their test conditions and limitations.

Explore the idea ↗