Start with the big idea
An LLM is a trained neural network that operates on numerical representations of text. In a typical autoregressive system, it repeatedly predicts and selects tokens conditioned on the available context.
Understand it in more detail
The input is turned into token IDs using a tokenizer. A learned embedding maps IDs to vectors, and position information helps the network distinguish order.
Transformer layers update representations using attention and feed-forward networks. The output layer produces scores over possible next tokens; a decoding strategy selects a token, which is appended to the context for the next step.
Training creates parameters through optimization on large datasets. Inference uses those learned parameters and the current context. Tools and retrieval may provide extra data, but neither is inherent to every language model.
Worked example
Prompt: 'The capital of France is'. A model may assign a relatively high score to a token corresponding to ' Paris'. That token is still a statistical continuation, not an independent database lookup.
A misconception to avoid
Next-token prediction is an important training/generation mechanism, not a proof that models cannot perform complex reasoning. Also, confident text is not evidence of factual truth.
Check your understanding
Think first, then open each answer to check your explanation. These self-checks do not record a score.
1. What is the difference between training and inference?
Training updates learned parameters. Inference uses them to produce outputs on an input.
2. Can a fluent answer still be false?
Yes. Verify important claims using reliable sources and appropriate tests.
Try it yourself
Sketch the journey from prompt characters to token IDs, embeddings, attention layers, token scores and final text. Circle the places where errors can occur.
Explain it back: Summarize the main idea in your own words and name one limitation or risk.
Compare other AI model terms
Explore the AI model families field guide to see how language models relate to vision models, small models, expert routing, masked-token training and tool-using systems.
Continue learning
Read an independent reference: https://huggingface.co/learn/llm-course/chapter1/1. This link is for reference, not a requirement to create an account. External websites may have different privacy and access policies.
Educational pilot. Terminology, age suitability and activities are subject to expert review. These pages do not send your answers to a server.