A Beginner’s Guide to the Fundamentals
A practical, concept-first guide for technology professionals and curious beginners
- Part 1 — What It Is and Why It Matters
- Part 2 — How It Works ← You are here
How Large Language Models Work
An LLM can be understood at a useful level without deriving the mathematics. The important idea is that text becomes numerical representations, passes through many transformer layers, and is used to predict what should come next.
Step 1: text becomes tokens
The tokenizer converts text into token IDs. The exact token boundaries depend on the model’s tokenizer. Common words may be one token, while unusual words may be split into several pieces. The model does not operate directly on visible characters or words; it operates on these numerical token representations.
Step 2: tokens become vectors
Each token ID is mapped to a vector representation called an embedding. The network transforms these vectors through many layers. During this process, representations become context-sensitive: the representation of a word such as “bank” differs depending on whether nearby text is about money or a river.
Step 3: transformer layers relate the context
Self-attention lets the model weigh relationships among tokens. Other transformations inside each layer refine the representation. Repeating this process across many layers lets the model capture syntax, semantics, patterns of reasoning, styles, and task structures that emerged during training.
Step 4: the model predicts a distribution for the next token
At generation time, the model calculates probabilities for possible next tokens. A decoding strategy chooses one token. That token is appended to the context, and the process repeats. Sophisticated behavior emerges because the model has learned rich internal representations and because each prediction is conditioned on the entire available context—not because it is simply looking up a canned phrase.
Figure 5 – The autoregressive generation loop
Why models can appear to reason—and why they can fail
Large models can learn patterns that support decomposition, analogy, code execution planning, mathematical transformations, and multi-step explanation. Some models are trained or run with techniques that allocate more computation to difficult problems. However, their outputs are still generated by a learned model under uncertainty. A fluent chain of reasoning can contain a hidden error, and a confident answer can still be wrong.
Common Mistake
“Next-token prediction” is a correct description of a common training and generation objective, but calling an LLM “just autocomplete” hides the important part: the network has learned complex representations that make those token predictions useful across many tasks.
The Transformer Architecture
Transformers are the architectural foundation of most modern LLMs. The key innovation is attention: a way to dynamically relate parts of a sequence to one another.
Attention in intuitive terms
Suppose the sentence says, “The server could not reach the database because it was offline.” To interpret “it,” the model must decide which earlier words are most relevant. Attention provides a learned mechanism for assigning importance to relationships in the context rather than relying only on a fixed-size memory of previous steps.
Query, Key, and Value
A common analogy is information retrieval. A Query represents what a token is looking for. Keys represent what other tokens offer for matching. Values contain the information that can be combined if a match is important. The analogy is useful, but the real mechanism is numerical: learned projections create query, key, and value vectors, and attention weights determine how information is mixed.
Position still matters
Attention alone does not inherently encode word order, so transformer systems include positional information. Different architectures implement this in different ways. The goal is the same: let the model distinguish “dog bites man” from “man bites dog.”
Transformer blocks
Figure 6 – Simplified transformer block
Encoder-only, decoder-only, and encoder-decoder
| Architecture | Strength | Typical role |
| Encoder-only | Builds representations of an input | Classification, embeddings, understanding tasks |
| Decoder-only | Generates sequences autoregressively | Most conversational LLMs and code generators |
| Encoder-decoder | Encodes input then generates output | Translation and sequence-to-sequence tasks |
Foundation Models and Large Language Models
A foundation model is a reusable base. An application can use it directly, adapt its behavior, ground it with external data, or place it inside a larger workflow.
What makes a model a foundation model?
A foundation model is trained broadly enough to support many downstream tasks rather than one narrow prediction target. It can be adapted through prompts, retrieval, fine-tuning, tools, or application logic. Language models are the most familiar example, but foundation models can also span vision, audio, and multimodal inputs.
General-purpose versus specialized models
General-purpose models are designed to handle a wide range of topics and instructions. Specialized models may be optimized for coding, mathematics, embeddings, safety classification, speech, or a specific industry. The best model for a workload is not necessarily the largest general-purpose model.
Open-weight versus proprietary
Open-weight models make trained model weights available under a license, allowing organizations to run, inspect, adapt, or redistribute them within the license terms. Proprietary models are normally consumed through a hosted service or API. “Open-weight” does not automatically mean fully open-source: training data, training code, or licensing terms may still be restricted.
Small Language Models and Large Language Models
Smaller models can be cheaper, faster, easier to deploy privately, and sufficient for well-defined tasks. Larger models tend to offer broader capability and robustness but need more compute and memory. Production systems increasingly route different tasks to different models rather than assuming one model should handle everything.
| Dimension | Smaller model | Larger model |
| Cost | Usually lower per request/self-host footprint | Usually higher |
| Latency | Often lower | Can be higher |
| Deployment | More feasible on edge or modest GPUs | Often requires larger accelerators or hosted APIs |
| Capability breadth | Best when task is bounded | Often stronger across diverse tasks |
| Privacy/control | Can be easier to self-host | Depends on deployment option |
The GenAI Model Lifecycle
The model lifecycle separates the expensive process of creating or adapting a model from the operational process of serving it to applications.
Figure 7 – Model lifecycle
Data and pre-training
Pre-training exposes a model to a very large dataset and optimizes it to learn general patterns. This is computationally expensive and is usually performed by model developers rather than ordinary enterprise application teams. Data quality, mixture, filtering, deduplication, and rights management can strongly affect the resulting model.
Instruction tuning and alignment
A raw pre-trained model may be good at continuing text but poor at behaving like an assistant. Instruction tuning trains it on examples of instructions and desired responses. Alignment techniques use preference or policy signals to make behavior more useful and safer. RLHF is one well-known family of methods; newer approaches can use AI-generated preference data or direct preference optimization techniques.
Evaluation before deployment
A model should be measured against the tasks it must perform. General benchmark scores are useful but not enough. An enterprise needs workload-specific evaluation sets covering accuracy, groundedness, safety, latency, cost, and failure modes.
Inference and operations
Inference is the production phase in which users and applications submit requests. Serving systems batch work, manage GPU memory, cache reusable information, enforce quotas, monitor latency, and scale capacity. The operational lifecycle includes observability, safety monitoring, model versioning, and controlled upgrades.
Beginner Mental Model
Most companies should not start by asking, “How do we train our own foundation model?” A more practical sequence is: select a model, evaluate it, ground it with data, add application controls, and only fine-tune if there is a clear reason.
Previous: Part 1 – What It Is and Why It Matters