What Is an LLM? Large Language Models Explained

Beginner 10 min read

What you'll learn

✓What a large language model actually is, under the marketing
✓How tokens and next-token prediction work
✓The difference between training and inference
✓What a context window is and why it matters
✓Why models hallucinate, and when LLMs are the wrong tool

Prerequisites

•A little programming background helps — see What Is Python?

Large language models — LLMs — sit behind ChatGPT, Claude, Gemini, and most “AI” features shipping in software today. The marketing around them is loud; the underlying ideas are surprisingly approachable. This post explains what an LLM actually is, how it works at a mechanical level, and where it fits (and doesn’t).

No prior machine learning background required.

What an LLM actually is

A large language model is a statistical model of text. Given some text as input, it predicts what text is likely to come next — one small piece at a time.

That is the whole core idea. Everything else — chat, summarization, code generation, agentic behaviour — emerges from doing this prediction extremely well at very large scale.

The “large” in LLM refers to two things:

Parameters. Modern models have hundreds of billions of internal numbers (weights) tuned during training. These weights encode what the model has learned about language.
Training data. Models are trained on enormous corpora — trillions of words of books, websites, code, and conversations.

The result is a function that takes a string in and produces a probability distribution over what comes next.

Tokens, not words

LLMs don’t read characters or words directly. They read tokens — short chunks of text, often two to five characters long. The word unbelievable might split into un, believ, able. Common words like the are usually a single token. Spaces and punctuation are tokens too.

A small mental model:

# Conceptual — actual tokenizers are more complex
text = "Hello, world!"
tokens = ["Hello", ",", " world", "!"]
# output: 4 tokens

Why tokens? They strike a balance between flexibility (the model can compose rare words from smaller pieces) and efficiency (sequences are shorter than character-by-character).

This matters in practice because:

API pricing is per token, not per word.
Context limits are measured in tokens.
A page of English is roughly 500–800 tokens.

Next-token prediction

At inference time, the model does one thing repeatedly: given the tokens so far, sample the next token.

A conceptual loop:

# Conceptual pseudocode
prompt = "The capital of France is"
tokens = tokenize(prompt)

for _ in range(20):
    probs = model.predict_next(tokens)   # probability over vocabulary
    next_token = sample(probs)            # pick one
    tokens.append(next_token)
    if next_token == END:
        break

print(detokenize(tokens))
# output: The capital of France is Paris.

A few consequences worth internalising:

The model has no plan. It chooses one token at a time. Coherence across a long response comes from each token being conditioned on everything before it, not from forethought.
Sampling involves randomness. A temperature knob controls how adventurous the choice is. Temperature 0 picks the most likely token every time; higher values diversify.
Streaming is natural. Because tokens are produced one at a time, you can show the response as it’s generated — that’s why chat UIs “type.”

Training vs inference

Two distinct phases:

Training is the expensive part. The model sees trillions of tokens of text and adjusts its weights so that, for each next-token prediction it makes during training, the probability it assigns to the actual next token goes up. This runs on thousands of GPUs for weeks or months and costs millions of dollars.

Inference is what happens when you use the model. The weights are frozen. The model takes your prompt, runs forward through the network, and produces tokens. Each request is independent — the model has no memory of previous conversations beyond what you include in the prompt.

This separation is why LLMs feel both magical and limited. The intelligence was baked in during training; inference is just executing it.

The context window

The context window is the maximum number of tokens the model can consider at once — prompt plus the response it’s generating.

Modern models have context windows from 32,000 tokens up to several million. A 200k-token window is roughly a 400-page book.

What lives in the context:

The system prompt (instructions to the model)
The conversation history so far
Any documents you’ve pasted or attached
The tokens the model is currently generating

Everything outside the window is invisible to the model. If a conversation grows longer than the window, the oldest tokens fall off and the model effectively forgets them. This is why long chats sometimes lose track of early instructions.

Reflection. Open the chatbot you use most and check its context window in the docs. Then notice: when you upload a long PDF, you are spending tokens out of that budget. The reply also consumes tokens from the same budget. Designing prompts is partly a budgeting exercise.

Why models hallucinate

A hallucination is when a model produces confident-sounding text that is factually wrong — a fake citation, an invented function name, a wrong date.

This isn’t a bug; it follows directly from how the model works. The model picks the next token that is statistically plausible given what came before. “Plausible” and “true” are correlated but not identical. When the model has seen lots of text about a topic, plausible-and-true overlap heavily. When it hasn’t — obscure facts, recent events, niche libraries — plausibility runs ahead of truth.

Ways the ecosystem mitigates this:

Retrieval-augmented generation (RAG) — give the model the source documents in its context so it doesn’t have to recall from weights.
Tool use — let the model call a calculator, a search engine, or a database instead of guessing.
Reinforcement learning from human feedback during training, which makes the model more likely to admit uncertainty.

None of these eliminate hallucinations. They reduce the rate. Treat LLM output as a confident draft that needs verification for anything that matters.

What LLMs are good at

Honest list:

Summarising and rephrasing text you give them. Low hallucination risk because the source is right there.
Drafting code, emails, plans, outlines — anything where a 70% starting point saves time even if you edit heavily.
Translation and tone shifting. Modern models are excellent at this.
Explaining concepts in plain language, especially in well-covered domains.
Extracting structured data from messy text into JSON or tables.
Pair-programming and rubber-ducking. Talking through a problem with a competent generalist.

What they are not good at

Exact arithmetic beyond small numbers. Use a calculator tool.
Looking up live facts. Models have a training cutoff; events after it are unknown unless you give them search.
Long-horizon planning that requires consistent state. Models drift over many turns without scaffolding.
Anything where being wrong is expensive and uncheckable. Legal advice, medical diagnoses, irreversible actions — these need a human in the loop.

A tiny example

What calling an LLM looks like in code, conceptually:

# Pseudocode in the style of most LLM SDKs
from some_llm_sdk import Client

client = Client(api_key="...")

response = client.chat(
    model="some-model-name",
    messages=[
        {"role": "system", "content": "You translate English to French."},
        {"role": "user", "content": "Hello, how are you?"},
    ],
)

print(response.text)
# output: Bonjour, comment allez-vous ?

Every major provider’s SDK looks roughly like this. You send messages with roles, you get text back. The interesting work is in the prompt design and how you wrap the call in your application — see Prompt Engineering Basics.

Try it yourself. Pick an LLM you have access to. Write three short prompts that ask the same question different ways — for example, “What is recursion?”, “Explain recursion like I’m a backend developer”, and “Explain recursion in exactly three sentences.” Compare the outputs. Notice how much the framing changes the answer. That observation is the foundation of prompt engineering.

When an LLM is the right tool

Strong fits:

Natural-language interfaces to software — chatbots, search, support.
Content transformation — summarise, translate, reformat.
Code assistance inside an editor.
Augmenting humans on knowledge work where a draft accelerates the human.

Weak fits, where simpler tools win:

Deterministic data transformations — write a Python function or SQL query.
Tasks with hard correctness requirements and no verification step.
High-volume cheap operations where the per-call cost matters more than quality.

LLMs are a tool, not a magic wand. Used in the right place, they replace weeks of work with hours. Used in the wrong place, they produce confident nonsense at scale.

Recap

You now know:

An LLM is a statistical model that predicts the next token of text
Training bakes in the intelligence; inference executes it on frozen weights
The context window is the model’s working memory, measured in tokens
Hallucinations are a structural consequence of plausibility-based generation
LLMs shine at language tasks; they struggle with exact facts and arithmetic
The interesting engineering is in prompts, retrieval, tools, and verification around the model

Next steps

Prompts are the steering wheel. The next post covers what actually works in prompting practice — beyond the folk wisdom.

→ Next: Prompt Engineering Basics

Questions or feedback? Email codeloomdevv@gmail.com.