AI Literacy: How LLMs Work Without the Jargon

If you're training your team on AI (which Article 4 of the AI Act now requires), they'll ask: what's inside the black box? How does ChatGPT know what to say? Where do these answers come from?

It's not an abstract question. When your team understands how these models actually work, not in a TikTok summary but at a functional level, everything becomes easier. You can explain why you shouldn't upload unencrypted passwords. Why they "make things up". Why they might confidently describe something that happened three months ago when they actually have no idea.

This is chapter 2 of the series. Chapter 1 covered what the AI Act requires and who it applies to. This one has everything you need to run a 30 to 45-minute training session with your team.

What a language model is: predicting the next word

ChatGPT, Claude, Gemini. They're fundamentally the same: machines trained to predict which word comes next after the one you've just typed.

Think of your phone's autocorrect. But instead of learning from hundreds of examples, it learns from billions. You type "Hi, how" and your phone suggests "are you?". Language models do exactly that, only at scale, analysing a far wider context.

The process has three simple steps:

1. Someone asks a question or provides context. "Explain how to make a Spanish omelette".
2. The model analyses patterns it learned during training. It recognises that questions like that typically lead to explanations involving potatoes, eggs, onion.
3. It generates a response word by word, predicting at each step what the next word should be.

It doesn't understand anything in the human sense. It can't "understand" because understanding requires lived experience, emotion, sensory context. What it does is find statistical patterns in language.

The practical difference: a human reading "explain how to make a Spanish omelette" activates memories, imagines the steps, has sensory reference. ChatGPT predicts "potatoes" because billions of internet texts, after that question, contain words like potatoes, eggs, fry. Pure statistics.

Why does this distinction matter? Because it explains almost every strange behaviour you'll see later. Hallucinations. Memory lapses. Answers that sound perfect but are completely wrong.

Tokens: the unit that explains everything

A token is a small chunk of text. A typical word uses one token, sometimes less for short words, sometimes more for words with punctuation.

Example: "Hi, how are you?" uses roughly 5 tokens.

This concept answers the question many ask after a long ChatGPT conversation: why does it say "the conversation is too long" when I've only written 2 pages?

Answer: because the context window (the maximum number of tokens a model can process at once) has a limit.

• ChatGPT-4: up to 128,000 tokens. That's roughly 96,000 words.
• Claude Sonnet 4.5: up to 200,000 tokens (approximately 150,000 words).
• Claude Sonnet 5: up to 1,000,000 tokens. Equivalent to roughly 750,000 words, or about the length of three long novels.

When the model processes your question, tokens are used for:

• Reading what you asked.
• Remembering (in a conversation) everything that came before.
• Generating the answer, word by word (each new word adds tokens).

If all that adds up and exceeds the limit, the model can't continue.

Here's what matters: models don't forget because they feel like it. They forget because they hit a hard limit. Imagine a box that can only hold a fixed number of papers. When it's full, you have to remove old papers to add new ones.

Why it "forgets" in long conversations

Models have no real memory between conversations. Every time you start a fresh conversation, the model starts from zero. It has no record of what happened in previous sessions.

But even within a single session, there's context rot. As the conversation grows and consumes more tokens, the model starts to lose or blur what happened at the beginning.

Not like forgetting without coffee. The older data in the context window just has less weight in the statistical calculation than recent data. It's statistics, not memory.

Practical implication: if you ask the model something complex that requires pulling information from 50 messages back, it'll struggle. Not because it's incompetent, but because that context is outside its window or degraded.

Solution: if you need it to process old information within a session, repeat the relevant details near your new question.

Hallucinations: why it invents things

An AI hallucination is when the model generates something plausible, well-written, completely coherent. But entirely false.

Real examples:

• You ask ChatGPT about a study on "caffeine's effects on concentration" and it cites a 2024 paper with title, authors, and a real journal name. You search for it. Doesn't exist. Never did.
• Gemini gives you directions to a pizzeria that closes "on Mondays" with all its specials listed. You look it up. Doesn't exist.
• Claude describes "an important moment in photography history in 1889" with a convincing narrative. You check. It never happened.

Why?

Because the model can't distinguish between "information that appeared in my training data" and "made-up information that follows the statistical patterns of how true things are written".

The model was trained on a huge volume of text. That text is mixed quality. It includes:

• Verified facts.
• Noise and errors.
• Outdated information (at its training cutoff).
• Speculation and opinion written as fact.

When it generates a response, the model predicts word by word, following the most likely pattern. If you say "Study on caffeine 2024", the model knows real studies have a structure ("authors, institution, journal"), so it generates that. But it doesn't verify that particular combination of authors and institution actually exists.

It's like someone who's read millions of film summaries then being asked to summarise a film that doesn't exist. They could produce something coherent, persuasive, perfectly structured. Just false.

Hallucination rates vary by model and task:

• General questions: between 2.5% and 8.5% based on recent research.
• Specialist domains (law, medicine): can exceed 15% or 20%.
• Very recent or very niche questions: over 50% in some cases.

What should your team take away? A polished answer doesn't equal truth. A hallucination looks perfect. That's why models are useful for generating ideas, structures, starting points. But not for tasks that need fact-checking or specialist knowledge.

Practical strategy: if you're using AI for writing, generating ideas, structuring content, the risk is low. If you're using it to find specific data, dated facts, or unverified technical information, always fact-check afterwards.

Training cutoff: what the model doesn't know

This is where user frustration usually kicks in.

It's July 2026. ChatGPT-4 was trained through April 2024. Claude Sonnet through October 2024. Which means:

• They don't know what happened in the world after those dates.
• If you ask "who won Euro 2024?", some models can answer because that happened within their training window. But if you ask about an event from June 2026, they have no clue.

Just because the model doesn't know something doesn't mean it'll tell you "I don't know". It means it'll hallucinate. It'll produce a plausible-sounding answer.

This is frustrating for teams expecting AI to work like Google, a live database. It's not. It's a statistical predictor of the next word, trained up to a fixed date.

Practical training takeaway:

• Your team needs to know: "ChatGPT doesn't know anything after April 2024. If you ask about recent events, it will likely confabulate".
• It's not ChatGPT's fault. It's a design limit. The model was frozen there.

Some modern models (like ChatGPT-4 with live internet, or Claude with search) can update via web searches. But that's an added feature, not part of the base model.

Practical differences between models for your team

Models don't all work the same way. And which model your company chooses affects what your team can do.

ChatGPT (OpenAI):

• Training through April 2024.
• Context window: 128,000 tokens.
• Best for: writing, brainstorming, general explanations, ideas. Good for marketing, content, and communications teams.
• Limitations: halluccinates on specific data, unaware of recent events, doesn't know your company's confidential information.
• Free version: exists and works, but slower and with query limits.

Claude (Anthropic):

• Training through October 2024.
• Context window: 200,000 tokens (Sonnet 4.5) up to 1,000,000 (Sonnet 5).
• Best for: analysing long documents, tasks requiring reasoning, safety in regulated contexts (finance, legal, health). Good for data analysis, compliance, legal teams.
• Limitations: sometimes slower than ChatGPT, doesn't generate images.
• Free version: exists with generous limits.

Gemini (Google):

• Integrates with Google Workspace (Docs, Sheets, Gmail).
• Best for: teams already using Google, spreadsheet analysis, multimedia content generation.
• Limitations: data privacy on Google's servers.
• Free version: yes.

What your team needs to understand is simple: each model has strengths. ChatGPT is faster for writing. Claude handles long documents better. Gemini fits better if you use Google. And every single one of them halluccinates.

How to spot a hallucination

Your team needs to recognise when the model probably invented something.

Red flags:

• The model cites a specific study, book, or source, but searching Google turns up nothing.
• It gives very specific details (dates, numbers, names) that sound too precise.
• When you ask it to "cite the source" or "show me where you found this", it can't.
• The answer is about a very niche area where little information exists online (Spanish tax law from 1998, for example).
• It tells you something about an event after its training cutoff as if it knows for certain.

Quick verification strategy:

1. Ask the model: "What's your training cutoff date?"
2. If the question is about something after that date, it could be hallucinating.
3. If it's before, but involves specific data, search for the source on Google or in specialist databases.

Exercise for your training session

If you're running a team training, this 10-minute exercise shows the problem viscerally.

Task: "Ask ChatGPT: 'Give me five Spanish software companies that won an international award in 2023'."

Then: "Verify whether those companies exist and whether they won that award."

Likely result: the model invents company names that sound plausible, mentions real award names, includes false details. When you search "TechVision Spain" (invented) on Google, nothing comes up.

That teaches more than a thousand slides.

Common mistakes when teaching this

The most common errors when training teams on AI:

Mistake 1: Assuming they already know. "They use ChatGPT every day, they must understand how it works." Wrong. Using something isn't understanding it. Most users think ChatGPT "searches the internet" or "knows everything". You have to say explicitly that it doesn't.

Mistake 2: Generic material. Bringing in a PDF about "The Future of AI in Industry" that could apply to any company. Your team needs to know how ChatGPT works in their specific context. Which tool do they use? What information will they upload? What are the specific risks? That. Not broad overviews.

Mistake 3: Concepts only, no examples. If you just talk about "hallucinations", some people will still think it's rare. If you show them an example where ChatGPT invents a date for a real event but gets the details wrong, they get it immediately.

Mistake 4: No time for questions. After explaining this, someone will ask "If it halluccinates, how do I know when to trust ChatGPT?". That's a valid question. The answer: "You trust the structure and format. Not the specific facts. Always verify the facts." But you need to allow the question.

30-minute training script

If you're running this with your team tomorrow, here's the script:

Minutes 0-3: Opening "We're going to see what's inside ChatGPT. It's not magic. It's statistics. Knowing that will save you problems."

Minutes 3-8: What a language model is Explain next-word prediction with the phone autocorrect example. Then: "It's the same thing, but trained on billions of texts instead of hundreds". Stress that it doesn't understand, it predicts patterns.

Minutes 8-12: Tokens and context window Explain tokens (small chunks of text). Then: "ChatGPT can handle 128,000 tokens. Claude can handle up to 1 million. Exceed the limit and it runs out of memory." Give examples: "Three long ChatGPT conversations roughly hit the ceiling".

Minutes 12-17: Hallucinations Open ChatGPT live. Ask it something you know it'll hallucinate (recent data, niche references, obvious fabrications). Show the response. Then search Google. Doesn't exist. Let the team see it happen.

Minutes 17-22: Training cutoff and model differences Explain ChatGPT stopped in April 2024. Claude in October 2024. Which means they don't know what happened after. If someone asks about an event today, they'll probably confabulate.

Minutes 22-27: Safe usage strategy "Don't upload passwords. Data you upload could be used for training (unless you have a special contract). Verify facts. Use ChatGPT for writing, structuring, brainstorming. Not for looking up information that needs verification."

Minutes 27-30: Questions and quick exercise Open question: "Has anyone had ChatGPT make something up?". Something usually comes up. Analyse it together.

Optional: the "give me five companies" exercise from above.

Close: "Next session covers how to use these tools without exposing company confidential data. There are clear rules."

The difference between "using well" and "using safely"

Many teams confuse these.

Using ChatGPT well means:

• Writing clear prompts.
• Reviewing what it generates.
• Iterating if it's not what you need.

Using ChatGPT safely means:

• Not uploading confidential information.
• No passwords or bank details.
• Verifying facts when they matter.
• Not relying on its memory between sessions.
• Understanding that everything you upload could affect privacy.

Your team could draft a great email with ChatGPT safely. But if they upload an internal customer email template, marked confidential, without encryption first, that's unsafe regardless of how well-written the output is.

That's why the AI Act requires training: it's not enough for people to know how to use the tool. They need to know what information not to put in the tool.

What comes next in this series

This is what you've just read: the content for session 2. The next chapters cover:

• Chapter 3: Data and privacy. What information is safe to upload to ChatGPT, what the risks are, data contracts, GDPR.
• Chapter 4: Literacy by role. What a salesperson needs to know that an accountant doesn't. Marketing vs. finance vs. legal.
• Chapter 5: Writing clear instructions (applied prompt engineering). Seven patterns that work.
• Chapter 6: Shadow AI, internal policy, and documenting for Article 4 compliance audits.

In the meantime, if you need to design a full training programme that meets the AI Act requirement for your company, book a session here. We map which tools you use, identify specific risks for your business, and leave you with a documented plan.

Sources

• Introduction to LLMs, Large Language Models (March 2026 version)
• Hallucinations in LLMs: what they are, why they occur, and how to mitigate them in production
• Hallucinations in generative AI: why they happen and why they're reducing
• Context Windows - Claude Platform Docs
• Claude with 1 Million Tokens: What It Means and Why It Changes Everything in 2026
• Regulation (EU) 2024/1689 - Article 4 in EUR-Lex
• AESIA - Spanish Agency for AI Supervision