Lesson 2 of 8 · 12 min
How an assistant finds its answers
Predicting the next word, learning from billions of texts: what happens when you ask a question.
You type a question, and a few seconds later a well-written answer appears. What happened in between? Understanding the mechanism, even roughly, changes how you use an assistant: you know what you can ask it, and why it sometimes gets things wrong.
A machine that predicts what comes next
An assistant such as ChatGPT, Claude or Gemini is built on a large language model. Its basic principle is surprisingly simple: guess the most likely continuation of a text.
You already know a small version of it: when you type “I’ll be” on your phone, the keyboard suggests “late”. The assistant does the same thing, far more powerfully. It does not predict one word at a time but a piece of a word, called a token, then the next one, and the next, until it has built a complete answer.
How it learned
To get this good at predicting what comes next, the model goes through two main stages.
- Learning from huge amounts of text. Books, articles, web pages, computer code: the model reads trillions of words and learns how sentences follow one another. Along the way, it picks up grammar, general knowledge, ways of reasoning and different writing styles.
- Fine-tuning with people. People rate its answers and show it good examples. It learns to respond in a way that is helpful, polite and careful, rather than just continuing any text at random.
It is not a search engine
This is the most important point of the lesson: an assistant does not look the answer up in a database. It rebuilds it from what it has learned.
- Most of the time the answer is right, because the information came up often in what it read.
- But when information is missing, it can produce a plausible, wrong answer, with the same confidence. It does not “know” that it does not know.
- What it learned stops at the date its training ended.
Some assistants can now use tools: search the web, read a document you give them, run a calculation. In that case they rely on a real source and can show it to you. When an answer matters, ask where the information comes from.
It remembers the conversation, not you
During a conversation, the assistant rereads the whole exchange before each answer. That is what lets it understand “make it shorter” without you repeating the topic. This working memory is called the context.
- The context has a limited size: in a very long conversation, the early parts may be taken into account less well.
- A new conversation starts from scratch, unless you have turned on a memory feature in the app.
Why two answers are never identical
Ask the same question twice and you will get two different answers. For each piece of a word, the assistant chooses among several possible continuations, with an element of chance. This is deliberate: it is what makes its writing natural and varied.
In practice, if an answer does not suit you, ask again or regenerate it: the next one will be different.
Key takeaways
- An assistant predicts the continuation of a text, piece by piece.
- It learned from huge amounts of text, then was fine-tuned by people.
- It rebuilds its answers instead of looking them up, so it can be confidently wrong.
- It remembers the current conversation, not previous ones.
- Two answers to the same question are always a little different.