Understanding AILesson 2/8

Lesson 2 of 8 · 12 min

How an assistant finds its answers

Predicting the next word, learning from billions of texts: what happens when you ask a question.

You type a question, and a few seconds later a well-written answer appears. What happened in between? Understanding the mechanism, even roughly, changes how you use an assistant: you know what you can ask it, and why it sometimes gets things wrong.

A machine that predicts what comes next

An assistant such as ChatGPT, Claude or Gemini is built on a large language model. Its basic principle is surprisingly simple: guess the most likely continuation of a text.

You already know a small version of it: when you type “I’ll be” on your phone, the keyboard suggests “late”. The assistant does the same thing, far more powerfully. It does not predict one word at a time but a piece of a word, called a token, then the next one, and the next, until it has built a complete answer.

How it learned

To get this good at predicting what comes next, the model goes through two main stages.

  1. Learning from huge amounts of text. Books, articles, web pages, computer code: the model reads trillions of words and learns how sentences follow one another. Along the way, it picks up grammar, general knowledge, ways of reasoning and different writing styles.
  2. Fine-tuning with people. People rate its answers and show it good examples. It learns to respond in a way that is helpful, polite and careful, rather than just continuing any text at random.

It is not a search engine

This is the most important point of the lesson: an assistant does not look the answer up in a database. It rebuilds it from what it has learned.

  • Most of the time the answer is right, because the information came up often in what it read.
  • But when information is missing, it can produce a plausible, wrong answer, with the same confidence. It does not “know” that it does not know.
  • What it learned stops at the date its training ended.

Some assistants can now use tools: search the web, read a document you give them, run a calculation. In that case they rely on a real source and can show it to you. When an answer matters, ask where the information comes from.

It remembers the conversation, not you

During a conversation, the assistant rereads the whole exchange before each answer. That is what lets it understand “make it shorter” without you repeating the topic. This working memory is called the context.

  • The context has a limited size: in a very long conversation, the early parts may be taken into account less well.
  • A new conversation starts from scratch, unless you have turned on a memory feature in the app.

Why two answers are never identical

Ask the same question twice and you will get two different answers. For each piece of a word, the assistant chooses among several possible continuations, with an element of chance. This is deliberate: it is what makes its writing natural and varied.

In practice, if an answer does not suit you, ask again or regenerate it: the next one will be different.

Key takeaways

  • An assistant predicts the continuation of a text, piece by piece.
  • It learned from huge amounts of text, then was fine-tuned by people.
  • It rebuilds its answers instead of looking them up, so it can be confidently wrong.
  • It remembers the current conversation, not previous ones.
  • Two answers to the same question are always a little different.

Quiz

1. What is the basic principle of an assistant like ChatGPT or Claude?
2. Why can an assistant state something false with confidence?
3. You open a new conversation. What does the assistant know about the previous one?
4. You ask the same question twice. What happens?
Next