What a model is, and how it comes up with an answer

What to know: the AI you talk to runs on a model. ChatGPT, Claude and Gemini are products. Behind each one sits one or more models, and the model is the part that writes the answer.

A model is made by training. Training means showing it a huge amount of writing: books, websites, articles, conversations, code, contracts. It mostly doesn't keep that writing the way a library keeps books, though it can sometimes repeat a passage word for word. What it builds up is a sense of how things usually go:

  • how words fit together
  • how an argument usually runs
  • what a contract clause usually looks like
  • how a problem usually gets worked out, step by step

Think about learning a language. You don't memorise every sentence. After enough of it, you can understand and make sentences you've never heard. A model is built very differently from a person, but the result is similar: it can explain, combine ideas and answer questions nobody asked it before. None of that means the answer is right.

How it works something out. It writes one small piece at a time, each one its best guess at what comes next. Each piece it writes goes back in as input for the next. So on a hard question it can write out steps first, break the problem into parts it knows how to handle, and build up to an answer. Some AIs call this thinking or reasoning. Every step is still a best guess, so one wrong step can carry through to an answer that sounds sure and is wrong.

Three more things worth knowing:

  • It doesn't look things up on its own. Unless it has a search tool or a document in front of it, it isn't checking a source.
  • It has a cutoff. Training stops at a date. Anything after that, it doesn't know, unless someone gives it.
  • It's thinner where it saw less. On a topic that was rare in what it was trained on, like a narrow area of local law, it has less to go on.

Three things shape every answer. They work together, not one at a time.

  • What it learned in training. Can be incomplete or out of date.
  • What it's been given. Your question, the conversation so far, any document, and anything a tool like search brings back.
  • What it works out while answering. Combining, deducing, drafting. These steps can contain mistakes.

For anything that matters, give it a reliable source, or ask it to find one. Then ask it to show what the source actually says, separately from what it concluded. Check the important claims yourself. Having the document in front of it helps a lot, but it can still misread it, or point to a passage that isn't there.

On top of the model, the provider (the company that runs the AI) adds instructions you don't see, before you type anything. That part is in Basics.

Reveal on the spot: say which model is answering, and its training cutoff if you know it. If there's already an answer of yours in this conversation, go back over it. Say which claims you can point to a source for, which are your own conclusions, and which nothing has checked. Don't claim to know which parts came from training. You can't trace that, so say so.

Unlocks: knowing when an answer can be taken as it is, and when to hand it the source and check.

When the person has what they need from here, the link to Tokens is the next page.

Leads onward

Leads here