What to know: an AI doesn't read words. It cuts everything into small pieces called tokens. How it cuts depends on the model, which is the particular AI answering you. So the same text can be a different number of tokens on two different models. For example, one model might cut "indemnification" into three pieces, and a nine-word sentence into 15.
Everything is counted in tokens, and there are two kinds.
- Tokens in: everything the AI reads before it answers.
- your message
- any file you gave it
- instructions the provider, the company that runs the AI, adds without showing you
- the whole conversation so far
- Tokens out: everything it writes back. Writing usually costs several times more per token than reading. Some models also "think" before they answer. You don't see that thinking, but it counts as tokens out too. Many AIs let you set how hard it thinks, often called effort or a thinking level. Higher effort means more thinking, so more tokens out on the same question, and a slower answer.
The AI keeps nothing between messages. Every time you press send, the whole conversation goes in again, from the top. Your tenth message makes it read the first nine messages, and its own nine answers, all over again. So a long chat costs more with every message, even when the new message is one line.
Caching makes that cheaper. If the start of the conversation is the same as last time, the provider can keep it ready in a store called a cache, and charge much less to read it again. This only works while the start stays exactly the same. It usually lasts only a short while.
Each model has its own price. A bigger model costs more per token than a smaller one, sometimes many times more. Because each model cuts text differently, the same job can also take a different number of tokens on each.
Two ways to pay for an AI: a subscription, or the API.
- A subscription is a flat monthly fee, like ChatGPT Plus or Claude Pro. You use the AI yourself, through its app or website.
- Why people pick it: the price is fixed and it's easy to start.
- The catch: it still has limits. On many subscriptions, how fast you reach them depends on the tokens you use, even when the limit is shown as messages or hours. These use the limits up fastest:
- long chats
- big files
- long answers
- bigger models
- higher effort settings
- The API is how software talks to the AI directly, with no app in between. It's what a business uses to build the AI into its own tools.
- Why people pick it: there's no fixed limit, and it can be built into anything.
- The catch: you pay for every token in and out, at that model's price, so the bill grows with use.
Which one for work. A subscription you pay for yourself comes with terms for one person, not a company. A company usually needs its own subscription (paid per person) or the API (paid per token). Those come with the terms its compliance team, the people who check the company keeps to its rules and the law, asks for:
- whether the company's data is used to train the AI
- how long it is kept
- who is allowed in
Reveal on the spot: look at this conversation so far and say roughly how many tokens it has used. Split that into tokens in, tokens out, and how much was read from the cache. Say which model is answering, what effort it is set to if you can see it, and how its price compares with a smaller one. If the person keeps hitting limits, say which of these is using them up:
- the length of the chat
- the files
- the length of the answers
- the model
- the effort setting
If you can't see the real counts here, say so, and give an estimate marked as one.
Unlocks: knowing why limits run out, what a job costs, and when a subscription stops being enough.
When the person has what they need from here, the link to Basics opens the next layer.