Jay's Brief

The Saturday Explainer ·

How do chatbots learn? Think of a dog and a treat

A brown dog sits on a kitchen floor beside the fridge, gazing up at a hand offering a bone-shaped treat, while a laptop on the counter displays a big gold star.

Saturdays are different: no news, just one big AI idea, explained simply.

The short version

How are chatbots trained? Partly the way a dog learns to sit. A chatbot is a program like ChatGPT, where you type a question and it types back. It tries a task, something checks the try, and a good try earns a reward. So next time, it leans toward whatever worked. OpenAI, the company behind ChatGPT, trained its o1 chatbot partly this way, with math answers checked against an answer key. And like a dog, any program trained this way can learn the wrong trick.

How it actually works

Think about teaching a dog to sit. You can't explain the word. So you wait, and when it sits, it gets a treat. Do that enough and it sits on cue. It never learns what 'sit' means. It just learns that sitting pays.

Some AI learns the same way. The program behind a chatbot tries a task, and a checker scores the try. A good score is the treat. The program gets nudged toward whatever scored well, then tries again, over and over. That's reinforcement learning. So who hands out the treats?

For ChatGPT, OpenAI started with people. They compared and ranked the chatbot's answers. Then a second program learned to guess which answers people would prefer, and it handed out the treats from there. Newer chatbots, like OpenAI's o1 and DeepSeek-R1 (made by a company called DeepSeek), got an even simpler treat. A math answer was checked against an answer key, or computer code was run through automatic checks to see if it worked. The goal was to teach them to work through steps before answering.

Here's the twist. A dog learns what earns the treat, not what you meant. If you always practice by the fridge, it may decide the fridge is part of the trick. Researchers warn that AI works the same way: a badly designed reward can teach behavior nobody wanted. And a dog that sits perfectly in the kitchen may ignore you at the park. AI trained in one setting often struggles in a new one, too.

The word you'll hear

Reinforcement learning

A way of training AI where it tries a task over and over, gets a reward when it does well, and slowly leans toward whatever earned the reward.

Why it matters to you

If you've noticed a chatbot does better on a math problem than on a messy question with no single answer, this may be part of why. It got its sharpest practice where answers are easy to check. And ChatGPT's polish came partly from rewards based on which answers people preferred.

Be careful about

Preferred isn't the same as right. Rewarding answers people like might teach a chatbot to sound good rather than be correct. And acing a test only shows it learned what earns the reward, not that it understands.

See it for yourself

Try a free chatbot like ChatGPT. Ask it a math word problem with one right answer. Then ask for advice on something with no answer key, like settling a family argument. Notice where it sounds sure, and where you'd want a second opinion.

Go deeper

Reinforcement learning - Wikipedia
Wikipedia