How a neural network differs from a regular program

A regular program is a set of instructions. A programmer writes: "if the total is over a thousand, apply a five percent discount." The computer obediently does it. Everything is predictable, and the program won't do anything beyond what's written.

Now try writing instructions for "how to spot a cat in a photo." Triangle ears? So does a fox. Whiskers? So does a seal. The cat's curled up in a ball, hiding behind a curtain, shot from above, and every rule falls apart. You can't solve problems like this with rules. That's where a neural network comes in: nobody explains to it what a cat looks like, they just show it cats.

How a neural network learns

Inside a neural network is a huge number of connections, and each has a "weight": how strongly it affects the answer. At first the weights are random and the network guesses blindly. It's shown a photo, says "dog," and gets told: wrong, that's a cat. The weights get nudged a tiny bit so the error is smaller next time.

Repeat that millions of times and the network starts recognizing cats in photos it has never seen. The number of these adjustable connections is called parameters. Large language models have billions of them, which is why training takes months and costs a fortune.

How a language model works

A language model was trained on mountains of text with one job: given the start of a phrase, guess the next word. "Twinkle, twinkle, little…" "star." Sounds like a game, but to guess well the model had to pick up grammar, facts, reasoning, and even a sense of humor. Being able to answer questions grew out of that as a side effect.

Two things follow from this that are worth remembering:

  1. The answers are different every time. The model picks the next word with a bit of randomness, otherwise it would sound like an answering machine.
  2. The model can make things up. Its job is to write plausible text, and plausible isn't always true. Hence books and quotes that don't exist. More in AI hallucinations.

Treat a neural network's answer like a draft from a sharp intern. Check dates, numbers, and names, and do your math on a calculator: the model predicts text, it doesn't calculate.

Different neural networks for different jobs

  • Language models write and hold conversations.
  • Image generators draw from a description.
  • Speech synthesis models turn text into a voice.
  • Recognition models identify faces, license plates, and handwriting.

These are different networks trained on different data. The only thing they share is the principle: learning from examples. How a neural network differs from AI in general is covered in AI vs. neural networks.

Why a neural network "remembers" the conversation

The model itself remembers nothing between messages. Every time, the service sends it the whole conversation again: your new message plus everything before it. The model reads it all and writes what comes next.

But there's a limit to how much the model can read at once. So services compress long conversations: older messages turn into a short summary, and the key facts about you are stored separately. How that works for AI characters is explained in AI character memory.

So does a neural network think?

No. It has no desires, intentions, or understanding in the human sense: it doesn't want to help you and it doesn't get hurt when you yell at it. But calling it "just autocomplete" isn't fair either. It's a powerful tool with real strengths and clear limits.

Its strengths show best in conversation. A character running on a language model stays in character, jokes in its own style, and remembers what you told it a week ago. Try it yourself: it's clearer than any article.

Talk to a neural network playing a role

Characters from anime, books, and movies, in English or Russian: pick one and see how a language model stays in character.

Find someone to talk to