How a neural network learns: neurons, weights, errors, and the brain analogy

A plain-language guide to neural network training: neurons, weights, datasets, loss, backpropagation, overfitting, generalization, and a CRNN walkthrough.

Published : 21 September 2026

Key takeaways

The model is a mathematical system, not a small brain. It has units loosely inspired by neurons, but the biology is richer, messier, and not copied layer for layer. Use the analogy to get the loop; drop it when it starts sounding like neuroscience.

Weights are where most of the “knowledge” lives. Each connection has a number that strengthens, weakens, or even reverses an incoming signal. Learning is the slow rewriting of those numbers.

The dataset is the experience. A network cannot invent the world it never saw. Variety, correct labels, hard cases, and real conditions matter as much as — often more than — a fashionable architecture.

One look is not training. A forward pass produces a prediction. A loss function scores the miss. Backpropagation asks which parameters are to blame. Gradient descent changes them a little. Then the next example arrives.

High score on the homework is not the goal. Overfitting is memorizing the training set. Generalization is handling a new serial number, a new font, a new photo — the thing you actually need.

A CRNN makes the loop concrete. Convolution finds visual marks, a recurrent layer reads them in order, and CTC lets the model learn text without you boxing every letter by hand.

What neural network learning actually is

People say “the model learned” the way they say a student learned: something that used to fail now succeeds on similar tasks. Inside the machine, that sentence has a narrower meaning. There is a function with millions or billions of adjustable numbers. You show it an input. It emits an output. A formula compares that output with the answer you already know. Then an algorithm nudges the numbers downhill on that error.

That is why a neural network is not given a handbook of rules. It is given experience in the form of labeled examples and a way to feel the size of a mistake. After enough cycles, the same function starts to behave as if it “knew” regularities — the look of a P, the fact that a hyphen often sits between a letter and digits — without anyone writing those regularities down.

The comparison with how the brain works during learning is tempting because both systems change with practice. Synapses in tissue and weights in a file both shift. Feedback matters. One pass is rarely enough. The differences are just as important. A brain is embodied, emotional, sleep-dependent, and full of memory systems that are not a single tensor of floats. An artificial network is an optimization procedure you can pause, copy, and restart. Our AI versus human intelligence longread stays with that architectural gap; this guide stays with the training loop itself.

Neurons, weights, and the limits of the brain analogy

A biological neuron, in one paragraph

A typical neuron collects signals on dendrites, combines them in the soma, and if the result crosses a threshold, sends a spike down the axon toward other cells. The meeting points are synapses. Their strength can rise or fall with use. That picture is already a classroom sketch: real cells have timing, neuromodulators, glia, and network dynamics that no exam diagram captures.

An artificial neuron, in one line of arithmetic

An artificial unit is simpler on purpose:

inputs → multiply by weights → add them up (plus a bias) → activation → output

If three numbers arrive, a cartoon version looks like this:

y = activation(x1*w1 + x2*w2 + x3*w3 + b)

Each weight is a coefficient of influence. A large positive weight amplifies that input. A negative weight can suppress it. The activation — a curve such as a rectifier — stops the whole network from collapsing into one giant linear equation. Stack many units into layers, and you get a neural network.

What you may compare, and what you should not

You may compare signals, connections, and gradual change. You should not map “layer 3” onto the visual cortex, or claim that a weight is a synapse, or that the model has a hippocampus. Hierarchical features in a vision net rhyme with hierarchical processing in vision, but the rhyme is computational, not anatomical. Language in the brain is a distributed network, not a folder per language; a trained model is also a distributed pile of numbers, not a dictionary entry labeled “P.” Both facts warn against neat cartoons.

The dataset is the experience

A person learns a letter by seeing it in many hands, sizes, and lighting conditions, trying, missing, and getting a correction. A model’s version of that life is the dataset: images paired with the text they should yield, or whatever input–output pairs the task needs.

human:  example → attempt → error → feedback → small correction
model:  image   → prediction → loss → weight update

Quality here is not a slogan. Diversity stops the net from treating one font as “what a P is.” Volume gives the optimizer enough chances to move. Correct labels matter because the loss trusts you: if you mark a B as P, the model will diligently learn the wrong lesson. Difficulty matters because a set of only easy, centered, high-contrast shots never prepares the camera phone in a warehouse. Class balance matters because a rare character will be ignored if almost every example is the same digit. Match to reality — domain shift — is the quiet killer: train on synthetic screenshots, deploy on greasy metal plates, and watch accuracy collapse.

A toy contrast for the letter P:

weak set:   P P P P P     (one font, one size, clean)
stronger:   P p P P P     plus fonts, tilts, blur, crops, noise

The weak set can produce a model that looks brilliant in the folder and helpless in the wild. That is not mysterious. It is the same reason a student who only practiced one worksheet struggles on a differently worded exam. Varied, spaced practice helps people generalize; varied, honestly labeled data helps models do the same job.

From a guess to a slightly better model

Training is not a vibe. It is a loop with named parts. A CRNN for reading text in an image is a good running example because you can point at each stage.

Forward pass: the model looks

The image goes in. A convolutional stack (CNN) turns pixels into visual features — edges, strokes, blobs that later become useful for letters. A recurrent stack, often LSTM units, walks those features in order, because text has a left-to-right (or right-to-left) life that a bag of local marks does not capture. A CTC head turns a stream of per-position symbol scores into a string, without demanding that you pre-draw a box on every character.

image → CNN features → LSTM sequence → CTC → text

The human rhyme is crude but serviceable: eyes, visual processing, putting marks in order, recognizing a word. The output of this pass is a prediction — maybe P-102 when the plate says P-103. A single forward pass is inference. It does not, by itself, teach anyone anything. The weights are still whatever they were this morning.

Loss: a number for “how wrong”

The loss function answers a quantitative question: how far is this prediction from the labeled truth? If the truth is P-103 and the model said P-102, the loss is not a scolding paragraph. It is a number. Large usually means “more wrong” for the chosen formula; training tries to drive that number down on average.

Loss is not the same as accuracy. Accuracy is a share of fully correct strings, or correct characters, depending on how you count. Loss can shrink while accuracy stays flat for a while, or spike on a few ugly examples while most items look fine. Different tasks use different losses: CTC for unaligned sequences, other formulas for classification or bounding boxes. The shared idea is only this: without a scalar error, the optimizer has nothing to climb down.

People receive feedback in language, grades, or a raised eyebrow. The model receives a float. Same job, narrower channel.

Backpropagation: who is to blame?

If the answer was wrong, the useful question is not “the model is bad.” It is which parameters, if tweaked, would change the error. Backpropagation sends the loss backward through the CTC head, through the recurrent stack, through the convolutional filters, using the chain rule of calculus so each weight gets a gradient: a local answer to “if I nudge you, what happens to the loss?”

loss
  ↑ CTC
  ↑ LSTM
  ↑ CNN
  ↑ image (unchanged — we do not “learn the photo”)

You need derivatives because the network is a composition of functions. The chain rule is the bookkeeping that makes deep stacks trainable instead of a mystical mess. You do not have to enjoy calculus to use the intuition: credit assignment flows against the arrows of the forward pass.

Gradient descent: a small rewrite of what the model “knows”

Once every weight has a gradient, the update is conceptually one line:

new weight = old weight − learning_rate × gradient

The learning rate is how boldly you step. Too small, and training crawls. Too large, and you leap over useful solutions, with a loss that thrashes. In a person, a single humiliation should not erase years of skill, and a single compliment should not rewrite a profession. The same patience belongs here: error should correct, not overwrite.

After the step, the next image meets a slightly different function. That is the whole engine. Optimizers used in practice (momentum, adaptive rates) are variations on this downhill walk, not a different philosophy.

Epochs, overfitting, and generalization

An epoch is one full pass through the textbook

If you have a thousand labeled images, one epoch means the optimizer has seen all thousand (usually in shuffled mini-batches). Epoch 1 is clumsy. Epoch 10 may already show stable strokes and digits. Epoch 50 may look impressive on the training folder. None of those numbers is magic. What matters is whether new images improve too.

Repeating the textbook helps because each pass meets a model that is no longer random. Patterns that were noise on Monday become usable features on Friday. Infinite repetition does not keep helping. Past a point the network starts to fit the sample, including its quirks, instead of the phenomenon.

Overfitting: the student who memorized the answer key

The human picture is familiar. Someone drills a hundred problems, recites them, and freezes on problem 101. In numbers:

train accuracy  99%
test accuracy   72%

Typical causes: too little data, too little variety, a model with more capacity than the task needs, training that ran too long, or leakage — test images that secretly sat in the training pile, data augmentation that used the test set, labels that encode the split. The remedies rhyme with good teaching: more honest data, augmentation (tilts, crops, noise — varied practice), dropout and other regularizers that stop the net from relying on one fragile cue, early stopping when the validation curve stops improving, and a clean train / validation / test split you do not peek at like an answer key.

This is cousin to the fluency illusion in studying with AI and to cramming that active recall is designed to catch. A smooth training loss feels like mastery. A held-out set is the quiz you did not write yourself.

Generalization: the property that makes the model useful

Generalization means the function behaves well on examples it was not trained on. The CRNN saw P-101 through P-104. Production needs P-237 in a new font, at dusk, with a scratch through the hyphen. Train accuracy cannot tell you that story. Only a validation set drawn from the same messy world you will deploy into can.

Transfer is the human name for a nearby idea: can you use last week’s method on a new chapter? Spacing plus variation helps brains and, in some experiments, artificial nets too — the spacing-and-variation study is worth reading as a sibling, not as a claim that your Anki settings should copy a fruit-fly protocol.

Layers, features, and where “memory” lives

Early convolutional layers tend to respond to simple structure: lines, edges, contrast, corners. Middle layers combine those into parts — a vertical stem plus a bowl that is P-ish. Deeper layers, plus the sequence model, assemble objects and strings. That hierarchy is a fact about this class of functions, repeatedly observed, not a proof that the model contains a little homunculus reading “letter P” in English.

simple marks → combinations → objects / characters → string → output

After training, the weights (and a few related buffers) are the main store of what was learned. Load them, and the behavior returns. Delete them, and the architecture is an empty costume. That is unlike human memory, which is not one file: working memory, episodic scenes, skills, affect, and context all participate, as the brain-during-learning guide stresses. Saying “weights = memory” is a slogan for “the parameters carry the fitted regularities.” It is not a theory of remembering your childhood.

A single weight almost never means a human concept. Useful structure is distributed. That is why you cannot usually open the file, highlight neuron 44821, and read “this is the hyphen detector” — sometimes a unit is unusually selective, often it is not.

CRNN in practice: order, CTC, and one full training step

A CNN is strong at “what is in this patch.” Text also needs “what comes after what.” After P-10, a human already expects a digit, not a random smudge. LSTM (and cousins) carry a summary of the left context while looking at the next slice of features. That is why a convolutional backbone alone is an awkward reader of serial numbers.

CTC exists because pixels do not arrive pre-aligned to characters. You would rather not hand-label the exact column where the 1 starts on every photo. CTC trains the sequence model to emit a string by considering many possible alignments, including blank symbols that eat stretching and pauses. You still need the correct string. You do not need a box on every glyph.

One labeled example, end to end:

Load the image and its string, say P-103.
CNN extracts a feature sequence.
LSTM (or another recurrent stack) contextualizes that sequence.
CTC produces a loss against P-103.
Backpropagation fills gradients for every trained weight.
The optimizer takes a step.
The next image meets a slightly altered network.

After thousands of such steps, random weights become crude detectors, then stabler features, then strings that more often match. The model was never handed the rule. The rule-like behavior is what a well-shaped parameter set does. Individual weights still do not “mean” P. The useful work is smeared across the stack.

Why the dataset often beats a fancier architecture

A large, fashionable network on a narrow, clean, mislabeled set will memorize a performance. A moderate CRNN on photos that look like production — glare, motion, rare characters, honest labels — has a chance to generalize. Architecture still matters: without a sequence head, you fight the task. Without enough capacity, you underfit. But teams routinely overspend on layers and underspend on what the model is allowed to see.

Domain shift is the adult version of the toy P example. The training world and the deployment world must overlap. If night photos, wide-angle distortion, or a new printer font never appear in training or augmentation, the loss on the old folder will not warn you. Error analysis — actually looking at the misses — then beats another twenty epochs.

The HiddenLogic rhyme is cognitive load and materials design: a brilliant explanation of a concept you never practice in exam conditions is the student version of a beautiful architecture on the wrong data.

What the analogy teaches about real training — and about studying

In a recognition project the loop is ordinary:

real photos → dataset → CRNN → error → weight update → next epoch → validation → change something

The “something” is a short menu, not a mystery: grow and clean the set, augment, train longer or stop earlier, change the learning rate, change the architecture, fix labels, read the failures. Each item maps onto a human study choice. More varied examples are more varied practice. Augmentation is interleaving of conditions. Extra epochs without a validation check are rereading the same notes until they feel true. A learning-rate disaster is the all-nighter that overwrites yesterday’s understanding. Label fixes are catching a wrong flashcard before you drill it for a month.

The analogy also protects you from a marketing story: that a chatbot “understands” because the prose is smooth. A trained net has statistical regularities. It does not have a Tuesday. When you use models as study tools, keep retrieval on your side of the keyboard — the study-with-AI guide is the practical companion. When a headline says a model has a brain, keep claim, evidence, and interpretation apart; that is the habit ThinkLens is for.

Common misconceptions

“It learned the rule for P”

It learned parameters that, on this distribution, tend to emit P in P-like places. That is not a verbal rule you could print and hand to a child.

“This layer is the visual cortex”

It is a stack of convolutions. Similar jobs, different hardware, different training history.

“Train accuracy is the score that matters”

It is the score on the homework you already assigned. Production is the unseen test.

“More epochs are always better”

Until they are not. Watch validation. Stop. Or regularize. Or get better data.

“A bigger model will save a bad set”

It may overfit faster. Capacity is not wisdom.

“CTC needs a box on every letter”

The point of CTC is that you usually do not.

“Weights are human memory”

They are the fitted parameters. Human memory is several systems, not a checkpoint file.

FAQ

How does a neural network learn?

It adjusts weights to reduce a measured error on labeled examples. After many such updates, predictions on similar new examples often improve. That is training, not a teacher dictating rules.

The loop is always some version of: input, prediction, loss, backward pass, optimizer step. Architecture changes the internals of the prediction; it does not replace the loop.

Is a neural network the same as a brain?

No. It is a mathematical model with a loose biological inspiration. Shared vocabulary — neuron, learning, memory — hides different mechanisms.

Brains have embodiment, sleep, emotion, and multiple memory systems. Networks have tensors, a loss, and an optimizer. Analogies help beginners; they should not drive product claims.

What is a weight in a neural network?

A number that scales how much one signal affects a unit. Learning changes those numbers. After training, the collection of weights is the main store of fitted behavior.

Positive and negative weights both matter. A weight is almost never a clean label like “this is the letter P.”

What is backpropagation?

An efficient way to compute how each parameter affected the loss, by sending error information backward through the layers. It uses the chain rule.

Without some form of credit assignment, you would not know whether to raise or lower a given weight. Backprop is that bookkeeping for differentiable stacks.

What is a loss function?

A formula that turns “wrong” into a number the optimizer can minimize. It is not the same as accuracy, which counts successes.

CTC is one such formula for sequences without per-character alignment. Other tasks use other formulas. The job is always: quantify the miss.

What is an epoch?

One pass through the full training set. Several epochs are normal; endless epochs risk overfitting.

You judge progress on a validation split you do not train on. That split is the honest quiz.

What is overfitting?

Fitting the training sample so tightly that new examples suffer. High train accuracy with weak test accuracy is the classic signature.

Causes include small or narrow data, excessive training, leakage, and unused capacity. Fixes start with data and regularization, not with wishful extra epochs.

What is generalization?

Doing well on examples the model did not train on, drawn from the world you care about. That is the actual product requirement.

A model that only recites P-101…P-104 has not learned to read plates. A student who only recites last year’s questions has the same problem.

How does a CRNN read text in an image?

A convolutional stack extracts visual features, a recurrent stack models order, and CTC maps the sequence to a string without requiring boxed letters.

That design matches the job: marks in space, plus a sequence, plus a label that is a string rather than a pixel mask.

Why do people say data matters more than architecture?

Because the model can only extract regularities that exist in the experience you gave it. A clever stack on the wrong photos still fails in production.

Architecture still has to be able to represent the task. Data decides whether that ability is used for the real world or for the folder.

What is a learning rate?

The step size in the weight update. Too small is slow; too large is unstable. It is a hyperparameter you choose, not something the net “understands.”

In the human analogy, it is how violently one mistake is allowed to rewrite prior skill. Moderate usually wins.

Does this mean my brain trains with backpropagation?

Not in any simple sense. Some theories borrow gradient-like ideas; the evidence for literal backprop in cortex is not a settled engineering fact.

Use the analogy for intuition about practice and feedback. Do not treat a training script as a map of your neurons.

Conclusion

A neural network does not receive a speech about letters. It receives examples, emits a guess, pays a numeric penalty, and rewrites its parameters a little. Repeat that enough times, with experience that looks like the real task, and behavior that looks like a rule can appear. The brain analogy earns its keep as a picture of practice and feedback. It fails as anatomy.

If you train models, pick one ugly real image this week, watch a single step of the loop, then look at the misses on a held-out set before you add another layer. If you only use models, keep the same discipline: a fluent answer is a forward pass, not proof of understanding — yours or the machine’s.