Neural Networks: How the Brain Inspired Computers

The Canteen Decision: Weighing Evidence in a Split Second

The recess bell rings. You have exactly eight minutes before your Computer Science period starts, and you're standing near the canteen. Should you run in and buy something, or wait for the next break? You don't sit down and work through this like a maths problem. In well under a second, your brain has already weighed how hungry you are, whether the queue looks long, whether you have coins in your pocket, and how strict your CS teacher is about latecomers — and you're already moving, one way or the other.

Notice what your brain did not do. It did not check "am I hungry?" first, finish that check, then move to "is the queue long?" one at a time like a list of if-statements running in sequence. Instead, thousands of neurons fired at once, each contributing a small amount of electrical evidence, and your decision emerged from all of that evidence being combined simultaneously. This "combine many weighted clues at once, then decide" trick is precisely what a group of mathematicians and engineers tried to copy when they built the first artificial neural networks — not to simulate a brain exactly, but to borrow its core computational idea. That is the subject of this chapter: how a real neuron works, how engineers turned that idea into arithmetic, and why stacking many of these simple arithmetic units together lets a computer solve problems a single one cannot.

Inside a Real Neuron, Briefly

A biological neuron has three important parts for our purposes. Dendrites are branching fibres that receive incoming signals from other neurons through junctions called synapses. Each synapse has its own strength — some connections pass on a strong signal, others barely register — built up through experience. The cell body (soma) collects all these incoming electrical signals and adds them up. The neuron's resting electrical state sits at roughly −70 millivolts; if the combined incoming signal pushes that value up past a threshold of roughly −55 millivolts, the neuron "fires": it sends a sharp electrical pulse called an action potential down its axon to the synapses connecting it to the next set of neurons. If the combined signal doesn't cross the threshold, nothing happens — the neuron stays silent.

Two ideas from this biological description turn out to matter enormously for computer science: (1) a neuron sums up multiple incoming signals of varying strength, and (2) it only "fires" once that sum crosses a threshold. In 1943, the neurophysiologist Warren McCulloch and the logician Walter Pitts published the first mathematical model that captured exactly these two ideas in arithmetic. In 1958, Frank Rosenblatt built on this to create the Perceptron, an actual machine (not just a mathematical idea) that could learn to classify simple patterns. Everything in this chapter grows out of their work.

Building an Artificial Neuron

An artificial neuron is a deliberate simplification. It does not model chemistry, timing, or the shape of dendrites. It keeps only the two ideas above and turns them into arithmetic:

• Inputs (x₁, x₂, x₃, …) — numbers representing the evidence coming in. In our canteen example, we can let each input be 1 if a condition is true and 0 if it's false: "Am I hungry?", "Is the queue long?", "Do I have money?"
• Weights (w₁, w₂, w₃, …) — numbers representing how much each input matters, playing the role of synapse strength. A positive weight means that input pushes the neuron toward firing; a negative weight means it pushes the neuron away from firing; a weight near zero means that input barely matters.
• Bias (b) — a single extra number representing the neuron's built-in tendency, independent of any input — how eager or reluctant it is to fire by default, before any evidence arrives.
• Weighted sum (z) — add up every input multiplied by its weight, then add the bias: z = w₁x₁ + w₂x₂ + w₃x₃ + … + b. This single number plays the role of "how much combined electrical signal has arrived at the soma."
• Activation function (f) — a rule that turns z into the neuron's actual output, playing the role of the firing threshold.

The diagram below shows all five pieces connected together for our canteen example, with the actual weights we're about to compute with.

Worked Example: Should You Go to the Canteen Right Now?

Let's assign real numbers. Suppose experience has taught you these weights: hunger matters a lot (w₁ = 3), a long queue matters a fair amount but in the discouraging direction (w₂ = −2), and having money matters moderately (w₃ = 2). Your baseline reluctance to break away from friends is a bias of b = −2. Today: you are hungry (x₁ = 1), the queue looks long (x₂ = 1), and you do have money (x₃ = 1).

Compute the weighted sum step by step:

z = w₁x₁ + w₂x₂ + w₃x₃ + b = (3×1) + (−2×1) + (2×1) + (−2) = 3 − 2 + 2 − 2 = 1

Now apply a step activation function, the simplest possible choice, matching the "fire or don't fire" nature of a real neuron: output 1 (go now) if z is greater than 0, otherwise output 0 (wait). Since z = 1, which is greater than 0, the neuron fires: go to the canteen now. Notice that even though the queue being long pulled the sum down by 2, hunger and having money together pulled it up by enough to win.

Here is the same computation as a short, traceable program:

def neuron(inputs, weights, bias):
    total = bias
    for x, w in zip(inputs, weights):
        total += x * w
    return 1 if total > 0 else 0

x = [1, 1, 1]        # hungry=1, queue_long=1, have_money=1
w = [3, -2, 2]
b = -2

print(neuron(x, w, b))   # -> 1  (go to the canteen)

Trace it by hand: total starts at the bias, −2. The loop adds 1*3 = 3 (total becomes 1), then 1*-2 = -2 (total becomes −1), then 1*2 = 2 (total becomes 1). The final total is 1, which is greater than 0, so the function returns 1 — matching our hand computation exactly.

Now change one fact: suppose you have no money today (x₃ = 0). Re-run the sum: z = (3×1) + (−2×1) + (2×0) + (−2) = 3 − 2 + 0 − 2 = −1. Since −1 is not greater than 0, the neuron does not fire: you wait. This is the whole point of weights and bias — they encode, in pure arithmetic, exactly how much each piece of evidence should count, and changing even one input can flip the decision once the sum crosses the threshold in the other direction.

From On/Off to Smooth Decisions: Activation Functions

The step function above is the most literal translation of "a real neuron either fires or it doesn't." But it has a practical problem for building larger, learnable systems: it is flat everywhere except for one sudden jump, so a tiny change in the weighted sum z almost never changes the output, and exactly at the jump the output changes by a huge amount instantly. This makes it very hard to answer the question "how should I adjust my weights slightly to do a little better next time?" — a question we will return to shortly.

Modern networks usually replace the step function with a smoother one. Two common choices, both of which you can compute directly from arithmetic and simple algebra:

• ReLU (Rectified Linear Unit): f(z) = max(0, z). If z is negative, output 0; if z is positive, output z itself. For example, f(−5) = 0 and f(3) = 3. It is simple, fast to compute, and is the default choice inside the hidden layers of most modern networks.
• Sigmoid: f(z) = 1 / (1 + e^−z), which squashes any real number into a smooth curve between 0 and 1, useful when you want the output to read like a probability. At z = 0, sigmoid gives exactly 0.5; for large positive z it creeps toward 1; for large negative z it creeps toward 0 — but unlike the step function, it never jumps, it slides.

The graph below plots the step function against the sigmoid curve for z ranging from −4 to 4, using the actual computed sigmoid values (for instance, sigmoid(1) ≈ 0.731, sigmoid(2) ≈ 0.881, sigmoid(−2) ≈ 0.119). Notice how the step function is a single instantaneous cliff, while the sigmoid rises gradually — that gradual rise is exactly what makes learning-by-small-adjustments possible.

Why One Neuron Isn't Enough: The Two-Way Switch Problem

Many Indian homes wire the staircase or hallway light with two switches — one at the bottom of the stairs, one at the top — using what electricians call a two-way switch circuit. Toggling either switch changes whether the light is on, and the wiring is arranged so that the light is ON only when the two switches are in different positions (one up, one down) and OFF when they match (both up or both down). If we call the switches A and B, with 1 meaning "up" and 0 meaning "down," the truth table looks like this:

• A = 0, B = 0 → Light = 0 (OFF)
• A = 0, B = 1 → Light = 1 (ON)
• A = 1, B = 0 → Light = 1 (ON)
• A = 1, B = 1 → Light = 0 (OFF)

In logic, this pattern is called XOR ("exclusive or"). Can a single artificial neuron compute it? A single neuron with two inputs draws a decision boundary that is always a single straight line across the A–B plane — everything on one side of the line outputs 0, everything on the other side outputs 1, because the weighted sum w₁A + w₂B + b is a linear expression. Plot the four points: the two ON points, (0,1) and (1,0), sit on one diagonal; the two OFF points, (0,0) and (1,1), sit on the other diagonal, forming an X pattern.

Try the dashed vertical line shown at A = 0.5, one natural attempt at a boundary. To its left sit (0,0) [OFF] and (0,1) [ON] — mixed. To its right sit (1,0) [ON] and (1,1) [OFF] — also mixed. Every straight line you try will have the same problem, because the ON points and OFF points alternate around the square. This is not a failure of imagination; it is a mathematical fact about straight lines, and it was proved rigorously by Marvin Minsky and Seymour Papert in their 1969 book Perceptrons. Their proof that a single-layer perceptron cannot compute XOR contributed to a sharp drop in funding and interest in neural network research for over a decade, a period historians of AI call the first "AI winter."

Stacking Neurons: How Hidden Layers Solve What One Neuron Cannot

The way out is not a cleverer single neuron — it is more than one neuron, arranged in layers. An input layer simply holds the raw inputs. A hidden layer contains neurons that each compute their own weighted sum and activation from the inputs, producing new intermediate values. An output layer takes those intermediate values as its own inputs and computes the final answer. Because each hidden neuron still draws a straight line, but a different one, and the output neuron combines their results, the network as a whole can carve out boundaries far more complex than any single line.

Here is a complete, verifiable construction that solves XOR using three step-activation neurons in two layers. Use the fact that XOR(A, B) is logically equivalent to (A OR B) AND NOT(A AND B) — light is on when at least one switch is up, but not when both are up.

• Hidden neuron h₁ computes OR: weights (1, 1), bias −0.5. Check all four cases: (0,0) → z = −0.5 → 0. (0,1) → z = 0.5 → 1. (1,0) → z = 0.5 → 1. (1,1) → z = 1.5 → 1. That is exactly the OR truth table.
• Hidden neuron h₂ computes NAND (NOT AND): weights (−1, −1), bias 1.5. Check: (0,0) → z = 1.5 → 1. (0,1) → z = 0.5 → 1. (1,0) → z = 0.5 → 1. (1,1) → z = −0.5 → 0. That is exactly the NAND truth table.
• Output neuron computes AND of h₁ and h₂: weights (1, 1), bias −1.5.

Now trace the full two-layer network on all four original inputs. For A=0, B=0: h₁=0, h₂=1, so the output neuron sees (0,1): z = 0+1−1.5 = −0.5 → 0. Correct, light OFF. For A=0, B=1: h₁=1, h₂=1, output sees (1,1): z = 1+1−1.5 = 0.5 → 1. Correct, light ON. For A=1, B=0: by the same symmetric reasoning, h₁=1, h₂=1, output = 1. Correct, light ON. For A=1, B=1: h₁=1, h₂=0, output sees (1,0): z = 1+0−1.5 = −0.5 → 0. Correct, light OFF. All four outputs match the XOR truth table exactly. No single neuron could do this; two neurons cooperating in a hidden layer could.

This is the general principle behind every neural network you will encounter, from a simple perceptron to the models used for image recognition or language translation: individual neurons are limited to straight-line (linear) decisions, but layering neurons so that later ones combine the outputs of earlier ones lets the network approximate boundaries of essentially any shape. The word "deep" in "deep learning" simply refers to networks with many hidden layers stacked this way.

How a Network Learns: A Preview

In every example above, we handed you the weights and biases already worked out. In real applications, nobody sits down and hand-derives weights for a network with millions of connections. Instead, engineers let the network learn them. The process works roughly like this: start with random weights and biases; feed the network an example whose correct answer you already know; compare the network's output to the correct answer to measure the error; then nudge every weight a small amount in whichever direction would have reduced that error. Repeat this over thousands or millions of examples, and the weights gradually settle into values that make correct predictions — possibly values very close to the OR/NAND/AND weights we derived by hand above, discovered automatically instead of by a human reasoning it out.

This nudging process is called gradient descent, and the technique for efficiently computing how much to nudge each weight in a multi-layer network is called backpropagation, popularised in a 1986 paper by David Rumelhart, Geoffrey Hinton, and Ronald Williams. Both rely on calculus you will study in later grades; the smooth activation functions from earlier in this chapter (sigmoid, ReLU) exist specifically because gradient descent needs a smoothly changing output to know which direction "a little better" lies in — something the all-or-nothing step function cannot provide.

Common Misconception: "An Artificial Neuron Is Basically a Tiny Brain Cell"

It is tempting to think that because the vocabulary is borrowed — "neuron," "fire," "synapse," "weight" — an artificial neural network is a working miniature brain. It is not, and the gap matters. A biological neuron is an electrochemical system: it fires using ion channels and neurotransmitters, its timing (exactly when it fires relative to other neurons) carries information, and a single neuron in your brain may have thousands of synaptic connections whose strengths change continuously through complex biological processes that are still not fully understood. The human brain contains roughly 86 billion neurons, each such a system.

An artificial neuron, by contrast, is nothing but the arithmetic expression f(w₁x₁ + w₂x₂ + … + b). There is no timing, no chemistry, no continuous adaptation happening inside a single computation — just a sum and a function applied to it. The "inspired by the brain" framing is historically accurate (McCulloch, Pitts, and Rosenblatt were explicitly trying to capture the sum-then-threshold behaviour of real neurons) and it is a genuinely useful design intuition, but it describes where the idea came from, not what the artificial version has become. Treating "neural network" as a literal claim about brain simulation leads students to badly overestimate what these systems understand, and to underestimate how much of their power comes from careful engineering of layers, weights, and training procedures rather than from any resemblance to biology.

Where This Shows Up Around You

The core mechanism in this chapter — combine several weighted pieces of evidence, pass the sum through an activation, let hidden layers capture non-linear patterns — is the same mechanism used far beyond canteen decisions. When a phone unlocks by recognising your face, layers of neurons are weighing patterns of pixel brightness the same way our hidden neurons weighed switch positions, just with millions of inputs instead of three. When a bank or a UPI payment app flags a transaction as potentially fraudulent, it is typically because a model has computed a weighted combination of signals — transaction amount, how it compares to your usual spending, how far the payment location is from your last one, time of day — and that weighted sum crossed a threshold, exactly like our canteen neuron crossing zero. The arithmetic you traced by hand in this chapter is not a simplified toy version of what production systems do; it is literally the same operation, repeated across many more neurons and many more layers.

Check Your Understanding

Q1. A neuron has inputs x₁=1, x₂=0, x₃=1 with weights w₁=2, w₂=−3, w₃=1 and bias b=−1, using the step activation (output 1 if z>0, else 0). Compute the output.

Answer: z = (2×1) + (−3×0) + (1×1) + (−1) = 2 + 0 + 1 − 1 = 2. Since 2 > 0, output = 1.

Q2. True or False: "A single artificial neuron with a straight-line decision boundary can be trained to correctly compute XOR if you just pick the right weights."

Answer: False. This is not a training limitation but a mathematical one — the ON and OFF points of XOR alternate diagonally, so no straight line, for any choice of weights and bias, can separate them. A hidden layer is required, as shown by the OR/NAND/AND construction in this chapter.

Q3. Using ReLU, f(z) = max(0, z), what is the output for z = −7 and for z = 4.5?

Answer: f(−7) = max(0, −7) = 0. f(4.5) = max(0, 4.5) = 4.5.

Q4. In the OR hidden neuron (weights 1, 1, bias −0.5), what is the smallest positive bias change that would flip its output for input (0,0) from 0 to 1, and why would that break the OR behaviour?

Answer: Currently z at (0,0) is −0.5. Raising the bias by any amount greater than 0.5 (for example, changing bias to 0.6, giving z = 0.1) would push z above 0, making the neuron output 1 for input (0,0) — but OR(0,0) should be 0, so this would make the neuron compute the wrong function.

Q5. In two or three sentences, explain in your own words why a biological neuron and an artificial neuron are described as "inspired by" rather than "identical to" one another.

Answer (model): A biological neuron is a physical electrochemical system whose firing depends on ion channels, neurotransmitter chemistry, and timing between thousands of synaptic connections. An artificial neuron keeps only the two most useful computational ideas — summing weighted inputs and comparing the sum to a threshold — and expresses them as plain arithmetic, discarding everything else about biology. The word "inspired" credits where the idea came from without claiming the artificial version reproduces how real neurons actually work.

Summary

• A biological neuron sums incoming signals from other neurons through weighted synaptic connections and fires an action potential once the sum crosses a threshold voltage.
• An artificial neuron copies only this sum-then-threshold behaviour: z = w₁x₁ + w₂x₂ + … + b, followed by an activation function f(z).
• The step function is the most literal "fire or don't" activation; ReLU (max(0,z)) and sigmoid (1/(1+e^−z)) are smoother alternatives that make gradual learning possible.
• A single neuron can only draw a straight-line decision boundary, so it cannot compute functions like XOR, where the two classes alternate diagonally — a fact proved by Minsky and Papert in 1969.
• Arranging neurons into an input layer, one or more hidden layers, and an output layer lets simple linear neurons combine into non-linear decision-making, as demonstrated by the OR-NAND-AND construction that correctly solves XOR.
• Real networks learn their weights automatically through gradient descent and backpropagation rather than having them hand-derived, using training examples to gradually reduce prediction error.
• An artificial neuron is a mathematical abstraction inspired by biology, not a working replica of a brain cell — it has no chemistry, no timing, and no true resemblance beyond the sum-and-threshold idea.

Think About It

Think about this: How would you explain neural networks: how the brain inspired computers to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.

Practice Exercises

Now it is time to practice! Complete these challenges to solidify your understanding:

• Exercise 1: Write a short program that demonstrates the core concept from this chapter. Test it with at least 3 different inputs.
• Exercise 2: Find a real-world example where neural networks: how the brain inspired computers is used in an Indian company (like TCS, Infosys, Flipkart, or ISRO). Write a paragraph explaining the connection.
• Exercise 3: Create a mind-map connecting neural networks: how the brain inspired computers to at least 3 other topics you have studied.