TensorFlow & Keras: Building Neural Networks

The problem with doing it all by hand

Suppose you want a computer to decide whether a student is likely to clear the pass mark, using just two numbers: hours studied and attendance percentage. A single artificial neuron can do this. It multiplies each input by a weight, adds a bias, and squashes the result into a probability between 0 and 1 using the sigmoid function. You can write that neuron in five lines of plain Python:

import math

w1, w2, b = 0.0, 0.0, 0.0      # weights and bias start at zero
x1, x2 = 8, 90                  # hours studied, attendance %
y_true = 1                      # 1 = passed

z = w1 * x1 + w2 * x2 + b
y_pred = 1 / (1 + math.exp(-z))
error = y_true - y_pred
print(y_pred, error)

Trace it: z = 0*8 + 0*90 + 0 = 0, so y_pred = 1 / (1 + math.exp(0)) = 1/2 = 0.5, and error = 1 - 0.5 = 0.5. The neuron currently has no idea whether the student passed, so it guesses 50-50 — correct. To make it learn, you would now have to nudge w1, w2, and b slightly in the direction that reduces the error, using calculus (a technique called gradient descent), and repeat that nudge thousands of times.

That is one neuron. A real network has dozens of neurons arranged in layers, and each one needs its own weights updated using derivatives that depend on every other neuron downstream of it — a chain-rule calculation called backpropagation. Writing that by hand for anything beyond a toy example is slow and extremely easy to get wrong. This is exactly the gap TensorFlow and Keras fill: TensorFlow does the calculus automatically, and Keras lets you describe the network's shape instead of its arithmetic.

What TensorFlow actually is

TensorFlow is a numerical computing library released by Google Brain in 2015. Its core job is to represent computations as graphs of mathematical operations on tensors (explained below), and to automatically compute derivatives of those operations — a feature called automatic differentiation, or autodiff. When you build a neural network in TensorFlow, you are not writing the gradient-descent update rule yourself; you describe the network's structure, and TensorFlow works out, layer by layer, exactly how much each weight contributed to the final error and adjusts it accordingly.

TensorFlow by itself is a fairly low-level toolkit — think of it as the engine, not the dashboard. Building a network directly in raw TensorFlow means manually creating weight tensors, writing the forward-pass arithmetic, and wiring up the training loop. That is where Keras comes in.

What Keras actually is

Keras is a high-level API for building neural networks. It was created by François Chollet as an independent project, and for years it could run on top of TensorFlow, Theano, or other backends. When Google adopted it as TensorFlow's official high-level interface, it was folded into TensorFlow as the tf.keras module — the way this chapter uses it, via from tensorflow import keras. Since Keras 3 (released in late 2023 and bundled by default from TensorFlow 2.16 onward), Keras has gone back to also being distributed as its own separate package, and it can now run on TensorFlow, JAX, or PyTorch as the underlying engine. None of that changes the code in this chapter: for everyday TensorFlow use, tf.keras remains the built-in, default interface, and every line below runs exactly as shown.

In short: TensorFlow computes the maths, and Keras gives you a small, readable vocabulary — layers, models, compile, fit — for describing what network you want, without writing that maths yourself.

Tensors: the data structure everything is built from

A tensor is simply a container for numbers, generalised across dimensions:

• A single number, like 7, is a rank-0 tensor (a scalar).
• A list of numbers, like [8, 90] (hours studied, attendance %), is a rank-1 tensor (a vector).
• A table of numbers — rows of students, columns of features — is a rank-2 tensor (a matrix). For example, eight students with two features each form a tensor of shape (8, 2): 8 rows, 2 columns.
• Stack multiple matrices together (say, a batch of colour images, each with height, width, and 3 colour channels) and you get a rank-3 or higher tensor.

Every input, every weight, and every output in a Keras network is a tensor. When you see a shape like (None, 4) later in this chapter, the None simply means "any number of rows" (Keras does not fix how many students you feed it at once), and 4 means "4 columns" — in that case, 4 numbers produced per student.

Setting up

Install TensorFlow with pip install tensorflow (Keras comes bundled as a dependency, so no separate install step is needed for the code in this chapter). Then import it and check the version:

import tensorflow as tf
from tensorflow import keras

print(tf.__version__)   # e.g. 2.16.1

Any version from 2.16 onward will use Keras 3 internally, which is what produces the exact output shown for model.summary() and model.fit() later in this chapter.

The problem we'll build: will this student clear the pass mark?

We'll use two features that will feel familiar from your own school reports: hours studied for a test, and attendance percentage (many CBSE schools set a minimum attendance requirement, often around 75%, before you can even sit an exam — so attendance genuinely correlates with outcomes). The label is 1 if the student passed, 0 if they didn't. This is a binary classification problem — exactly two possible outputs.

Notice the two features live on very different numeric scales: hours studied ranges roughly 0–10, while attendance ranges 0–100. In a serious project you would rescale both to a comparable range before training, typically with a keras.layers.Normalization layer, because a feature with naturally larger numbers can dominate a network's early learning simply due to its scale, not because it's more important. We'll skip that step here to keep the focus on how Sequential, compile, and fit work, but it's worth remembering for real datasets.

Building the network with the Sequential API

keras.Sequential lets you build a network by listing its layers in order, like items on a pipeline — data flows in at the top and out at the bottom, through each layer exactly once:

model = keras.Sequential([
    keras.Input(shape=(2,)),
    keras.layers.Dense(4, activation='relu'),
    keras.layers.Dense(1, activation='sigmoid')
])

Reading this line by line:

• keras.Input(shape=(2,)) declares that every example fed into this network is a vector of 2 numbers — our two features. It doesn't do any computation; it just fixes the shape of the data the first real layer should expect.
• keras.layers.Dense(4, activation='relu') is a "fully connected" hidden layer of 4 neurons. "Dense" means every one of these 4 neurons receives a connection from every input — here, both features. Each neuron computes its own weighted sum plus bias, then applies the ReLU activation function, which is simply max(0, z): if the weighted sum is negative, output 0; otherwise, pass it through unchanged. ReLU is the standard choice for hidden layers because it's cheap to compute and helps the network learn faster than older activation functions.
• keras.layers.Dense(1, activation='sigmoid') is the output layer: a single neuron that takes the 4 numbers from the hidden layer, computes one more weighted sum plus bias, and applies the sigmoid function to squash it into a value between 0 and 1 — a predicted probability of "passed."

You'll notice we used keras.Input(shape=(2,)) as the first entry rather than passing input_shape=(2,) directly to the first Dense layer. Older Keras code often does the latter, and it still runs, but current Keras 3 prefers an explicit Input layer and will print a harmless warning if you skip it. Using Input is now the recommended style, so that's what we'll use throughout.

A gentle look at why sigmoid uses e

The sigmoid function is defined as:

σ(z) = 1 / (1 + e^−z)

Here, e (≈ 2.718) is a fixed mathematical constant, the same way π (≈ 3.14159) is a fixed constant — it isn't something you choose, it's a specific number that shows up naturally in growth and decay calculations. It's used in sigmoid because it produces a smooth S-shaped curve whose slope is easy to calculate at every point, which matters a lot once TensorFlow starts computing gradients through this function thousands of times during training.

Let's compute one value by hand, the way TensorFlow does internally. Suppose after training, for one student, the output neuron's weighted sum works out to z = 2 (a fairly confident positive signal). Then:

e^-2 ≈ 0.135
sigmoid(2) = 1 / (1 + 0.135) = 1 / 1.135 ≈ 0.881

A predicted probability of 0.881 means the network is fairly confident (88.1%) this student passed. If z were a large negative number instead, say z = -5, then e^5 ≈ 148.4, giving sigmoid(-5) = 1/149.4 ≈ 0.007 — a confident "did not pass." Whatever the weighted sum is, sigmoid always squeezes it into the (0, 1) range, which is exactly what you want for a probability.

Counting parameters by hand

Before you even run the code, you can work out exactly how many numbers this network will learn — its parameters (all the weights and biases combined).

A common misconception is that "a Dense layer with 4 neurons has 4 parameters." It doesn't — each neuron has one weight per incoming connection, plus one bias of its own. Count connections, not just neurons:

• Hidden layer (Dense(4) fed by 2 inputs): each of the 4 neurons connects to both of the 2 inputs, giving 2 × 4 = 8 weights, plus 4 biases (one per neuron) = 12 parameters.
• Output layer (Dense(1) fed by the 4 hidden neurons): 4 × 1 = 4 weights, plus 1 bias = 5 parameters.
• Total: 12 + 5 = 17 parameters.

Now check that against Keras itself:

model.summary()

On TensorFlow 2.16+ (Keras 3), this prints a box-drawn table with a byte-size annotation next to every parameter count, like this:

Model: "sequential"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                   ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ dense (Dense)                  │ (None, 4)              │            12 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_1 (Dense)                 │ (None, 1)              │             5 │
└─────────────────────────────────┴────────────────────────┴───────────────┘
 Total params: 17 (68.00 B)
 Trainable params: 17 (68.00 B)
 Non-trainable params: 0 (0.00 B)

Everything lines up with the hand count: 12 for the hidden layer, 5 for the output layer, 17 total. The (68.00 B) is new in Keras 3 — it's the memory the parameters occupy, and you can check it yourself: each parameter is stored as a 32-bit float, which is 4 bytes, so 17 × 4 = 68 bytes. (The exact column widths you see may differ slightly depending on your terminal, but the box-drawing structure and the byte annotations shown here are exactly what Keras 3 produces — this is a real change from older Keras 2 versions, which printed a plain two-column table with no byte sizes at all.) Also notice (None, 4) and (None, 1): None is the batch dimension (any number of students), and 4 or 1 is how many numbers come out per student from that layer.

Compiling: telling Keras how to learn

model.compile(
    optimizer='adam',
    loss='binary_crossentropy',
    metrics=['accuracy']
)

Compiling doesn't train anything yet — it configures three things the training loop will need:

• optimizer='adam': the algorithm that decides how to adjust every weight after seeing the error, an improved version of the plain gradient-descent update rule from the opening example. Adam adapts its step size automatically per parameter, which is why it's the default choice for most networks.
• loss='binary_crossentropy': the formula used to measure how wrong a prediction is, specifically suited to problems with two classes (pass/fail). It penalises confident-but-wrong predictions much more heavily than cautious ones — predicting 0.99 for a student who actually failed costs far more than predicting 0.6 for the same mistake.
• metrics=['accuracy']: a number reported purely for humans to read during training — the fraction of predictions that matched the true label. Unlike the loss, accuracy plays no role in how weights are updated.

Training: model.fit() and what its output means

Here is a tiny training set of 8 students — small on purpose, so we can trace exactly what happens:

import numpy as np

X = np.array([
    [2, 60], [8, 90], [5, 75], [1, 50],
    [9, 95], [3, 65], [7, 85], [4, 70]
], dtype='float32')

y = np.array([0, 1, 1, 0, 1, 0, 1, 1], dtype='float32')

model.fit(X, y, epochs=2, batch_size=4)

An epoch is one full pass through the training data. With 8 students and batch_size=4, Keras splits each epoch into 8 ÷ 4 = 2 steps — it looks at 4 students, computes the average error, updates every weight once, then repeats for the next 4. With epochs=2, this whole process happens twice. On TensorFlow 2.16+ (Keras 3), the printed output uses a solid Unicode progress bar rather than the older ===== ASCII style, and it lists metrics alphabetically — so accuracy appears before loss:

Epoch 1/2
2/2 ━━━━━━━━━━━━━━━━━━━━ 1s 87ms/step - accuracy: 0.5000 - loss: 0.7134
Epoch 2/2
2/2 ━━━━━━━━━━━━━━━━━━━━ 0s 4ms/step - accuracy: 0.5000 - loss: 0.7042

The 2/2 is the step counter reaching its final value; the exact accuracy and loss numbers you get will differ slightly from these, because Keras initialises every weight to a small random value before training starts (that's why the opening hand example set weights to zero only as a simplified illustration — real Dense layers never actually initialise at exactly zero). If you want the same numbers every time you run the code, fix the randomness first with tf.random.set_seed(42) before building the model. What is guaranteed to match, regardless of your random seed, is the format: the block-character progress bar and the alphabetical metric order are fixed by Keras 3 itself, not by your data or your weights.

With only 8 examples and 2 epochs, don't expect the accuracy to climb much — that's expected and not a bug. Real training uses far more data and many more epochs; this example is deliberately kept small enough to trace by hand.

Making predictions

new_student = np.array([[6, 80]], dtype='float32')
prediction = model.predict(new_student)
print(prediction)

Note the double square brackets: model.predict always expects a batch, even a batch of one, so a single student's 2 features must be wrapped in an outer list to make the shape (1, 2) — 1 row, 2 columns — matching what the network's Input(shape=(2,)) expects. The output is a small array like [[0.63]], a probability between 0 and 1. To turn that into a hard yes/no answer, you'd typically threshold it: "pass" if prediction[0][0] >= 0.5 else "fail".

Visualising the network

The diagram below shows exactly the network we built: 2 input features, a 4-neuron ReLU hidden layer, and a 1-neuron sigmoid output, with every connection drawn as a line — count them and you'll find 8 lines between the inputs and the hidden layer, and 4 more between the hidden layer and the output, matching the 8 + 4 = 12 weights we counted by hand (plus 5 biases, not drawn, since a bias belongs to a neuron rather than a connection).

Test your understanding

Work these out before checking the answer — each one uses exactly the parameter-counting method from this chapter.

1. How many parameters does keras.Sequential([keras.Input(shape=(3,)), keras.layers.Dense(5, activation='relu'), keras.layers.Dense(2, activation='softmax')]) have, layer by layer?
   Answer: Hidden layer: 3 inputs × 5 neurons = 15 weights + 5 biases = 20 parameters. Output layer: 5 × 2 = 10 weights + 2 biases = 12 parameters. Total: 20 + 12 = 32. You can confirm this by calling model.summary() after building it.
2. If you train with 20 samples and batch_size=5, how many steps will each epoch show in the progress line (like the 2/2 you saw above)?
   Answer: 20 ÷ 5 = 4, so Keras will print 4/4 at the end of each epoch.
3. Why does model.predict() need [[6, 80]] (nested brackets) instead of just [6, 80] for one student?
   Answer: Keras layers always expect a batch dimension first. [6, 80] has shape (2,) — a single flat vector — but the network's Input(shape=(2,)) expects shape (batch_size, 2). Wrapping it as [[6, 80]] gives shape (1, 2): a batch containing one student.
4. A hidden layer's ReLU activation outputs exactly 0 for a particular student. Does that mean the layer has 0 parameters for that student?
   Answer: No — parameters (weights and biases) belong to the layer's structure and exist regardless of any particular input; ReLU outputting 0 just means that neuron's weighted sum was negative or zero for this one student. The next student through the same layer, with different feature values, can easily produce a positive output from the very same neuron.

Summary

TensorFlow supplies the numerical engine — tensors as the data structure, and automatic differentiation to compute how every weight should change after an error. Keras (accessed here through tf.keras, part of the Keras 3 generation bundled since TensorFlow 2.16) supplies the vocabulary for describing a network without writing that calculus yourself: Sequential to stack layers, Dense for fully connected layers with an activation function, compile to choose an optimizer and loss function, fit to run training for a chosen number of epochs and batch size, and predict to run new data through the trained network. A Dense layer's parameter count is always (inputs × neurons) weights plus one bias per neuron — a calculation you can and should do by hand before ever calling model.summary(), both to check your understanding and to catch a wrongly-shaped network before you waste time training it. And because Keras 3 changed how its output looks — box-drawn summary tables with byte sizes, block-character progress bars with alphabetically ordered metrics — it's worth always checking tf.__version__ first, so the output you see on your own machine matches what you expect from what you read.

Think About It

Think about this: How would you explain tensorflow & keras: building neural networks to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.

Practice Exercises

Now it is time to practice! Complete these challenges to solidify your understanding:

• Exercise 1: Write a short program that demonstrates the core concept from this chapter. Test it with at least 3 different inputs.
• Exercise 2: Find a real-world example where tensorflow & keras: building neural networks is used in an Indian company (like TCS, Infosys, Flipkart, or ISRO). Write a paragraph explaining the connection.
• Exercise 3: Create a mind-map connecting tensorflow & keras: building neural networks to at least 3 other topics you have studied.