Open the camera app on any phone and take a photo of your class report card. To you, it is marks and a photo. To a neural network trying to read those marks automatically, it is nothing but numbers arranged in a grid — thousands of brightness values, one per pixel, sitting in rows and columns. Every single operation a deep learning model performs — recognising your face to unlock a phone, converting a photo of handwritten Hindi numerals into typed digits, or predicting tomorrow's rainfall from satellite images — comes down to doing arithmetic on giant grids of numbers, over and over, extremely fast. PyTorch's answer to "how do we store and compute on these grids efficiently?" is a single data structure called the tensor. This chapter builds that data structure from the ground up, using numbers you can check by hand.
Why Not Just Use a Python List?
Suppose you want to store the marks of 3 students across 4 subjects using plain Python. You would write something like marks = [[85, 90, 78, 92], [70, 65, 88, 74], [95, 91, 89, 96]] — a list of lists. This works for small examples, but it breaks down badly the moment a real neural network gets involved, for two concrete reasons.
First, a Python list is a list of pointers to separate Python integer objects scattered across memory. Every time you add two lists element-by-element, Python must loop through them one number at a time, and each addition involves the overhead of a full Python object (type-checking, reference counting, memory lookup) — even though the actual arithmetic is trivial. A real neural network layer might multiply and add millions of numbers per image, so this per-number overhead adds up to seconds or minutes where it should take milliseconds.
Second, a plain list has no idea it is supposed to represent a rectangular grid. Nothing stops you from writing [[85, 90, 78, 92], [70, 65]] — a "matrix" with rows of different lengths, which is nonsense for the matrix mathematics that neural networks are built on. There is also no way to say "run this computation on the GPU instead of the CPU," because a Python list is just a list.
PyTorch tensors solve both problems. A tensor stores its numbers in one contiguous block of memory, all of the same data type, so operations can be handed off in bulk to highly optimised C++ and CUDA code that processes many numbers in parallel — on a CPU's vector units, or on a GPU's thousands of cores. A tensor also always has a well-defined, fixed rectangular shape, so there is no ambiguity about its structure. This is the entire reason tensors exist: they are the data structure that makes large-scale numerical computation for deep learning fast and well-defined.
Building the Idea: From a Single Number to a Stack of Grids
Rather than memorising a definition, build it up one step at a time using data you already understand.
Step 1 — a single number. Suppose you want to store just one UPI transaction amount: ₹250. That is a single number with no direction and no list around it. In tensor language this is called a scalar, and it has zero axes — there is no row, no column, nothing to index into. We say it has 0 dimensions, written as rank 0 or "0-D."
Step 2 — a line of numbers. Now suppose you record four UPI payments made today: ₹250, ₹1200, ₹45, ₹900. This is a single row of related numbers — one axis along which you can move (payment 1, payment 2, payment 3, payment 4). This is called a vector, and it has 1 dimension (1-D).
Step 3 — a grid of numbers. Now go back to the report card: 3 students, each with 4 subject marks. To locate any one mark you now need two pieces of information — which student (row) and which subject (column). Two independent directions of movement means two axes. This is a matrix, with 2 dimensions (2-D).
Step 4 — a stack of grids. Now consider a small colour photograph, 2 pixels wide and 2 pixels tall. A colour image is not one grid of numbers — it is three grids stacked on top of each other, one grid recording the red brightness of every pixel, one for green, one for blue. To locate any one number you now need three pieces of information: which colour channel, which row, which column. Three axes means 3 dimensions (3-D), and this stack-of-grids object is what PyTorch calls, generically, a tensor.
This is the key idea to hold onto: scalar, vector, and matrix are just the 0-, 1-, and 2-dimensional special cases of one general idea — the tensor. In PyTorch, every one of these — the single UPI amount, the list of four payments, the report-card grid, the stacked colour image — is stored using the exact same object type, torch.Tensor. The word "tensor" simply means "an array of numbers with some number of axes," and that number of axes can be 0, 1, 2, 3, or more.
Creating Tensors in Code
PyTorch is a Python library; after installing it (pip install torch), you bring it into a program with import torch. The most direct way to build a tensor is torch.tensor(), handing it a Python number, list, or nested list. Trace through what each line below actually produces.
import torch
# Step 1: a scalar (0-D) - a single UPI payment
x = torch.tensor(7)
print(x) # tensor(7)
print(x.ndim) # 0
print(x.shape) # torch.Size([])
# Step 2: a vector (1-D) - four UPI payments in rupees
amounts = torch.tensor([250, 1200, 45, 900])
print(amounts) # tensor([ 250, 1200, 45, 900])
print(amounts.ndim) # 1
print(amounts.shape) # torch.Size([4])
# Step 3: a matrix (2-D) - marks of 3 students in 4 subjects
marks = torch.tensor([
[85, 90, 78, 92],
[70, 65, 88, 74],
[95, 91, 89, 96]
])
print(marks.ndim) # 2
print(marks.shape) # torch.Size([3, 4])
Walk through why the shapes come out the way they do. x.shape prints torch.Size([]) — an empty set of brackets — because a scalar has no axes to measure the length of. amounts.shape prints torch.Size([4]) because there is one axis, and it has 4 entries along it. marks.shape prints torch.Size([3, 4]) because there are two axes: the first (rows/students) has length 3, and the second (columns/subjects) has length 4. Notice the order matters — shape always lists outermost axis first, so (3, 4) means "3 groups of 4," matching exactly how the nested list was written: 3 inner lists, each with 4 numbers.
PyTorch also gives you shortcuts that build tensors without typing every number: torch.zeros(2, 3) makes a 2×3 grid filled with 0.0, torch.ones(4) makes a length-4 vector of 1.0, and torch.arange(0, 10, 2) makes an evenly spaced sequence, producing tensor([0, 2, 4, 6, 8]) — starting at 0, stepping by 2, stopping before 10. These are useful for initialising tensors before filling them with real data, or for quick testing.
Two Attributes You Must Never Confuse
This is a point where students consistently trip up, so name it directly: ndim (the number of axes) is not the same thing as the number of values stored in the tensor. A student who has not internalised the difference will look at marks above — 12 numbers in total — and guess marks.ndim should be 12, or will look at a 3×4 matrix and think "2 dimensions" means "2 numbers." Neither is correct.
ndim counts axes — independent directions you can move along to locate a value. marks has 2 axes (student-direction and subject-direction), regardless of whether it holds 12 marks or 12,000. The total count of individual numbers stored is a separate quantity, given by marks.numel() ("number of elements"), which multiplies the shape together: 3 × 4 = 12. A matrix of shape (100, 50) still has ndim = 2, even though numel() = 5000. Keep these two ideas — "how many axes" versus "how many numbers total" — strictly separate, because CBSE-style exam questions will often ask for one when a student instinctively answers with the other.
print(marks.numel()) # 12 (3 rows x 4 columns)
print(marks.ndim) # 2 (row-axis and column-axis)
print(marks.dtype) # torch.int64
dtype is the third essential attribute: it tells you what kind of number each entry is. Since marks was built from Python integers, PyTorch automatically chose torch.int64. If even one number in the list had a decimal point, PyTorch would switch the entire tensor to torch.float32 — a tensor must hold one single data type for every entry, which is exactly the memory-layout discipline that makes it fast. Try it:
a = torch.tensor([1, 2, 3])
print(a.dtype) # torch.int64
b = torch.tensor([1.0, 2.0, 3.0])
print(b.dtype) # torch.float32
c = a.float() # cast to float32
print(c) # tensor([1., 2., 3.])
print(c.dtype) # torch.float32
Indexing and Slicing: Reaching Into a Tensor
Because a tensor's axes are ordered and fixed-length, you can reach exactly one number, one row, or one column using the same square-bracket indexing you already know from Python lists — extended to handle multiple axes at once, separated by commas.
marks = torch.tensor([
[85, 90, 78, 92], # student 0
[70, 65, 88, 74], # student 1
[95, 91, 89, 96] # student 2
])
print(marks[0]) # tensor([85, 90, 78, 92]) - all of student 0's marks
print(marks[0, 2]) # tensor(78) - student 0, subject index 2
print(marks[:, 1]) # tensor([90, 65, 91]) - subject index 1, every student
Check the middle line by hand: marks[0, 2] asks for row index 0, column index 2. Row 0 is [85, 90, 78, 92]; counting from index 0, position 2 is 78 — the third mark, not the second, because indexing starts at 0, a rule carried over directly from Python lists. The last line uses : to mean "every index along this axis" — marks[:, 1] reads "every row, but only column 1," pulling out the second subject's mark for all three students: 90, 65, 91.
Reshaping: Same Numbers, Different Grid
Reshaping rearranges how many numbers sit along each axis without changing the numbers themselves or their order in memory — as long as the total count matches. This matters constantly in deep learning, where a flat sequence of pixel values coming off a camera sensor often needs to be reorganised into the (height, width) grid the model expects.
t = torch.arange(1, 7) # tensor([1, 2, 3, 4, 5, 6])
print(t.shape) # torch.Size([6])
r = t.reshape(2, 3)
print(r)
# tensor([[1, 2, 3],
# [4, 5, 6]])
r2 = t.reshape(3, 2)
print(r2)
# tensor([[1, 2],
# [3, 4],
# [5, 6]])
Both reshapes are valid because 2 × 3 = 6 and 3 × 2 = 6 — the total numel() is preserved; PyTorch simply reads the original 6 numbers in order and refills them into the new shape, filling the last axis first. Trying t.reshape(2, 4) would fail immediately, because 2 × 4 = 8 does not equal the 6 numbers actually available — a runtime error PyTorch raises rather than silently guessing. You will also encounter t.view(2, 3), which does the identical job; the difference is that view() requires the tensor's data to already sit contiguously in memory and raises an error otherwise, while reshape() handles both cases, copying the data if it must. As a beginner, prefer reshape() and reach for view() once you understand memory layout in more depth.
Arithmetic and Broadcasting
Tensor arithmetic works element-by-element by default, and the real power appears when the two tensors involved do not have identical shapes. Suppose your teacher awards uneven grace marks per subject — 5 for the first subject, 3 for the second, 2 for the third — as a single vector, and you want to apply it to every student's row in one line rather than writing a loop.
marks2 = torch.tensor([
[85, 90, 78],
[70, 65, 88]
])
grace = torch.tensor([5, 3, 2])
result = marks2 + grace
print(result)
# tensor([[90, 93, 80],
# [75, 68, 90]])
marks2 has shape (2, 3) and grace has shape (3,) — different ranks entirely, yet the addition works. PyTorch's broadcasting rule compares shapes starting from the rightmost axis: here, both have a final axis of length 3, so they align; the missing leading axis on grace is treated as length 1 and is virtually copied to match marks2's 2 rows, without actually duplicating any memory. So row 0 becomes 85+5, 90+3, 78+2 = 90, 93, 80, and row 1 becomes 70+5, 65+3, 88+2 = 75, 68, 90 — exactly what the printed result shows. Broadcasting is what lets you add a single bias number, or a single per-column adjustment, to an entire batch of data in one vectorised instruction instead of writing nested loops.
Beyond 2 Axes: Images and Batches
Return to the tiny 2×2 colour image from earlier. Its red, green, and blue channels are each a 2×2 grid, and stacking the three grids gives a 3-D tensor:
image = torch.tensor([
[[255, 0], [0, 255]], # red channel
[[ 0, 255], [0, 0]], # green channel
[[ 0, 0], [255, 255]] # blue channel
])
print(image.shape) # torch.Size([3, 2, 2])
print(image.ndim) # 3
The shape (3, 2, 2) reads as (channels, height, width) — this exact ordering, channels before height and width, is PyTorch's convention for image tensors, and it is worth knowing explicitly because it is a genuine, easy-to-miss trap: TensorFlow/Keras instead defaults to (height, width, channels), so a tensor that loads correctly in one framework can silently have its axes misread in the other if you assume the wrong order.
Real training rarely processes one image at a time — it processes a batch, adding a fourth axis on the front. Consider a crop-disease detection model trained on leaf photographs, processing 32 images at a time, each resized to 64×64 pixels in colour: this batch is a single 4-D tensor of shape (32, 3, 64, 64) — batch size, channels, height, width. The same pattern extends further still: a multispectral satellite image, of the kind used in remote-sensing work, might carry far more than 3 channels — one grid per wavelength band the sensor records — giving a tensor shaped (bands, height, width) with, say, 8 or 12 channels instead of 3. The rank-2 report-card matrix, the rank-3 single image, and the rank-4 image batch are all the same underlying idea applied one axis further each time.
A Preview: Device and Gradients
Two tensor attributes matter once you start training actual networks, even though this chapter stops short of using them in depth. A tensor's .device tells you whether its numbers live in ordinary computer memory (cpu) or on a graphics card's memory (cuda); moving a tensor with tensor.to('cuda') is what lets its arithmetic run on a GPU's thousands of parallel cores instead of a CPU's handful — the entire reason training large networks on a CPU alone can take days where a GPU takes hours. Separately, creating a tensor with torch.tensor([1.0, 2.0], requires_grad=True) tells PyTorch to start tracking every operation performed on it, so that later — during training — it can automatically work out how to adjust that tensor's numbers to reduce a model's error. That mechanism, called autograd, is the subject of its own chapter; for now, simply recognise the flag when you see it, and know that it turns an ordinary tensor into one PyTorch is prepared to learn from.
Two Misconceptions Worth Correcting Now
Misconception 1: "Tensor is just a fancier word for matrix." A matrix is specifically the 2-axis case. A tensor is the general term covering every rank — a single scalar is a 0-D tensor, a list of UPI amounts is a 1-D tensor, a report card is a 2-D tensor, a colour photo is a 3-D tensor, and a batch of photos is a 4-D tensor. Every one of these, in PyTorch code, is stored in an identical object type, torch.Tensor — the class does not change with rank, only the shape attribute does. Saying "matrix" when you specifically mean 2 axes is fine; saying "matrix" to describe a 4-D image batch is a real, board-exam-relevant error.
Misconception 2: "The number of dimensions tells you how much data is inside." As shown above with numel() versus ndim, these are unrelated quantities. A shape of (1000, 1000) has only 2 axes yet holds a million numbers, while a shape of (2, 2, 2, 2, 2) has 5 axes yet holds only 32 numbers. When a question asks for the "dimensionality" or "rank" of a tensor, it wants the count of axes — read off as the length of the shape tuple — never the total element count.
Reading the Diagram
The figure below places all four ranks side by side, using the exact numeric examples worked through above, so you can trace how each shape tuple corresponds to the picture: an empty tuple for the single number, one bracketed length for the row of payments, two lengths for the report-card grid, and three lengths for the stacked colour channels.
Rank, Axes, Order: The Same Idea, Different Names
One more piece of vocabulary bookkeeping before practice. In PyTorch code you will see .ndim and the length of .shape used interchangeably — they always agree. In CBSE Computer Science / Informatics Practices coursework and in general data-science writing, the same quantity is sometimes called the rank or the order of the tensor. All three words — dimensionality, rank, order — mean exactly the count of axes. If an exam question asks for the "order of the tensor," it wants the same number marks.ndim would give you: 2, not 12, and not 3 or 4.
Active Recall
Q1. Cricket ball-by-ball data for one over is stored as a single list of 6 runs scored: [1, 4, 0, 6, 2, 1]. What rank tensor is this, and what is its shape?
Answer: One axis (the ball number within the over), so it is a 1-D tensor, rank 1, shape (6,).
Q2. Now extend Q1 to a full T20 innings: runs scored on each of 6 balls, across each of 20 overs. What shape would this tensor have, and what does each axis represent?
Answer: Shape (20, 6) — a 2-D tensor (matrix) — where the first axis (length 20) is the over number and the second axis (length 6) is the ball-within-the-over.
Q3. A tensor t has t.shape equal to torch.Size([5, 7]). Without running any code, state t.ndim and t.numel().
Answer: ndim is 2, because shape lists exactly 2 lengths. numel() is 5 × 7 = 35 — the two are unrelated: one counts axes, the other counts stored numbers.
Q4. You have a = torch.tensor([10, 20, 30]) with shape (3,), and b = torch.tensor([[1, 2, 3], [4, 5, 6]]) with shape (2, 3). What does a + b evaluate to, and why is it allowed even though the shapes differ?
Answer: It is allowed by broadcasting, because comparing from the rightmost axis, both have length 3 there, and a's missing leading axis is treated as length 1 and repeated across b's 2 rows. Row 0: 10+1, 20+2, 30+3 = 11, 22, 33. Row 1: 10+4, 20+5, 30+6 = 14, 25, 36. Result: tensor([[11, 22, 33], [14, 25, 36]]), shape (2, 3).
Q5. A batch of 16 black-and-white (single-channel) X-ray scans, each 128×128 pixels, is stored as one PyTorch tensor following the standard (batch, channels, height, width) convention. Write its shape as a tuple.
Answer: (16, 1, 128, 128) — batch size 16, 1 channel because the scans are grayscale rather than colour, height 128, width 128.
Summary
A PyTorch tensor is a single, unified data structure for numeric arrays of any rank, built specifically so that large numeric computations run fast (contiguous, single-dtype memory laid out for vectorised CPU/GPU execution) and unambiguously (a fixed rectangular shape at every rank). A scalar is a 0-axis tensor, a vector is 1-axis, a matrix is 2-axis, and stacking matrices — such as colour channels of an image, or images into a batch — produces 3-, 4-, and higher-axis tensors, all handled by the identical torch.Tensor type. The three attributes to check first on any unfamiliar tensor are .shape (the length along each axis), .ndim (how many axes there are — never confuse this with the total element count, which is .numel()), and .dtype (what kind of number is stored, since a tensor cannot mix types). Indexing with comma-separated positions and colons reaches into any axis; .reshape() rearranges a tensor's axes while preserving its data and total count; and broadcasting lets tensors of compatible-but-different shapes combine element-wise without you writing an explicit loop. Every neural network you build going forward — its input data, its internal weights, its output predictions — is, underneath, nothing more than tensors like these, combined through operations exactly like the ones practiced in this chapter.
Think About It
Think about this: How would you explain pytorch tensors: foundation of deep learning to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.