Data Analysis with Pandas

The Trouble With Parallel Lists

Imagine your class teacher hands you the unit-test marks for five students and asks you to find out who scored above 75% overall. You decide to store the data the way you already know: as plain Python lists, one list per subject, with each student occupying the same position (index) in every list.

names   = ['Ananya', 'Rohit', 'Priya', 'Karthik', 'Meera']
maths   = [88, 72, 95, 60, 45]
science = [91, 68, 89, 55, 50]
english = [79, 74, 92, 65, 58]

totals = []
for i in range(len(names)):
    totals.append(maths[i] + science[i] + english[i])

print(totals)

# Output:
[258, 214, 276, 180, 153]

This works, but notice how fragile it is. The only thing connecting "Ananya's Maths mark" to "Ananya's Science mark" is that both happen to sit at position 0 in two completely separate lists. If a new student is added to names but a teacher forgets to add their Science mark, every list silently falls out of sync, and maths[3] will quietly start referring to the wrong person's marks. There is no name attached to the number 60 in the maths list — you have to remember it belongs to Karthik because of where it sits. Add a fourth subject, and you must remember to update the loop everywhere it appears. This is exactly the kind of bug that is easy to introduce and hard to notice, because Python never complains — it just gives you wrong answers with complete confidence.

What we actually want is a single object that keeps a student's name and all their marks glued together as one row, remembers which column is "Maths" and which is "Science" by name rather than by position, and lets us ask questions like "average Maths mark" or "everyone above 75%" without writing a loop by hand. That object is called a DataFrame, and it is the central idea of the pandas library.

Meet pandas: A Table That Remembers Its Own Labels

pandas is a Python library built for working with tabular data — data that naturally lives in rows and columns, exactly like a mark sheet, a bank passbook, or an IRCTC ticket-booking history. Its two core objects are the Series and the DataFrame.

A Series is a single column of data where every value has a label attached to it, called an index. Think of it as one list, but each entry also carries a name tag. A DataFrame is simply a collection of Series that all share the same index — in other words, a full table, where each column is a Series and every row is identified by a shared row label.

Let's rebuild the marks data properly, this time as a dictionary where each key becomes a column:

import pandas as pd

data = {
    'Roll': [1, 2, 3, 4, 5],
    'Name': ['Ananya', 'Rohit', 'Priya', 'Karthik', 'Meera'],
    'Maths': [88, 72, 95, 60, 45],
    'Science': [91, 68, 89, 55, 50],
    'English': [79, 74, 92, 65, 58]
}
df = pd.DataFrame(data)
print(df)

# Output:
   Roll     Name  Maths  Science  English
0     1   Ananya     88       91       79
1     2    Rohit     72       68       74
2     3    Priya     95       89       92
3     4  Karthik     60       55       65
4     5    Meera     45       50       58

The bold numbers on the far left (0, 1, 2, 3, 4) are the index — pandas creates this automatically unless you tell it otherwise. Notice that "Karthik's row" is now one indivisible unit: his Roll number, name, and all three marks travel together labelled by index 3, no matter how the table gets reordered later. This single change — from five separate lists to one labelled table — is what makes real data analysis practical instead of error-prone.

Reading Real Data From a File

Data you receive in real life almost never arrives as a hand-typed dictionary. It usually comes as a CSV file (comma-separated values) — the format Google Forms exports quiz results in, the format IRCTC lets you download your booking history in, and the format most school result portals export mark sheets in. pandas reads these directly:

df = pd.read_csv('class8_marks.csv')

print(df.shape)      # (5, 5)  -> 5 rows, 5 columns
print(df.columns)    # Index(['Roll', 'Name', 'Maths', 'Science', 'English'], dtype='object')
print(df.head())     # first 5 rows, useful when a file has thousands of rows

df.shape returns a pair (rows, columns) — a fast way to sanity-check that the file loaded correctly before you do anything else. df.head() is important once files get large: if a CSV has 40,000 rows (a whole term's attendance log, say), printing all of it would flood your screen uselessly. head() shows just the first five by default so you can eyeball the structure. You can also check what type of data each column actually holds:

print(df.dtypes)

# Output:
Roll        int64
Name       object
Maths       int64
Science     int64
English     int64
dtype: object

This matters more than it looks. If a marks column somehow got read in as object (text) instead of int64 — which happens if even one cell in the CSV contains something like "absent" instead of a number — every arithmetic operation on that column will fail or behave strangely. Checking dtypes early is a habit worth building.

The Anatomy of a DataFrame

Before going further, it helps to see the pieces named clearly in one picture: the index (row labels), the columns (field names), and the fact that any single column, pulled out on its own, is a Series.

The orange band marks the Maths column, which — pulled out alone — is a Series. The blue band marks row index 2, which is Priya's complete record: every column, one shared label. This distinction between "a column (Series)" and "a row (record)" and the direction pandas means by axis will come up constantly, because most aggregate functions default to working down columns (axis=0).

Selecting Columns, Rows, and Cells

To pull out one column, use square brackets with the column name:

print(df['Maths'])

# Output:
0    88
1    72
2    95
3    60
4    45
Name: Maths, dtype: int64

Misconception check: a very common mistake is assuming df['Maths'] and df[['Maths']] mean the same thing. They don't — the number of square brackets changes the type you get back.

print(type(df['Maths']))
# <class 'pandas.core.series.Series'>

print(type(df[['Maths']]))
# <class 'pandas.core.frame.DataFrame'>

print(df[['Maths']])

# Output:
   Maths
0     88
1     72
2     95
3     60
4     45

Single brackets with one name give you a Series (a single labelled column, printed as a list of values). Double brackets — really a list of column names inside the selection brackets, ['Maths'] — give you back a DataFrame, a full table with just one column in it. The difference matters because a Series and a one-column DataFrame don't support quite the same operations, and code that expects one will sometimes silently misbehave with the other.

To select rows, pandas gives you two different tools, and mixing them up is the single most common beginner error: .loc[] selects by label (the index value you see printed on the left), while .iloc[] selects by position (0 for the first physical row, 1 for the second, regardless of what its label says). As long as the index hasn't been reordered, they look identical and this distinction seems pointless — until you sort the data.

df_sorted = df.sort_values('Maths', ascending=False)
print(df_sorted)

# Output:
   Roll     Name  Maths  Science  English
2     3    Priya     95       89       92
0     1   Ananya     88       91       79
1     2    Rohit     72       68       74
3     4  Karthik     60       55       65
4     5    Meera     45       50       58

print(df_sorted.iloc[0])   # position 0 = the physically first row after sorting
# Roll             3
# Name         Priya
# Maths           95
# Science         89
# English         92
# Name: 2, dtype: object

print(df_sorted.loc[0])    # label 0 = whoever originally had index label 0
# Roll             1
# Name        Ananya
# Maths           88
# Science         91
# English         79
# Name: 0, dtype: object

Sorting rearranged the rows but did not renumber the index — Priya still carries label 2, Ananya still carries label 0, exactly as they did in the original table. So .iloc[0], asking for "whatever sits first right now," correctly returns Priya, the highest scorer. But .loc[0], asking for "the row whose label is literally 0," returns Ananya — because that was always her label, sorting or not. Students who assume .loc means "location = position" get tripped up here every time; the safer mental model is .loc reads a name tag, .iloc counts seats.

Vectorized Computation: Adding New Columns

Go back to the opening loop that summed three lists by hand, one student at a time. pandas replaces that entire loop with one line:

df['Total'] = df['Maths'] + df['Science'] + df['English']
df['Percentage'] = (df['Total'] / 3).round(2)
print(df)

# Output:
   Roll     Name  Maths  Science  English  Total  Percentage
0     1   Ananya     88       91       79    258       86.00
1     2    Rohit     72       68       74    214       71.33
2     3    Priya     95       89       92    276       92.00
3     4  Karthik     60       55       65    180       60.00
4     5    Meera     45       50       58    153       51.00

No for i in range(...) anywhere. df['Maths'] + df['Science'] + df['English'] adds all five Maths values to all five Science values to all five English values, position by matching position, in one shot — this is called a vectorized operation. Under the hood, pandas hands the work to NumPy, which runs the addition as a tight compiled loop rather than an interpreted Python loop executing line-by-line. Functionally the result is identical to writing the loop yourself, but it is far less error-prone (no index to accidentally mismanage) and, as row counts grow into the thousands, noticeably faster, because the loop is no longer paying Python's per-iteration overhead. Percentage is computed the same way: Total divided by 3 because there are three subjects out of 100 marks each, rounded to two decimal places with .round(2) so Rohit's 71.333... becomes a clean 71.33.

Aggregate Functions: Summarizing a Column

Once a column exists, pandas can summarize the whole thing in one call — no accumulator variable, no manual loop:

print(df['Maths'].mean())   # 72.0
print(df['Maths'].max())    # 95
print(df['Maths'].min())    # 45
print(df['Maths'].sum())    # 360

These all operate "down the column" — axis=0 in the diagram above — combining five separate numbers into one. There's also df['Maths'].describe(), which returns several of these at once (count, mean, standard deviation, minimum, the three quartiles, and maximum) in a single call — for our Maths column that would report a mean of 72.0, a minimum of 45, a median (the 50% mark) of 72.0, and a maximum of 95, along with a standard deviation that measures how spread out the marks are (a topic covered properly in a later statistics chapter).

Filtering With Boolean Masks

Finding "everyone who scored 75% or above" without pandas would mean writing an if check inside a loop and building a new list of matches by hand. In pandas, a comparison on a column produces something called a boolean mask — a Series of True/False values, one per row, that pandas can then use like a stencil:

mask = df['Percentage'] >= 75
print(mask)

# Output:
0     True
1    False
2     True
3    False
4    False
Name: Percentage, dtype: bool

print(df[mask])

# Output:
   Roll    Name  Maths  Science  English  Total  Percentage
0     1  Ananya     88       91       79    258       86.00
2     3   Priya     95       89       92    276       92.00

df['Percentage'] >= 75 compares every value in the column to 75 and returns True or False for each row. df[mask] then keeps only the rows where the mask is True — wherever the stencil has a hole, the row shows through; wherever it doesn't, the row is skipped. You can write this in one line as df[df['Percentage'] >= 75], and this pattern — build a condition, index the DataFrame with it — is the standard way to filter rows in pandas, replacing loops entirely.

Sorting Data

sort_values() reorders rows by a column, without touching the index labels:

print(df.sort_values('Percentage', ascending=False))

# Output:
   Roll     Name  Maths  Science  English  Total  Percentage
2     3    Priya     95       89       92    276       92.00
0     1   Ananya     88       91       79    258       86.00
1     2    Rohit     72       68       74    214       71.33
3     4  Karthik     60       55       65    180       60.00
4     5    Meera     45       50       58    153       51.00

ascending=False gives a rank-list, highest first — exactly what a class teacher preparing a result-topper list would need. Notice, as demonstrated earlier with .loc and .iloc, that the index column on the left (2, 0, 1, 3, 4) stays attached to the original student, not to the new row position — the index is a permanent label, not a position counter.

Grouping: Summarizing by Category

Suppose instead of marks, you're tracking your own UPI spending for a week — a genuinely common Grade 8 use case, since most students now see UPI transaction histories on a parent's phone:

transactions = pd.DataFrame({
    'Date':     ['2026-08-01', '2026-08-02', '2026-08-02', '2026-08-03', '2026-08-04', '2026-08-05'],
    'Category': ['Food', 'Recharge', 'Food', 'Travel', 'Food', 'Recharge'],
    'Amount':   [150, 199, 80, 320, 60, 49]
})

print(transactions.groupby('Category')['Amount'].sum())

# Output:
Category
Food        290
Recharge    248
Travel      320
Name: Amount, dtype: int64

groupby('Category') works in three conceptual steps, often called split-apply-combine: first it splits the six rows into three buckets sharing the same Category value (three Food rows, two Recharge rows, one Travel row); then it applies .sum() to the Amount column within each bucket separately (150+80+60=290 for Food, 199+49=248 for Recharge, 320 for Travel); then it combines the three results into one small summary table. To see the biggest spending category first:

print(transactions.groupby('Category')['Amount'].sum().sort_values(ascending=False))

# Output:
Category
Travel      320
Food        290
Recharge    248
Name: Amount, dtype: int64

This same split-apply-combine pattern is exactly how a school would compute "average Maths mark per Section" from a full-school marks CSV, or how a shop would compute "total sales per product category" from a day's UPI receipts — grouping by a category column and aggregating another column is one of the most frequently used operations in real data analysis.

Handling Missing Data

Real data is rarely complete. A student absent for one test leaves a gap in the marks sheet:

marks_with_gap = pd.DataFrame({
    'Name': ['Ananya', 'Rohit', 'Ibrahim'],
    'Science': [91, 68, None]
})
print(marks_with_gap)

# Output:
      Name  Science
0   Ananya     91.0
1    Rohit     68.0
2  Ibrahim      NaN

Two things happen here worth noticing. First, None becomes NaN ("Not a Number") — pandas' standard marker for a missing value. Second, the whole Science column switches from whole numbers to decimals (91.0, 68.0 instead of 91, 68) — because a column can only hold one data type, and there is no integer version of "missing," pandas is forced to convert the entire column to a floating-point type that can represent NaN. You can detect and handle these gaps directly:

print(marks_with_gap['Science'].isna())

# Output:
0    False
1    False
2     True
Name: Science, dtype: bool

filled = marks_with_gap['Science'].fillna(marks_with_gap['Science'].mean())
print(filled)

# Output:
0    91.0
1    68.0
2    79.5
Name: Science, dtype: float64

isna() returns a boolean mask marking exactly where the gaps are — the same mask idea used earlier for filtering. .mean() automatically ignores NaN when computing the average, so it correctly computes (91+68)/2 = 79.5 from the two real scores, and fillna() then plugs that average into Ibrahim's missing slot. This is one common, simple strategy called mean imputation — filling a gap with the column's average rather than leaving it blank or wrongly treating it as a zero, which would unfairly drag down his average.

Where pandas Fits in Your CBSE Journey

pandas is not a side tool — it is the standard data-handling library taught later in CBSE's own Computer Science and Informatics Practices curriculum, where entire units are devoted to Series and DataFrames, the exact two objects you've just learned here. Building comfort with column selection, filtering, sorting, and grouping now — on a small, five-row table you can check by hand — means that when the same ideas reappear on a fifty-thousand-row dataset in senior school or a competitive exam, the underlying logic is already familiar; only the scale changes.

Active Recall

1. Using the original df (marks, unsorted, indexed 0–4), what does df.iloc[2] return, and does it differ from df.loc[2]? Think it through: since df was never sorted, index label and physical position still match for every row, so both should point to the same student — Priya.
2. Write the single line of code that selects all students who scored below 60% overall in the marks example. Hint: reuse the boolean-mask pattern, but flip the comparison and the threshold: df[df['Percentage'] < 60].
3. If you ran df['Total'].sum() on the marks DataFrame, what number would you expect, and how could you check it by adding the five Total values yourself? 258+214+276+180+153 = 1081 — a good habit is verifying at least one pandas aggregate by hand the first time you use it.
4. In the transactions example, if a seventh row were added with Category "Travel" and Amount 100, what would groupby('Category')['Amount'].sum() report for Travel? 320 + 100 = 420 — grouping recomputes the sum across every row sharing that category, including newly added ones.
5. Why does df['Maths'] return something different in type from df[['Maths']], even though they display similarly? Single brackets select one column as a Series; double brackets pass a list of column names and always return a DataFrame, even with just one column inside it.

Summary

• A pandas DataFrame is a labelled table; a Series is a single labelled column. Every DataFrame column is a Series, and every row is one linked record identified by an index label.
• pd.read_csv() loads real data files; df.shape, df.columns, df.dtypes, and df.head() are the first checks worth running on any new dataset.
• df['col'] returns a Series; df[['col']] returns a one-column DataFrame — a common source of bugs when confused.
• .loc[] selects by index label; .iloc[] selects by physical position. They only look identical when the index has never been reordered.
• Arithmetic on whole columns (df['A'] + df['B']) is vectorized — computed all at once via NumPy, replacing manual for-loops entirely, and scaling far better as row counts grow.
• .mean(), .sum(), .max(), .min(), and .describe() summarize a column (axis=0, down the rows) in one call.
• A boolean mask (df['col'] >= x) marks each row True/False; df[mask] keeps only the True rows — this is how filtering works in pandas.
• sort_values() reorders rows by a column's values without renumbering the index.
• groupby() follows the split-apply-combine pattern: split rows into buckets by a category column, apply an aggregate to each bucket, and combine the results into a summary.
• Missing values appear as NaN; isna() finds them and fillna() — often with the column mean — fills them in.