AI Computer Institute
Expert-curated CS & AI curriculum aligned to CBSE standards. A bharath.ai initiative. About Us

Data Collection and Cleaning: Garbage In, Garbage Out

📚 Introduction to Machine Learning⏱️ 21 min read🎓 Grade 9
✍️ AI Computer Institute Editorial Team Updated: August 2026 CBSE-aligned · Peer-reviewed · 21 min read
Content curated by subject matter experts with IIT/NIT backgrounds. All chapters are fact-checked against official CBSE/NCERT syllabi.

Meena's Google Form and a Prediction That Made No Sense

Meena, a Class 9 student, wants to build a tiny machine learning model that predicts a classmate's exam marks from the number of hours they study per day. It sounds simple: collect some data, find the pattern, let the computer learn it. She sends a Google Form to her 8 closest classmates asking two questions — "How many hours do you study per day?" and "What did you score in the last test (out of 100)?" — and pastes the responses into a table.

id  hours   marks
 1    2      55
 2    3      60
 3    ?      65     <- this student left the hours field blank
 4    4      70
 5    4      70     <- identical to row 4, submitted twice by mistake
 6    5      78
 7    1      40
 8    6     950     <- marks is out of 100 — this is impossible

Meena is impatient, so she skips straight to the "exciting" part: she asks her program to compute the average marks scored across her class, treating this table as-is. She gets 173.5 out of 100. That number cannot exist — no exam gives more than 100 marks — yet her program produced it confidently, because a computer does not know that 950 is a typo. It only knows how to add and divide. If Meena had trained an actual prediction model on this table without looking closely at it first, the model would have quietly absorbed that broken 950 as a real data point and skewed every prediction it ever made afterward.

This is the entire chapter in one small disaster. Before you can teach a machine to find patterns, you have to make sure the data you hand it actually reflects reality. Getting that right — and fixing it when it doesn't — is called data collection and cleaning, and the principle behind why it matters has a name every AI practitioner uses constantly: Garbage In, Garbage Out (GIGO).

What "Garbage In, Garbage Out" Actually Means

GIGO is often repeated as a vague slogan — "bad data gives bad results" — without explaining the mechanism. Here is the mechanism. A machine learning model does not know anything about the real world beyond what sits in its training table. It does not know that a test is out of 100, that a person cannot be −5 years old, or that "CSK" and "Chennai Super Kings" refer to the same team. It treats every number and every symbol in the dataset as ground truth and searches for a mathematical rule that fits those exact values as closely as possible.

So when Meena's table contains a duplicate row, the model effectively believes that combination of hours and marks is twice as important as it should be, because it appears twice — this is called duplication bias. When it contains a missing value, many algorithms either crash, silently drop the entire row (throwing away the marks value too, which was perfectly valid), or fill the gap with a lazy default like zero, which is worse than having no answer at all. When it contains an impossible value like 950, the model doesn't reject it as impossible — it tries to find a pattern that explains why "6 hours of study" sometimes leads to "950 marks," which drags its whole understanding of the relationship between hours and marks off course. The model isn't "confused" the way a person would be; it is doing exactly what it's designed to do — fit the data it was given — and the data was wrong. Garbage in, garbage out is not a metaphor. It is a direct, mechanical consequence of how these algorithms work: they have no concept of "this looks wrong," only "this is what I was told."

Where the Data Comes From: Collection Methods

Before you can clean data, you need to understand how it was collected, because different collection methods introduce different kinds of problems. Data collection is usually split into two categories:

  • Primary data — collected first-hand, for your specific purpose. Meena's Google Form is primary data. So is a temperature sensor on a weather balloon, a camera on a traffic signal counting vehicles, or a shopkeeper logging every sale in a register.
  • Secondary data — data someone else already collected, that you reuse. A dataset of past IRCTC train punctuality records published by the Ministry of Railways, or a public dataset of India's rainfall by district from the India Meteorological Department (IMD), is secondary data if you didn't collect it yourself.

Within primary collection, the common methods are surveys and forms (like Meena's), sensors and IoT devices (a smart electricity meter logging your home's power usage every 15 minutes), automated logs (an app recording every button you tap, or a UPI app logging every transaction), APIs that pull data from another system, and manual observation (a researcher physically counting footfall at a market). Each method has a characteristic failure mode. Surveys suffer from people skipping questions or misreading them. Sensors suffer from hardware glitches, power cuts, and network dropouts that create gaps. Logs suffer from duplicate events when a system retries a failed request. Manual observation suffers from human miscounting and fatigue.

There's a second, subtler problem in collection that no amount of cleaning can fix afterward: sampling bias. Suppose Meena had only sent her form to the top 8 scorers in her class instead of a random mix. Her data would be perfectly clean — no missing values, no duplicates, no impossible numbers — and still useless, because it would only describe how hours-vs-marks behaves for strong students, not for the class as a whole. A cleaned dataset built from a biased sample still produces a biased model. This is why the collection step and the cleaning step are both part of the same responsibility: cleaning fixes damaged data, but only good collection design can prevent the data from misrepresenting the population in the first place. CBSE's Artificial Intelligence curriculum (Subject Code 417) reflects this by placing Data Acquisition as its own explicit stage in the AI Project Cycle, right after Problem Scoping and before Data Exploration — collection is treated as a design decision, not just a data-entry chore.

Four Ways Data Goes Bad

Meena's 8-row table happens to contain a clean example of each of the four most common data quality problems you will meet again and again in real datasets.

1. Missing values. Row 3's "hours" field is blank. This happens when a form field is optional, a sensor loses connection, or a page in a paper survey is torn. You cannot simply pretend the field says zero — a blank is not the same statement as "this student studied zero hours." Zero would be a false claim you invented; a blank is an honest admission that the value is unknown.

2. Duplicates. Rows 4 and 5 are identical — same hours, same marks. This typically happens when a form is submitted twice (a double-click, a page refresh), or when the same sensor reading gets logged by two overlapping systems. Duplicates silently give one real observation extra voting power in any average, count, or pattern the model computes.

3. Outliers and errors — and why they are not the same thing. Row 8's marks value of 950 is not just "unusual," it is impossible — it violates the known rule that marks are out of 100. That makes it an error, and errors should be corrected or removed. But suppose instead row 8 read "1 hour, 95 marks" — a student who studied very little yet scored very high. That value is unusual, but it is not impossible. It is a genuine outlier: a real, valid data point that happens to sit far from the rest. Deleting genuine outliers just because they are inconvenient throws away real information — maybe that student already knew the topic from a coaching class. Confusing "unusual" with "wrong" is one of the most common data-cleaning mistakes beginners make.

4. Inconsistent formatting. Meena's table doesn't show this, but it is extremely common in real datasets: the same fact written in different forms. A "team" column might contain "CSK," "Chennai Super Kings," and "chennai super kings" as three different-looking values that actually mean one thing. A "date" column might mix "13/08/2026" and "August 13, 2026." A "study hours" column might mix hours and minutes without saying which. Left uncleaned, a model literally cannot tell that "CSK" and "Chennai Super Kings" belong to the same group — it will treat them as two unrelated categories and split your data (and your pattern-finding power) between them.

Cleaning Meena's Dataset, Step by Step

Good cleaning is not "delete anything that looks weird." It is a deliberate, ordered sequence of checks, each with a clear rule for what to do when a check fails. Here is Meena's dataset represented as a list of Python dictionaries, and the three cleaning steps applied to it in order.

students = [
    {"id": 1, "hours": 2,    "marks": 55},
    {"id": 2, "hours": 3,    "marks": 60},
    {"id": 3, "hours": None, "marks": 65},
    {"id": 4, "hours": 4,    "marks": 70},
    {"id": 5, "hours": 4,    "marks": 70},
    {"id": 6, "hours": 5,    "marks": 78},
    {"id": 7, "hours": 1,    "marks": 40},
    {"id": 8, "hours": 6,    "marks": 950},
]

# Step 1: remove exact duplicate rows, keeping the first occurrence
seen = set()
deduped = []
for row in students:
    key = (row["hours"], row["marks"])
    if key not in seen:
        seen.add(key)
        deduped.append(row)
# deduped now has 7 rows: row 5 was dropped because (4, 70)
# was already seen when row 4 was processed.

# Step 2: fix impossible values using a domain rule (marks must be 0-100)
for row in deduped:
    if row["marks"] is not None and row["marks"] > 100:
        row["marks"] = row["marks"] / 10   # 950 was a misplaced-decimal typo -> 95.0

# Step 3: fill the missing "hours" value using the mean of the known values
known_hours = [row["hours"] for row in deduped if row["hours"] is not None]
mean_hours = sum(known_hours) / len(known_hours)
for row in deduped:
    if row["hours"] is None:
        row["hours"] = mean_hours

mean_marks = sum(row["marks"] for row in deduped) / len(deduped)
print(mean_hours, mean_marks)

Let's trace this by hand, the way you'd be expected to on a CBSE exam. Step 1 walks through all 8 rows and builds a "seen" set of (hours, marks) pairs. Row 4 produces the key (4, 70) and adds it to the set. Row 5 produces the exact same key (4, 70) — since it's already in the set, row 5 is skipped. Every other row has a unique key, so all of them survive. Result: 7 rows remain (ids 1, 2, 3, 4, 6, 7, 8).

Step 2 checks every row's marks against the rule "marks cannot exceed 100." Only row 8, with marks = 950, breaks this rule. Since a common real-world cause of "extra digit" typos is a misplaced decimal or an accidental keystroke, we correct it to 950 / 10 = 95.0 — a plausible, in-range value — rather than deleting the whole row and losing a valid hours reading.

Step 3 collects every row's hours value except row 3's (which is None), giving the list [2, 3, 4, 5, 1, 6]. That list sums to 21 and has 6 entries, so mean_hours = 21 / 6 = 3.5. Row 3's missing hours field is then filled with 3.5 — not a guess pulled from nowhere, but the best single estimate available from the rest of the class, a technique called mean imputation.

The final, cleaned marks values are 55, 60, 65, 70, 78, 40, 95 — seven numbers, all inside the valid 0–100 range. Their sum is 463, so mean_marks = 463 / 7 ≈ 66.14. Compare that to the 173.5 Meena got from the raw, unchecked table. The raw average wasn't just "a bit off" — it was a number that couldn't possibly represent a real percentage, and any model trained on it would have inherited that same broken sense of scale. The cleaned average, 66.14, is a number you could sanity-check against real classroom marks and actually trust.

Two Paths for the Same Raw Data Data Collection Surveys, sensors, logs, APIs Raw Data (messy) id hours marks 1 2 55 2 3 60 3 ? 65 missing 4 4 70 5 4 70 duplicate 6 5 78 7 1 40 8 6 950 impossible Naive average of "marks" here = 173.5 out of 100 — impossible! Cleaning Steps 1. Remove exact-duplicate rows 2. Check values against known limits, fix real errors 3. Fill missing values sensibly (e.g. mean imputation) 4. Standardize formats/units so equal values match Keep genuine outliers — they carry real information. Clean Dataset 7 rows, all values in valid range ML Model Trains learns hours -> marks pattern Reliable Prediction (mean = 66.1) Skip Cleaning Model trains on 950, duplicates, and a blank field taken as 0 Garbage In -> Garbage Out Predicted marks: meaningless, possibly >100 or negative

Misconception 1: "If I Just Collect More Data, the Model Will Improve"

This feels intuitive because it's true in one narrow sense: more data reduces the effect of random noise, since random errors tend to cancel out when averaged over a large enough sample. But it is false for systematic problems — errors that repeat in the same direction every time. Imagine Meena's form has a bug that silently drops the decimal point from every marks value above 90 (turning 95 into 950). That is not a rare, random accident — it will happen to every single high-scoring student who fills the form, forever, no matter how many responses she collects. Sending the form to 800 students instead of 8 does not fix this; it just produces 800 rows with the same 1-in-8 corruption rate, at a much larger, harder-to-spot scale. Data quality problems caused by a flawed collection method scale up together with the data — they never average themselves away. This is why an ML engineer's first instinct when a model performs badly should never be "get more data" — it should be "look at the data I already have."

Misconception 2: "Any Extreme Value in the Data Should Be Deleted"

We already touched on this, but it's worth stating directly because it trips up almost every beginner: an outlier is not automatically wrong. A student who studies 1 hour a day and scores 95 marks is statistically unusual, but perfectly real — maybe they attend a strong coaching class, maybe the syllabus overlaps with a subject they already love. That data point is telling you something true and important: hours-studied is not the only factor driving marks. Deleting it to make your average "look cleaner" doesn't clean your data — it deletes information and quietly makes your model less honest about how the real world behaves. The correct question is never "does this look weird?" — it is "does this violate a rule I know must be true?" A mark of 950 violates the rule "marks are out of 100." A mark of 95 from a 1-hour-a-day student violates no such rule; it is just surprising.

Check Your Understanding

  • A weather sensor in a rural IMD station loses power for 3 hours every day at the same time, so those readings are always missing. Would collecting data from this sensor for 5 years instead of 5 days fix the gap problem? Explain why or why not, using the systematic-vs-random idea from this chapter.
  • A dataset of 10 UPI transactions has amounts (in rupees): 200, 450, 450, 300, 5000000, 150, 600. One value is a duplicate and one is almost certainly a data-entry error (an extra zero or two). Identify both, propose a fix for each, and compute the mean transaction amount before and after cleaning.
  • A survey column called "city" contains the entries "Bengaluru," "Bangalore," and "bengaluru" for the same city. What kind of data quality problem is this, and what cleaning step fixes it? Why would leaving it uncleaned hurt a model that groups data by city?
  • A classmate says, "This value of 2 hours studied and 95 marks looks suspicious, I'm deleting it to clean the data." Explain what question they should ask before deleting it, and under what condition it would actually be correct to remove or investigate it further.
  • Using the imputation method shown in this chapter (fill a missing value with the mean of the known values), what would you fill in for a missing "hours" value if the known hours in the rest of the dataset were 2, 2, 2, 2, and 20? Compute the mean, then explain briefly why mean imputation can be misleading when one real value (20) is very different from the rest.

Summary

A machine learning model can only be as trustworthy as the data it learns from, because it has no way to independently know what's true — it accepts every row of its training table as fact. Data collection methods (surveys, sensors, logs, APIs, manual observation) each introduce characteristic problems, and a biased sample stays biased no matter how carefully you clean it afterward, which is why CBSE's AI Project Cycle treats Data Acquisition as its own deliberate design stage. Once data is collected, four problems recur constantly: missing values, duplicate rows, outliers/errors, and inconsistent formatting — and the crucial skill is telling a genuine outlier (unusual but real, worth keeping) apart from an error (impossible given a known rule, worth fixing or removing). Cleaning is not random deletion; it is an ordered sequence of checks — deduplicate, validate against known limits, impute missing values sensibly, standardize formats — each traceable and justifiable, as we did by hand for Meena's dataset, turning an impossible 173.5-out-of-100 average into a trustworthy 66.14. And more data is not a substitute for clean data: random noise shrinks as your sample grows, but a systematic collection flaw grows right along with it. Garbage In, Garbage Out isn't a warning label — it's a precise description of what a model does with whatever you give it.

Practice Exercises

Now it is time to practice! Complete these challenges to solidify your understanding:

  • Exercise 1: Write a short program that demonstrates the core concept from this chapter. Test it with at least 3 different inputs.
  • Exercise 2: Find a real-world example where data collection and cleaning: garbage in, garbage out is used in an Indian company (like TCS, Infosys, Flipkart, or ISRO). Write a paragraph explaining the connection.
  • Exercise 3: Create a mind-map connecting data collection and cleaning: garbage in, garbage out to at least 3 other topics you have studied.
← Supervised vs Unsupervised Learning: Two ApproachesYour First ML Classifier: Building a Spam Filter →

Found this useful? Share it!

📱 WhatsApp 🐦 Twitter 💼 LinkedIn