Two students in the same coding club build a machine learning model to predict apartment prices in Bengaluru. They use the exact same algorithm, the exact same 200 rows of data, and the exact same number of hours tuning it. Student A's model is off by an average of 42 lakh rupees per prediction. Student B's model is off by 6 lakh. Student B did not use a fancier algorithm or a bigger computer. Student B simply prepared the input columns differently before the algorithm ever saw them. That difference — what you feed the algorithm, not just which algorithm you pick — is called feature engineering, and it is often the single biggest factor separating a model that works from one that doesn't.
This chapter teaches you exactly what Student B did, step by step, using real arithmetic you can trace by hand.
What exactly is a "feature"?
In machine learning, a feature is one measurable input column that the algorithm uses to make its prediction. If you're predicting an apartment's price, useful features might include the built-up area in square feet, the number of bedrooms, and the locality. The column you're trying to predict — price, in this case — is called the target or label, and it is not a feature; it's the answer the model is trying to learn to produce.
Here is the problem nobody warns you about: the columns that arrive in a raw dataset are almost never in a form a mathematical model can use directly. A machine learning model, underneath all the terminology, is a mathematical function — it multiplies numbers by weights and adds them up. It cannot multiply a weight by the text string "2 BHK". It cannot make sense of the string "1000 - 1200" sitting in a column that's supposed to be square footage. Feature engineering is the deliberate process of transforming raw, messy, real-world data into clean numeric columns that actually carry the information the model needs — without losing meaning, and without accidentally adding false meaning.
The raw data that breaks a model
Suppose you scraped a small dataset of Bengaluru apartment listings. Three rows look like this:
Row A: total_sqft="1000 - 1200", size="2 BHK", location="Whitefield", price=65 (lakh rupees)
Row B: total_sqft="2400", size="4 BHK", location="Indiranagar", price=210 (lakh rupees)
Row C: total_sqft="800", size="1 BHK", location="Whitefield", price=38 (lakh rupees)
Look closely at what's wrong here. The total_sqft column mixes single numbers with ranges — some field agents measured a flat once, others measured it as a range across similar units in the same building, so the raw text isn't even a consistent format. The size column buries a number inside a string. The location column is pure text with no inherent order or scale. None of these three columns can be fed straight into a model. Fixing them, in order, is what the rest of this chapter walks through: cleaning, extraction, derivation, and encoding/scaling.
Step 1 — Cleaning: turning inconsistent text into one number
The total_sqft column needs a rule that handles both formats and always outputs a plain number. A sensible rule: if the value is a range, take the midpoint; if it's already a single number, use it as-is.
def clean_sqft(sqft_str):
if '-' in sqft_str:
parts = sqft_str.split('-')
low = float(parts[0])
high = float(parts[1])
return (low + high) / 2
else:
return float(sqft_str)
Trace it by hand for Row A: clean_sqft("1000 - 1200"). The string contains a '-', so it splits on that character into ["1000 ", " 1200"]. Converting each part with float() automatically ignores the surrounding spaces, giving low = 1000.0 and high = 1200.0. The function returns (1000.0 + 1200.0) / 2 = 1100.0. For Row B, clean_sqft("2400") has no '-', so it falls straight to float("2400"), giving 2400.0. For Row C, the same path gives 800.0.
Notice what just happened: we didn't just "convert text to a number" — we made a modelling decision (use the midpoint) and encoded it as a repeatable rule. That decision is defensible, but it is a decision, and a different one (e.g., always take the lower bound, because builders round up) would produce a different, equally valid feature. Feature engineering always involves choices like this; there is rarely one "correct" answer, only more or less reasonable ones.
Step 2 — Extraction: pulling a hidden number out of text
The size column stores useful information — the bedroom count — wrapped inside a string. We extract it rather than clean it, because the number was always there; we just need to unwrap it.
def extract_bhk(size_str):
return int(size_str.split(' ')[0])
Trace: extract_bhk("2 BHK") calls .split(' '), which breaks the string at the space into the list ["2", "BHK"]. Index [0] picks out "2", and int("2") converts it to the integer 2. For Row B, "4 BHK" becomes 4. For Row C, "1 BHK" becomes 1. This same pattern — split a string, grab the piece that matters, convert its type — is one of the most common moves in all of feature engineering, because so much real-world data arrives as human-readable text that bundles a number with a unit or a label.
Step 3 — Derivation: creating a brand-new feature that didn't exist before
This is the step that separates feature engineering from mere data cleaning, and it's where Student B's model likely pulled ahead. A derived feature is a new column computed from existing ones, built because the combination carries more predictive signal than either original column alone.
Consider price and cleaned total_sqft. Separately, a model has to learn the relationship between them from scratch, across every row, using up its limited capacity to discover a pattern that is really just division. If we instead hand the model a pre-computed price per square foot, we've done that arithmetic for it and given it a number that's directly comparable across differently sized apartments — a much more informative signal for judging whether a locality is expensive.
def price_per_sqft(price_lakhs, total_sqft):
price_rupees = price_lakhs * 100000
return round(price_rupees / total_sqft, 2)
Trace for Row A: price_per_sqft(65, 1100). First, 65 * 100000 = 6500000 rupees (since 1 lakh = 100,000 rupees). Then 6500000 / 1100 = 5909.0909..., rounded to two decimals: 5909.09 rupees per square foot. For Row B: 210 * 100000 = 21000000, divided by 2400, gives exactly 8750.0. For Row C: 38 * 100000 = 3800000, divided by 800, gives exactly 4750.0.
Look at what this single derived number reveals that was invisible in the raw columns: Row B's apartment is over 45% more expensive per square foot than Row A's, and nearly 85% more than Row C's, even though all three are notionally "in Bengaluru." A locality effect is hiding inside that number, and we haven't even used the location column yet. This is the essence of feature engineering: using domain knowledge — here, simple knowledge of how real estate pricing works — to reshape raw numbers into a form that exposes the pattern the model is being asked to find.
Step 4 — Encoding: converting categories into numbers correctly
Now for location, a text column with no numeric meaning at all. Here is where many beginners make a costly mistake.
Common misconception: "just number the categories"
It's tempting to write Whitefield = 1, Indiranagar = 2, Yelahanka = 3 and be done with it. This is called label encoding, and for a column like location, it is wrong — not inelegant, actually wrong. By assigning these numbers, you've silently told the model that Yelahanka is "three times" Whitefield and that Indiranagar sits exactly halfway between them on some scale. A regression model will happily multiply that fake ordering by a weight and let it distort its predictions, because it has no way to know you didn't mean it. Label encoding is only safe when categories genuinely have an order — for example, an availability column with values "Ready to Move" < "Within 3 months" < "Within 6 months" does have a real ranking, so numbering it 0, 1, 2 is legitimate. Locality names have no such order, so numbering them is not.
The correct tool for unordered categories is one-hot encoding: create one new binary (0 or 1) column per category, and mark a 1 only in the column matching that row's actual value.
def one_hot(value, categories):
return [1 if value == c else 0 for c in categories]
Across our three rows, the unique locations are Indiranagar and Whitefield, so categories = ["Indiranagar", "Whitefield"] (sorted alphabetically, so the order is fixed and repeatable). Trace one_hot("Whitefield", categories): the list comprehension checks each category in order — is "Whitefield" == "Indiranagar"? No, so 0. Is "Whitefield" == "Whitefield"? Yes, so 1. Result: [0, 1]. For Row B, one_hot("Indiranagar", categories) gives [1, 0]. For Row C, same as Row A: [0, 1]. Now every row has two new numeric columns — loc_Indiranagar and loc_Whitefield — and no fake ordering has been introduced. The trade-off, which you should notice honestly: a dataset with 40 different localities becomes 40 new columns. This is a real cost of one-hot encoding (it can make your feature table very wide), but it is the price of encoding categories truthfully.
Step 5 — Scaling: putting every feature on comparable footing
After cleaning, extraction, derivation, and encoding, look at the range each feature spans: total_sqft ranges into the thousands, bhk sits between 1 and 5, and the one-hot columns are just 0 or 1. Many algorithms — especially ones that measure distance between data points, like k-nearest neighbours, or ones that use gradient descent to learn, like linear regression trained iteratively — are sensitive to these differences in scale. A change of 1 in total_sqft is numerically tiny compared to its own range, but a change of 1 in bhk is enormous relative to its range. Left uncorrected, the algorithm can end up treating square footage as "more important" purely because its raw numbers are bigger, which has nothing to do with its actual predictive value.
Two standard fixes: min-max scaling, which squeezes every value into the range 0 to 1, and standardization (z-scoring), which re-centres values around a mean of 0 measured in units of standard deviation.
def min_max_scale(value, min_val, max_val):
return (value - min_val) / (max_val - min_val)
def z_score(value, mean, std_dev):
return (value - mean) / std_dev
Trace min_max_scale using our three cleaned sqft values (800, 1100, 2400), so min_val = 800 and max_val = 2400. For Row A: (1100 - 800) / (2400 - 800) = 300 / 1600 = 0.1875. For Row B: (2400 - 800) / 1600 = 1600 / 1600 = 1.0 — the maximum value always scales to exactly 1. For Row C: (800 - 800) / 1600 = 0.0 — the minimum always scales to exactly 0.
Trace z_score with a separate example: suppose across a larger dataset the sqft column has mean 1450 and standard deviation 600, and we want to score Row A's value of 1100. (1100 - 1450) / 600 = -350 / 600 = -0.5833, rounding to -0.58. This tells us Row A's apartment is a little more than half a standard deviation smaller than the average listing — a meaningful statement that the raw number 1100 doesn't make on its own.
The full pipeline, traced end to end
Putting every step together on our three rows produces this final, fully numeric feature table (target price is kept aside, not used as an input feature):
Row sqft_scaled bhk price_per_sqft loc_Indiranagar loc_Whitefield (price, not a feature)
A 0.1875 2 5909.09 0 1 65
B 1.0 4 8750.00 1 0 210
C 0.0 1 4750.00 0 1 38
Every single value in that table traces back, by a rule you could re-derive by hand, to the original messy text. That traceability matters: if the model performs badly, you can walk backward through this exact pipeline to find out whether the problem is a bad rule (maybe the midpoint assumption for sqft ranges was wrong) rather than treating the model as an unexplainable black box.
The diagram below shows this same pipeline as a single picture — five transformations turning unusable raw text into a model-ready row of numbers.
Common misconception: "more features always make a smarter model"
It feels intuitive that giving a model more information should only help. In practice, piling on features that are irrelevant, redundant, or measured with noise usually makes the model worse, not better. Three concrete reasons this happens. First, a redundant feature like adding both total_sqft and price_per_sqft alongside a third column that is just total_sqft multiplied by 10.76 (to convert to square metres) tells the model nothing new but gives it more numbers to get confused by. Second, an irrelevant feature — say, the listing agent's phone number, converted to a number — has no real relationship with price, but with a small dataset, the model can accidentally "discover" a coincidental pattern in it and rely on that noise, which then fails badly on new data. This is called overfitting, and unnecessary features are one of its most common causes. Third, every additional feature is one more dimension the model has to search across, and with a fixed, limited amount of training data, spreading that same data thinly across more dimensions makes every pattern harder to detect reliably — a phenomenon informally called the "curse of dimensionality." The actual goal of feature engineering is not maximum information; it is maximum useful information per feature, which sometimes means deliberately leaving a raw column out.
Where this fits in the AI project cycle
If you've studied the AI Project Cycle taught in CBSE's Artificial Intelligence curriculum — Problem Scoping, Data Acquisition, Data Exploration, Modelling, Evaluation — feature engineering is the core technical work of the Data Exploration stage. It happens after data is acquired but strictly before modelling begins, because a model trained on badly engineered features cannot be rescued afterward by a better algorithm; the information loss or the false signal introduced at this stage propagates into every prediction the model ever makes. This is also why, in real projects, data scientists commonly report spending far more time preparing features than they spend training or tuning the model itself.
Practice: test your understanding
Work through each question before checking the reasoning that follows it.
1. A dataset has a column date_of_birth with values like "2011-05-14". You want to predict a student's exam performance. What single derived feature would typically be far more useful to a model than the raw date string, and why?
Reasoning: the raw date string has no direct numeric relationship with performance, but age (roughly, current year minus birth year) is a meaningful number the model can actually use in comparisons and arithmetic — this is derivation, exactly like turning total_sqft and price into price_per_sqft.
2. True or false: assigning Delhi = 1, Mumbai = 2, Bengaluru = 3 in a city column is a safe way to make the column numeric.
Reasoning: false. Cities have no real order, so this is label encoding applied to an unordered category, which invents a fake ranking the model will wrongly treat as meaningful. One-hot encoding is the correct choice here.
3. Given total_sqft = "1400 - 1600", use the clean_sqft rule from this chapter to compute the cleaned value.
Reasoning: split on '-' to get 1400 and 1600, average them: (1400 + 1600) / 2 = 1500.0.
4. Across a dataset, sqft ranges from a minimum of 500 to a maximum of 3000. Using min-max scaling, what value does sqft = 1250 map to?
Reasoning: (1250 - 500) / (3000 - 500) = 750 / 2500 = 0.3.
5. Why can adding more features to a model sometimes make it perform worse on new, unseen data?
Reasoning: irrelevant or redundant features can let the model latch onto coincidental noise in the training data (overfitting) instead of the real underlying pattern, and spreading a fixed amount of training data across more dimensions makes genuine patterns statistically harder to find.
Summary
- A feature is a single measurable input column a model uses to predict a target; raw data almost never arrives in a form a model can use directly.
- Cleaning converts inconsistent formats (like sqft ranges mixed with single values) into one consistent numeric rule.
- Extraction pulls a number that already exists, hidden inside a text string (like the bedroom count inside
"2 BHK"), out into its own column. - Derivation creates an entirely new feature by combining existing ones (like
price_per_sqft), often exposing a pattern — such as locality-driven price differences — that no single raw column showed on its own. - Encoding turns unordered categories into numbers without inventing a false ranking; one-hot encoding is the safe default, while label encoding (0, 1, 2, ...) is reserved for categories that truly have an order.
- Scaling (min-max or z-score standardization) puts every feature on comparable numeric footing so that distance- and gradient-based algorithms don't wrongly favour a feature just because its raw numbers happen to be larger.
- More features are not automatically better: irrelevant or redundant features can cause overfitting and make patterns harder to detect, so every feature engineering decision should be justified, not just added for volume.
- Feature engineering sits in the Data Exploration stage of the AI Project Cycle, strictly before modelling, because errors introduced here cannot be fixed by a better algorithm afterward.
Think About It
Think about this: How would you explain feature engineering: the art of data preparation to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.