Regular Expressions in Python

Suppose your school's admission office gets a text message every time a parent replies to the Class 8 orientation SMS. Some replies look like "Confirming, my number is 9876543210", others like "call 8123456789 after 6pm", and a few just have the number with no words around it at all. The office wants a program that reads a thousand such messages and pulls out every 10-digit mobile number, automatically, without a human reading each one.

Try to do this with the string tools you already know — .find(), .split(), slicing — and you hit a wall immediately. text.find("9876543210") only works if you already know the exact number you're looking for. But you don't know it in advance. Every message has a different number. What you know is not the number itself, but its shape: ten digits in a row, usually starting with 6, 7, 8, or 9 (that's how Indian mobile numbers are allocated). You need a way to search for a shape, not a fixed piece of text. That is exactly the problem regular expressions solve.

A Pattern Is a Description, Not a Copy

Think of it like a form at a government office — say, a PIN code box on an IRCTC address form. The box doesn't demand one specific PIN code; it demands "exactly six digits." Any sequence of six digits fits the box; "Delhi" does not, and "12345" (only five digits) does not either. The box describes a pattern of valid input, not a single valid value.

A regular expression, usually shortened to regex, is a tiny language for writing exactly this kind of description, so a program can check text against it. In Python, this language is used through the built-in re module. Here is the mobile-number problem solved with it:

import re

text = "Confirming, my number is 9876543210, thanks!"
match = re.search(r"\d{10}", text)
print(match.group())   # 9876543210

re.search() scans through text looking for the first place where the pattern \d{10} fits, and returns a match object if it finds one (or None if it doesn't). \d means "any single digit," and {10} means "exactly ten times in a row." Read together, \d{10} says "ten digits, one after another" — precisely the shape we wanted, without ever writing down 9876543210 anywhere in our code.

Notice the r right before the opening quote: r"\d{10}". This is a raw string, and it matters more than it looks. In ordinary Python strings, a backslash starts an escape sequence — "\n" is a newline, "\t" is a tab. Regex patterns use the backslash constantly for their own purposes (\d, \s, \b and more), and Python's string parser doesn't know these mean anything special to re, so it tries to interpret them as escape sequences first. For sequences like \d that aren't valid Python escapes, recent Python versions will even print a warning telling you to use a raw string instead. A raw string tells Python "leave every backslash exactly as I typed it," which is what the re module needs to see. Rule of thumb for this whole chapter: every regex pattern gets an r in front of it.

The Building Blocks: Character Classes

Before combining anything into full patterns, you need the individual symbols a regex is built from. These fall into two families: symbols that stand for a type of character, and symbols that control how many times something repeats.

The most-used character classes are:

• \d — any digit, 0 through 9. Its opposite, \D, matches any character that is not a digit.
• \w — any "word character": letters (upper or lower case), digits, and the underscore. \W is everything else — spaces, commas, punctuation.
• \s — any whitespace: space, tab, or newline. \S is any non-whitespace character.
• . (a plain dot) — any single character at all, except a newline.
• [6-9] — a custom set, meaning "any one character from 6 to 9." You can also write [abc] for "a, b, or c," or [A-Za-z] for "any letter, upper or lower case."

Here is exactly where most beginners trip: the dot . does not mean "anything," it means "any one character," and it deliberately excludes the newline. Watch what happens when we test the pattern c.t against several strings glued together with spaces and one newline:

import re
sample = "cat cot c8t c\nt cut"
print(re.findall(r"c.t", sample))
# ['cat', 'cot', 'c8t', 'cut']

Four matches are found — cat, cot, c8t, cut — but not the "c" followed by a newline followed by "t" in the middle of the string. The dot happily matched a letter, another letter, and even a digit sitting between the c and the t, because all of those are "one character." But it refused to match across the newline. This is a real source of bugs when people scan multi-line text (like a paragraph pasted from a PDF) expecting . to sweep across line breaks — by default in Python's re, it never does.

Quantifiers: How Many Times?

A character class says what kind of character to expect; a quantifier, written right after it, says how many of them in a row:

• * — zero or more times
• + — one or more times (at least one must appear)
• ? — zero or one time (makes something optional)
• {n} — exactly n times
• {m,n} — between m and n times (inclusive)

So \d{10} is "exactly 10 digits," \d+ is "one or more digits" (any length), and \d{6,10} is "between 6 and 10 digits." Let's see {m,n} in action, because it reveals something important about how greedy the engine is:

import re
print(re.findall(r"a{2,4}", "a aa aaa aaaa aaaaa"))
# ['aa', 'aaa', 'aaaa', 'aaaa']

Trace this by hand. The single "a" at the start never satisfies "at least 2," so it's skipped entirely — it doesn't even appear in the output. "aa" gives exactly 2. "aaa" gives exactly 3. "aaaa" gives exactly 4. Then comes "aaaaa", a run of five a's: the engine is greedy by default, meaning it grabs as many repetitions as the quantifier allows — up to 4 — and stops there, leaving the fifth a behind, unmatched, because a lone a doesn't satisfy {2,4} on its own. That's why the printed list has four entries, the last one only 4 a's long, not 5.

Anchors: Pinning a Pattern to a Position

So far our patterns can match anywhere inside a string. Sometimes you need to insist that the match happens at a specific position — the very start, the very end, or the whole string top to bottom. Three tools do this:

• ^ — anchors to the start of the string
• $ — anchors to the end of the string
• \b — a "word boundary": the invisible edge between a word character and a non-word character (or the edge of the string)

The word boundary matters more than it seems. Suppose you search a sentence for the word "cat":

import re
sentence = "The cat scattered the cats"
print(re.findall(r"cat", sentence))
# ['cat', 'cat', 'cat']
print(re.findall(r"\bcat\b", sentence))
# ['cat']

Without boundaries, the plain pattern cat matches inside "scattered" and inside "cats" too, because it's just looking for those three letters appearing anywhere, in any surrounding context. It finds three matches: the standalone "cat", the "cat" hiding inside s-cat-tered, and the "cat" inside cat-s. Adding \b on both sides forces the match to have a word-character-to-non-word-character edge immediately before and after — which only the standalone "cat" satisfies, since scattered has letters on both sides of its "cat" and cats has a letter (s) immediately after. This is exactly the difference between "contains this text" and "is this exact word."

match() vs search() vs fullmatch() — A Common Mix-Up

Python's re module gives you three closely related functions, and mixing them up is one of the most common beginner mistakes:

• re.match(pattern, text) — checks only at the very beginning of the string. If the pattern doesn't fit starting at position 0, it fails, even if the pattern would fit somewhere later in the string.
• re.search(pattern, text) — scans the whole string, left to right, and returns the first place the pattern fits, wherever that is.
• re.fullmatch(pattern, text) — the strictest of the three: the pattern must match the entire string, start to end, with nothing left over on either side.

Here is the misconception, made concrete. A student assumes re.match() works like re.search() and is confused when it "randomly" fails:

import re
s1 = "9876543210 is my number"
s2 = "My number is 9876543210"

print(re.match(r"\d{10}", s1))
# <re.Match object; span=(0, 10), match='9876543210'>
print(re.match(r"\d{10}", s2))
# None
print(re.search(r"\d{10}", s2))
# <re.Match object; span=(13, 23), match='9876543210'>

In s1, the ten digits sit right at position 0, so match() succeeds. In s2, the sentence starts with the letter "M", so match() gives up immediately without even looking further into the string — it returns None. Yet the number is clearly present in s2, just not at the start; search() finds it fine, starting at index 13. The rule to remember: use search() when the pattern could be anywhere in the text (the usual case), and reach for match() only when you specifically need "starts with."

Putting It Together: Validating an Indian Mobile Number

Now we can build a real validator, not just a finder. A valid 10-digit Indian mobile number must start with 6, 7, 8, or 9, followed by exactly 9 more digits — and nothing else should be attached before or after it (no extra digits, no letters glued on):

import re

pattern = r"^[6-9]\d{9}$"

for number in ["9876543210", "1234567890", "98765432100", "987654321"]:
    is_valid = bool(re.fullmatch(pattern, number))
    print(number, is_valid)

# 9876543210  True
# 1234567890  False   (starts with 1, not 6-9)
# 98765432100 False   (11 digits, too long)
# 987654321   False   (9 digits, too short)

Read the pattern left to right, the way the regex engine reads it: ^ plants a flag at the start. [6-9] demands the very next character be 6, 7, 8, or 9. \d{9} then demands exactly nine more digits, of any value. $ plants a flag at the end, refusing to let anything trail after those nine digits. Chained together, the whole pattern accepts a string only if it is exactly "one digit from 6-9, then nine more digits, then nothing else." re.fullmatch() checks the pattern against the whole string in one go, which is why we use it here instead of search() — we're not looking for the number buried in other text, we're validating that an entire input is a number, the same job a sign-up form does when you type your mobile number into it.

The diagram below traces this same pattern against two example strings, character by character, the way the regex engine actually walks through them:

Capturing Groups: Pulling Out the Parts You Care About

Sometimes you don't just want to know whether something matched — you want to pull specific pieces out of it. Parentheses () create a capturing group: a labelled sub-section of the pattern whose matched text you can retrieve separately. This is exactly how you'd pull the day, month, and year out of a date written the Indian way, DD-MM-YYYY:

import re

text = "Independence Day: 15-08-1947, Republic Day: 26-01-1950"
dates = re.findall(r"(\d{2})-(\d{2})-(\d{4})", text)
print(dates)
# [('15', '08', '1947'), ('26', '01', '1950')]

Notice what changed compared to re.findall(r"\d+", text), which would simply return a flat list of number-strings. Because our pattern has three groups in parentheses, findall() now returns a list of tuples, one tuple per full match, each tuple holding exactly what each parenthesised group captured — the day, then the month, then the year, in the order the groups appear in the pattern. This is a general rule worth memorising: whenever a pattern passed to findall() contains groups, the result is tuples of the captured pieces, not the whole match.

When you only need one match and want to refer to its pieces by name instead of position, use a named group, written (?P<name>...):

import re

text = "Independence Day: 15-08-1947"
m = re.search(r"(?P<day>\d{2})-(?P<month>\d{2})-(?P<year>\d{4})", text)
print(m.group("year"), m.group("month"), m.group("day"))
# 1947 08 15

This reads far more clearly than remembering "group 3 is the year" — useful when a pattern has many groups, and standard practice in real code that parses structured text like log files, roll numbers, or admit-card IDs.

Greedy vs Lazy Matching: The Second Big Misconception

Quantifiers like * and + are greedy by default: they try to match as much text as possible, and only back off if the overall pattern would otherwise fail. This causes a specific, predictable kind of surprise when your pattern uses .* to span "everything in between" two markers. Consider a snippet of HTML with two tags:

import re

html = "<b>bold</b> and <i>italic</i>"
print(re.findall(r"<.*>", html))
# ['<b>bold</b> and <i>italic</i>']

print(re.findall(r"<.*?>", html))
# ['<b>', '</b>', '<i>', '</i>']

A beginner writing <.*> usually expects it to grab one tag at a time: <b>, then </b>, and so on. Instead it grabs the entire string in one giant match, from the very first < all the way to the very last >. Why? Because .* is greedy: it first tries to consume the whole rest of the string, then checks whether a > follows — and since the string does end in >, that greedy attempt succeeds immediately, so the engine never backs off to a shorter match. Adding a ? right after the *, giving .*?, flips it to lazy (also called non-greedy) matching: now it tries to consume as little as possible, checking after every single character whether a > follows, stopping at the very first opportunity. That produces the four separate tag matches most people actually want. The lesson: whenever you use .* or .+ between two markers and get one suspiciously huge match instead of several small ones, greediness is almost always the cause — and .*? is almost always the fix.

Two More Everyday Tools: sub() and split()

Besides finding and validating, regex is commonly used to change text. re.sub(pattern, replacement, text) replaces every match of pattern with replacement — useful, for instance, when masking a mobile number the way banking apps do before displaying it:

import re
masked = re.sub(r"\d{6}$", "XXXXXX", "9876543210")
print(masked)
# 9876XXXXXX

Here \d{6}$ means "the last six digits" (six digits, anchored to the end of the string), so sub() finds that trailing chunk and swaps it for XXXXXX, leaving the first four digits, 9876, untouched — exactly the masking pattern you see on a UPI app's confirmation screen.

re.split(pattern, text) is like the ordinary .split() you already know, except the separator can be a pattern instead of one fixed character. This matters for real-world messy data, where the gap between fields isn't always a single consistent character:

import re
line = "Aditi,15,Delhi   Rohan,14,Mumbai"
print(re.split(r"\s{2,}", line))
# ['Aditi,15,Delhi', 'Rohan,14,Mumbai']

Splitting on \s{2,} ("two or more whitespace characters in a row") correctly separates the two student records at the run of spaces between them, while leaving the single spaces inside each record (if there were any) untouched — something a plain line.split(" ") could not do cleanly, since it would also break apart wherever a single space happens to sit.

Where This Fits: CBSE and Beyond

Pattern-based text processing is a core idea in Computer Science and Informatics Practices — input validation (checking that a form field "looks like" a PIN code, an email, or a mobile number before accepting it) is one of the most frequently asked practical-file style questions once you reach Class 11 and 12 CS/IP, where the full re module is used more heavily for text and file processing. Everything you've learned here — character classes, quantifiers, anchors, groups — is exactly the vocabulary those later chapters build on; nothing here is a simplified toy version that will need to be re-learned. Getting comfortable now with reading a pattern character-by-character, the way the diagram above traces it, is what makes those later, longer patterns readable instead of intimidating.

Check Your Understanding

1. What does re.match(r"\d{4}", "The year 2026") return, and why?
   Answer: None. match() only checks starting at position 0, and the string starts with "T", not a digit. re.search() would find "2026" instead, since it scans the whole string.
2. Write a pattern that matches a 6-digit Indian PIN code and nothing else (no extra digits before or after).
   Answer: r"^\d{6}$" used with re.fullmatch(). The ^ and $ anchors are essential — without them, \d{6} alone would happily match six digits found anywhere inside a longer number.
3. Given re.findall(r"(\w+)@(\w+)", "mail me at rahul@school"), what is returned, and why a list of tuples rather than a list of strings?
   Answer: [('rahul', 'school')]. Because the pattern contains two parenthesised groups, findall() returns the captured pieces as a tuple per match, not the full matched text.
4. Why does re.findall(r"go.*gle", "googlegoogle") return one long match spanning both words instead of two separate ones, and how would you fix it?
   Answer: .* is greedy, so it stretches as far as possible while the rest of the pattern can still succeed, consuming across both occurrences of "google". Changing it to .*? (lazy) makes it stop at the earliest possible point, correctly yielding two separate matches.

Summary

A regular expression describes the shape of text you're looking for, not one fixed piece of text, which is why it can find a mobile number, PIN code, or date without ever being told the exact value in advance. In Python, patterns are written as raw strings (r"...") and used through the re module: re.search() to find a match anywhere, re.match() to check only the start, re.fullmatch() to demand the entire string fit the pattern, re.findall() to collect every match (or every captured group, as tuples, when the pattern has groups), re.sub() to replace matches, and re.split() to break text apart on a pattern instead of a fixed character. Character classes (\d, \w, \s, ., and custom sets like [6-9]) describe what kind of character to expect; quantifiers (*, +, ?, {m,n}) describe how many; anchors (^, $, \b) pin a match to a specific position. Two mistakes trip up almost every beginner and are worth carrying forward deliberately: confusing match() (start-only) with search() (anywhere), and forgetting that * and + are greedy by default, which can silently turn several small matches into one giant one unless you add a ? to make them lazy.

Think About It

Think about this: How would you explain regular expressions in python to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.