The Problem Every Form on the Internet Has to Solve
Imagine you are writing the Python code behind your school's online admission form. A student types a 6-digit PIN code, a 10-digit mobile number, and their name. Before your program saves anything, it must check: is this actually a PIN code? Is this actually a mobile number? Get this wrong and your database fills up with garbage — PIN codes with letters in them, phone numbers with 7 digits, names with numbers in them.
Your first instinct might be to write this check by hand.
def is_valid_pin(code):
if len(code) != 6:
return False
for ch in code:
if not ch.isdigit():
return False
return True
print(is_valid_pin("560001")) # True (Bengaluru GPO)
print(is_valid_pin("56A001")) # False (letter A found)
Trace it: is_valid_pin("560001") first checks len(code) != 6 — the string has exactly 6 characters, so this is False and we continue. The loop then walks each character — '5', '6', '0', '0', '0', '1' — and every one passes ch.isdigit(), so we fall through to return True. For "56A001", the length check passes (still 6 characters), but the loop hits 'A', finds not 'A'.isdigit() is True, and immediately returns False.
This works. But now add a mobile number rule ("10 digits, first digit must be 6, 7, 8, or 9"), a PAN card rule ("5 letters, then 4 digits, then 1 letter"), and an email rule. You'd be writing a new hand-rolled loop-and-condition function for every single pattern, and every one is a fresh place to introduce a bug. Programmers who write a lot of text-checking code — which is most programmers, most of the time — got tired of this decades ago and built a dedicated mini-language just for describing patterns in text. That language is called a regular expression, or regex for short. Python gives you access to it through a built-in module called re.
What a Regular Expression Actually Is
A regular expression is a short string that describes a set of strings, not one specific string. When you write the regex cat, you are describing every piece of text that contains the three characters c, then a, then t, in that order, next to each other. Plain letters and digits in a regex are called literals — they match themselves and nothing else.
import re
text = "The cat sat on the mat"
result = re.search("cat", text)
print(result)
Trace this: re.search scans the string from left to right looking for the first place the pattern "cat" occurs. text is "The cat sat on the mat". Counting characters from zero: T(0) h(1) e(2) (3) c(4) a(5) t(6) ... The substring "cat" begins at index 4 and ends right before index 7. So the output is:
<re.Match object; span=(4, 7), match='cat'>A Match object is not just a yes/no answer — it remembers exactly where the match happened, which is why span=(4, 7) is reported. If nothing matches, re.search quietly returns None instead of raising an error, which is why you almost always wrap it in an if before using it.
match, search, and fullmatch — Three Ways to Look
Python's re module gives you three closely related functions, and mixing them up is the single most common regex mistake beginners make. Here is the difference, shown directly:
import re
text = "The cat sat"
print(re.match("cat", text)) # None
print(re.search("cat", text)) # Match, span=(4, 7)
print(re.fullmatch("cat", text)) # None
re.match only checks whether the pattern matches starting at index 0 — the very beginning of the string. Since text starts with "The ", not "cat", it fails and returns None. re.search is more relaxed — it looks anywhere in the string, finds "cat" at index 4, and succeeds. re.fullmatch is the strictest of the three: the entire string, start to end, must match the pattern with nothing left over. Since text has extra text after "cat", it fails too.
This distinction matters enormously for validation. If you use re.match(r"\d{6}", "560001extra"), it will happily report a match — \d{6} matches the first six characters "560001", and match never looks at what comes after. If your goal is to reject "560001extra" as an invalid PIN code, re.match alone will not do it — you would wrongly accept a bad input. This is a genuine, common bug: reaching for re.match when you actually need re.fullmatch, or need to anchor the pattern explicitly with ^ and $ (covered below). For strict validation, always use re.fullmatch or explicit start/end anchors.
Metacharacters: Characters That Mean Something Special
Literal characters like c, a, 9 match themselves. But regex becomes powerful because certain characters are reserved to mean something other than themselves — these are called metacharacters. The core set you need is:
.— matches any single character (except a newline)^— anchors to the start of the string (or negates, inside[ ])$— anchors to the end of the string*— the previous item, 0 or more times+— the previous item, 1 or more times?— the previous item, 0 or 1 time (optional){m,n}— the previous item, between m and n times[ ]— a character class: match any one character from this set( )— a group: captures part of the match for later use|— alternation: this pattern OR that pattern\— escape character, or starts a shorthand class like\d
Each of these deserves a proper worked example, because misusing any one of them produces patterns that look right but silently match the wrong thing.
Character Classes: Choosing From a Set of Characters
A character class, written with square brackets, matches exactly one character from whatever set you list inside. [6-9] matches a single character that is 6, 7, 8, or 9 — the hyphen inside brackets means "range," not "minus." [A-Z] matches one uppercase letter. You can combine ranges: [A-Za-z0-9] matches one letter (either case) or one digit.
Because "any digit" and "any letter" come up constantly, Python provides shorthand classes:
\d— any digit, same as[0-9]\w— any "word" character: letter, digit, or underscore\s— any whitespace character: space, tab, newline\D,\W,\S— the exact opposites (capital letter = negation)
Placing a ^ as the first character right after the opening bracket flips the meaning of a class to "anything except these." [^0-9] matches any character that is not a digit. This is the one place where ^ does not mean "start of string" — inside brackets it means negation, outside brackets it means an anchor. Confusing these two uses of the same symbol is a genuine, common beginner mistake, so it is worth saying explicitly: ^ outside [ ] is an anchor; ^ as the first character inside [ ] is negation. Everywhere else, a caret is just a literal caret.
Let's use this to check a real, well-known Indian document format: a PAN (Permanent Account Number) card, which always follows the pattern of five uppercase letters, four digits, and one uppercase letter — for example ABCDE1234F.
import re
pan_pattern = r'^[A-Z]{5}[0-9]{4}[A-Z]$'
print(bool(re.match(pan_pattern, "ABCDE1234F"))) # True
print(bool(re.match(pan_pattern, "abcde1234f"))) # False (lowercase)
print(bool(re.match(pan_pattern, "ABCDE123F"))) # False (only 3 digits)
Read the pattern piece by piece, left to right, exactly the way the regex engine does: ^ anchors to the start. [A-Z]{5} means "an uppercase letter, exactly 5 times" — we'll explain {5} properly in the next section. [0-9]{4} means "a digit, exactly 4 times." [A-Z] means one more uppercase letter. $ anchors to the end. For "ABCDE1234F": ABCDE satisfies [A-Z]{5}, 1234 satisfies [0-9]{4}, F satisfies the final [A-Z], and we're exactly at the end of the string, so $ succeeds. For "abcde1234f", lowercase letters do not belong to the class [A-Z], so the very first character already fails to match. For "ABCDE123F", after matching ABCDE and three digits 123, the engine needs a fourth digit for {4} but finds the letter F instead, so the match fails.
Quantifiers: Saying How Many Times
Quantifiers control repetition. * means "zero or more," + means "one or more," ? means "zero or one" (optional), and {m,n} gives you exact control. {6} means exactly 6 times. {2,4} means between 2 and 4 times. {2,} means 2 or more, with no upper limit.
The ? quantifier is perfect for optional spelling variants:
import re
text = "I like color and colour, both spellings"
print(re.findall(r'colou?r', text))
The pattern is the literal letters c, o, l, o, then u? (the letter u, appearing 0 or 1 times), then r. Scanning the text, the engine first tries to match starting at "color": c-o-l-o matches directly, then u? checks the next character — it's r, not u — so u? simply matches zero times (that's what "optional" means), and then r matches the final r. Later, at "colour," u? finds an actual u present and consumes it before matching r. Both succeed, so the output is:
['color', 'colour']Now put {m,n} to work for validation. Recall the PIN code and mobile number checks from the opening section — here they are properly, in regex:
import re
def is_valid_pin(code):
return bool(re.fullmatch(r'\d{6}', code))
def is_valid_mobile(number):
return bool(re.fullmatch(r'[6-9]\d{9}', number))
print(is_valid_pin("560001")) # True
print(is_valid_pin("56001")) # False, only 5 digits
print(is_valid_mobile("9876543210")) # True
print(is_valid_mobile("5876543210")) # False, starts with 5
print(is_valid_mobile("98765432100")) # False, 11 digits
Trace is_valid_mobile("98765432100") carefully, since it's the trickiest one: the pattern is [6-9] (exactly one character, first digit 6 through 9) followed by \d{9} (exactly nine more digits) — ten characters total. The string has eleven digits. Because we used re.fullmatch, the entire string must be consumed by the pattern with nothing left over. Even though the first ten characters "9876543210" would match perfectly on their own, there's an eleventh character "0" dangling at the end with nothing left in the pattern to match it against, so fullmatch correctly reports failure. This is exactly why fullmatch (or anchoring with ^...$) matters for validation — without it, a length-11 string could slip through by matching just its first ten characters.
Anchors and the Anatomy of a Full Pattern
We've been using ^ and $ already; now let's see the whole mobile-number pattern diagrammed against a real string, character by character, the way the regex engine actually walks through it.
Notice what each anchor is actually doing: ^ does not match a character — it matches a position, the point before the very first character. $ similarly matches the position right after the last character. That's why they're drawn as dashed lines between characters in the diagram rather than as boxes around a character — anchors are "zero-width," they don't consume anything, they just constrain where the surrounding pattern is allowed to sit.
Groups: Capturing Pieces of a Match
Parentheses do two things at once: they group parts of a pattern together (so a quantifier can apply to the whole group), and by default they also capture whatever text matched inside them, so you can pull it out afterward. This is enormously useful for extracting structured data out of free text.
import re
text = "Republic Day: 26-01-2026"
pattern = r'(\d{2})-(\d{2})-(\d{4})'
match = re.search(pattern, text)
print(match.group(0)) # 26-01-2026 (the whole match)
print(match.group(1)) # 26 (day)
print(match.group(2)) # 01 (month)
print(match.group(3)) # 2026 (year)
The pattern has three separate groups, each wrapped in its own parentheses: the first (\d{2}) matches and remembers two digits, then a literal hyphen (outside any group, so it's not captured), then a second (\d{2}), another hyphen, then (\d{4}) for four digits. match.group(0) (or just match.group()) always returns the entire matched text; group(1), group(2), group(3) return what each individual set of parentheses captured, numbered left to right by their opening bracket. This is how you'd pull the day, month, and year out of a date string as three separate, usable pieces instead of one undifferentiated blob of text.
Three Workhorse Functions: findall, sub, and split
Beyond checking whether a pattern matches, the re module gives you tools to pull out every match, replace matches, and split text on a pattern.
findall returns every non-overlapping match as a list:
import re
text = "Rahul scored 45 runs off 32 balls, hitting 4 fours and 2 sixes."
print(re.findall(r'\d+', text))
The pattern \d+ means "one or more digits, as many as possible in a row." Scanning left to right, the engine finds four separate runs of digit characters: 45, 32, 4, and 2. Every non-digit character in between (letters, spaces, commas) is simply skipped over until the next run of digits is found. Output:
['45', '32', '4', '2']sub (short for "substitute") finds matches and replaces them, which is exactly how apps mask sensitive data:
import re
mobile = "9876543210"
masked = re.sub(r'\d{6}$', 'XXXXXX', mobile)
print(masked)
The pattern \d{6}$ means "six digits, right at the end of the string." In "9876543210" (10 characters), the last six characters are "543210"; the first four, "9876", are left untouched because the pattern only matches starting from a position where exactly six digits remain until the end. The replacement text 'XXXXXX' is substituted in for that matched piece, giving:
9876XXXXXXsplit breaks a string apart wherever the pattern matches, instead of on a single fixed character:
import re
items = "apple, banana;cherry, mango"
print(re.split(r'[,;]\s*', items))
The pattern [,;]\s* matches a comma or semicolon, followed by any amount of extra whitespace (including none). This handles messy, inconsistently-formatted input in one line: a comma-plus-space, a semicolon-with-no-space, and a comma-plus-two-spaces are all recognized as "the same kind of separator," so the output is a clean list:
['apple', 'banana', 'cherry', 'mango']Notice how much this saves you compared to writing separate .split(",") and .split(";") calls and then stripping whitespace off every piece by hand.
Common Misconception #1: The Dot Matches Anything, Not Just a Dot
Beginners very often write . expecting it to mean "a literal period," because that's what it looks like. It does not. . is a metacharacter meaning "any single character except a newline." If you actually want to match a literal period, you must escape it: \.. Skipping the backslash is one of the most common real-world regex bugs, and it's easy to demonstrate why it's dangerous rather than just harmless:
import re
text = "Price is 3x14 or 3.14"
print(re.findall(r'\d+.\d+', text)) # unescaped dot — buggy
print(re.findall(r'\d+\.\d+', text)) # escaped dot — correct
Trace the first, buggy pattern: \d+ (one or more digits), then . (any single character at all — remember, unescaped), then \d+ again. Scanning "3x14": \d+ greedily matches digits, but the very next character is x, not another digit, so \d+ settles for matching just "3". Then the unescaped . — which is happy to match any character — matches the letter x. Then \d+ matches "14". The whole thing "3x14" counts as a match, even though there's no decimal point anywhere near it! Then scanning "3.14": the same logic matches 3, then . matches the actual period, then 14. So the buggy pattern's output is:
['3x14', '3.14']That first result is a real bug — a program looking for decimal numbers has just also matched something that clearly isn't one. The fix is a single backslash: \. matches a literal period and nothing else, so "3x14" correctly fails to match (there's no literal period between the 3 and 14) and only the genuine decimal survives:
['3.14']The lesson generalizes: any time your pattern needs to match one of the metacharacters listed earlier as a literal character — a real dot, a real question mark, a real parenthesis — you must escape it with a backslash first.
Common Misconception #2: Quantifiers Are Greedy By Default
Quantifiers like *, +, and {m,n} are greedy by default — they try to consume as much text as possible while still letting the overall pattern succeed, and only backtrack (give characters back) if forced to. This surprises people who expect a quantifier to stop at the "obvious" natural boundary.
import re
html = "<b>bold</b> and <i>italic</i>"
print(re.findall(r'<.*>', html))
print(re.findall(r'<.*?>', html))
The pattern <.*> means "a literal <, then any characters at all (zero or more, greedy), then a literal >." The engine finds the first < at the very start of the string, and then, because * is greedy, tries to stretch .* across as much of the remaining string as it possibly can — all the way to the end. It then backtracks one character at a time only until it finds a > to satisfy the final piece of the pattern. Since the last > in the whole string is at the very end (closing </i>), that's what it settles for. The entire string, from the first < to the last >, counts as a single match:
['<b>bold</b> and <i>italic</i>']That is almost never what you want when picking out individual tags. Adding a ? right after a quantifier makes it non-greedy (or "lazy") — it now tries to match as little as possible, only expanding when forced to. .*? stops at the very first > it can reach:
['<b>', '</b>', '<i>', '</i>']Each individual tag is captured separately, which is almost always the intent. The rule to remember: *, +, and ? are greedy; add a trailing ? to any of them (*?, +?, ??) to make them lazy instead.
Putting It Together: A Simplified Email Check
Now combine character classes, quantifiers, and anchors to validate something more elaborate — an email address for a school sign-up form:
import re
def is_valid_email(addr):
pattern = r'^[\w.+-]+@[\w-]+\.[a-zA-Z]{2,}$'
return bool(re.match(pattern, addr))
print(is_valid_email("student@aici.org")) # True
print(is_valid_email("not-an-email")) # False
print(is_valid_email("student@aici")) # False
Read this piece by piece, the way you should read any unfamiliar regex: ^ anchors to the start. [\w.+-]+ matches one or more characters that are letters, digits, underscores, dots, plus signs, or hyphens — this is the local part before the @ (things like student.name+school are legal here). @ is a literal at-sign. [\w-]+ matches the domain name part. \. is an escaped, literal period — remember Misconception #1, if we left this unescaped it would accidentally accept a domain with no dot in it at all, as long as some character sat where the dot should be. [a-zA-Z]{2,} matches the extension — at least two letters, with no upper limit (so both .in and .org and .education are accepted). $ anchors to the end.
For "not-an-email": [\w.+-]+ can actually consume the entire string, since hyphens are inside its class — but then the pattern demands a literal @, and the string has run out of characters, so the match fails. For "student@aici": everything matches up through the domain "aici", but then the pattern demands a literal . followed by at least two letters, and there's nothing left in the string, so it fails too, correctly rejecting a domain with no top-level extension.
One honest caveat worth knowing: this pattern is deliberately simplified. The full technical specification for what counts as a valid email address (RFC 5322) is famously long and allows for strange edge cases most people never see in practice — quoted local parts, IP-address domains, and more. Production systems generally use a simplified check like this one for the format, and confirm the address actually works by sending a verification email. A regex tells you the text has the right shape; it can't tell you whether the mailbox behind it is real.
How Python Actually Runs a Regex, in Brief
It helps to have a mental model of what re.search is doing under the hood, since it explains both the greedy behavior above and why patterns can occasionally run slowly on adversarial input. The engine tries to match the pattern starting at position 0 of the string. If the whole pattern succeeds from there, it stops and reports that match. If it fails partway through, it doesn't give up on the string — it advances the starting position to 1, and tries the entire pattern again from there, then position 2, and so on, until either a match is found or every starting position has been tried. This "try, fail, backtrack, retry" process is exactly why greedy quantifiers first grab everything and then give characters back one at a time when a later part of the pattern needs them — the engine is always searching for any way to make the whole pattern succeed, not necessarily the most "sensible-looking" way to a human.
Quick Reference: Building a Pattern From Scratch
When you need to write a new regex for a validation task, work through these questions in order, which mirrors exactly how we built the mobile number and PAN patterns above: What literal characters or separators must appear exactly, unchanged? What positions need a class of allowed characters ([ ], or a shorthand like \d/\w)? How many times does each piece repeat (?, *, +, or an exact {m,n})? Does the match need to be anchored to the whole string with ^...$ (or built with re.fullmatch), or is finding a match anywhere in a larger text acceptable? Do you need to extract pieces afterward with groups ( )? Answering these five questions in order turns "write a regex" from guesswork into a repeatable procedure.
Practice: Active Recall
Work through these before checking the explanations that follow each one.
1. What does re.findall(r'\d{2,4}', "In 2026, class 8 has 45 students") return?
Answer: ['2026', '45'] — 8 is a run of only 1 digit, and \d{2,4} requires at least 2 digits in a row, so it's skipped.
2. Write a pattern that matches a string made up of exactly three letters followed by exactly three digits, and nothing else, such as "ABC123".
Answer: r'^[A-Za-z]{3}\d{3}$' (or check with re.fullmatch(r'[A-Za-z]{3}\d{3}', s), which anchors both ends automatically).
3. Why does re.match(r'\d+', "12 apples") succeed even though the string contains a non-digit character?
Answer: re.match only requires the pattern to match starting at position 0; it never demands that the whole string be consumed, so it happily matches just the "12" and stops, ignoring everything after.
4. Given re.sub(r'[aeiouAEIOU]', '*', "Regular Expressions"), what is the output?
Answer: "R*g*l*r *xpr*ss**ns" — every vowel, uppercase or lowercase, is replaced one at a time by the character class.
5. A pattern r'<p>.*</p>' is applied with re.findall to a string containing three separate <p>...</p> paragraphs one after another. Will it return three separate matches?
Answer: No — because * is greedy, it will stretch from the very first <p> to the very last </p>, swallowing all three paragraphs (and the tags between them) into one single match. Use .*? to get three separate matches instead.
Summary
A regular expression is a compact language for describing a set of possible strings rather than one exact string, which is why it replaces long chains of hand-written if/for validation logic. Python exposes this language through the re module. re.search looks anywhere in a string; re.match only checks from the start; re.fullmatch requires the entire string to match, which is what you almost always want for validation. Character classes ([ ], \d, \w, \s, and their negations \D, \W, \S) choose one character from a set. Quantifiers (?, *, +, {m,n}) control how many times something repeats, and are greedy by default — add a trailing ? to make them lazy. Anchors ^ and $ pin a match to the start or end of the string and consume no characters themselves. Parentheses ( ) group pieces of a pattern and capture the matched text for later extraction via .group(). findall, sub, and split are the three workhorse functions for extracting, replacing, and dividing text using a pattern instead of a fixed character. And the two mistakes to watch for above all others: an unescaped . matches any character, not a literal dot — escape it as \. when you mean a real period; and re.match succeeding does not mean the whole string is valid, only that the pattern was found starting at position zero — reach for re.fullmatch, or explicit ^...$ anchors, whenever you're validating an entire field rather than searching within free text.