Redis: Lightning-Fast Data Caching

The Tatkal Rush Problem

Every day at 10:00 AM, Tatkal booking opens on IRCTC for a popular train. In the first thirty seconds, lakhs of people across the country hit "refresh" on the same handful of trains, checking the same question over and over: "Is seat B1-45 on train 12951 still available?" Now imagine that every single one of those refresh clicks travels all the way to the main database and asks it to look up the row for that seat, check its status, and send the answer back. The database has to search its indexes, read data off a physical disk, and format a response — and it has to do this thousands of times per second, for the same handful of seats, because everyone is asking about the same trains at the same moment.

This is not a hypothetical problem. It is one of the most common bottlenecks in real software systems: an enormous number of requests asking for the same small set of information, hitting a storage system that was never designed to answer the same question a million times a second. A relational database sitting on disk is built for accuracy, structure, and durability — not for answering an identical question instantly, again and again, under extreme load. Something faster needs to sit in front of it. That "something" is what this chapter is about: an in-memory data store called Redis, and the general idea of caching that it is built to solve.

What "Caching" Really Means

Before touching Redis specifically, understand caching itself, because Redis is just one very good implementation of this one idea.

Think about how you actually study for an exam. Your textbook lives on a shelf across the room. If you needed a formula from Chapter 4 every ten seconds while solving numerical problems, you would not walk to the shelf, open the book, find the page, read the formula, and walk back each time. You would write that one formula on a sticky note and keep it right next to you. The full textbook is still the "source of truth" — it has everything, correctly and completely. But the sticky note is a small, fast, temporary copy of the one piece of information you need right now, kept close at hand so you don't pay the "walk to the shelf" cost every single time.

That sticky note is a cache. Formally: a cache is a smaller, faster storage layer that keeps copies of frequently-used data close to where that data is needed, so that repeated requests for the same data can be answered without going back to the original, slower source. The original source (your textbook, or in software, the main database) is still authoritative. The cache is a shortcut, not a replacement.

Why RAM Beats Disk: A Speed Worked Example

To understand why Redis in particular is used for caching, you need one concrete fact about computer hardware: reading data from RAM (main memory) is dramatically faster than reading it from a disk, whether that disk is an SSD or a traditional hard drive. These are rough, well-known order-of-magnitude figures used across computer science to reason about system speed:

• Reading from RAM: roughly 100 nanoseconds
• Reading from an SSD: roughly 100,000 nanoseconds (about 0.1 millisecond)
• Reading from a spinning hard disk: roughly 10,000,000 nanoseconds (about 10 milliseconds)

Let's turn that into simple ratios, the kind of arithmetic you already know:

SSD vs RAM  = 100,000 / 100        = 1,000 times slower
HDD vs RAM  = 10,000,000 / 100     = 100,000 times slower

An SSD is roughly a thousand times slower than RAM, and a spinning hard disk is roughly a hundred thousand times slower. These are approximate, order-of-magnitude numbers — real hardware varies — but the gap is real and it is enormous. A traditional database like MySQL or PostgreSQL ultimately stores its data on disk (even with clever caching layers of its own), because disk storage keeps your data safe even when the power goes off. RAM, by contrast, is volatile — its contents vanish the instant power is lost. That trade-off — RAM is blazing fast but forgets everything on restart, disk is much slower but remembers — is the entire reason caching systems exist as a separate layer instead of everyone just using RAM for everything.

Redis's core idea is simple: keep an entire dataset in RAM, so that reads and writes happen at RAM speed instead of disk speed. That single design decision is why Redis can answer a lookup in around a millisecond when the equivalent disk-based database query might take 100–300 milliseconds under load.

Meet Redis: The REmote DIctionary Server

Redis (the name stands for REmote DIctionary Server) is an open-source, in-memory data store created in 2009 by Salvatore Sanfilippo, written in the C programming language. At its core, Redis is a key-value store: you give it a unique key (a name), and it stores a value under that key, exactly like a phone contacts app where the "key" is a person's name and the "value" is their phone number. You don't search through every contact one by one — you look up "Ananya" and instantly get her number back. Redis works the same way, except the lookup happens in RAM and typically completes in well under a millisecond.

This makes Redis a NoSQL database, specifically of the "key-value" category — one of the four broad families of NoSQL systems (the others being document stores, column-family stores, and graph databases). This matters for your CBSE database concepts: a relational database like MySQL organizes data into tables with rows and columns and enforces strict relationships between them through primary and foreign keys. Redis has no tables, no rows, no SQL query language, and no built-in concept of relationships between records. You cannot ask Redis "find all seats where price is less than ₹500 and class is Sleeper," the way you could with a SQL WHERE clause. What you get in exchange for giving up that structure is raw speed and simplicity for a narrower kind of question: "given this exact key, what is its value, right now?"

Redis's Data Structures: More Than Just Strings

A common first impression is that Redis just stores text under a name, like a giant dictionary of strings. That undersells it. Redis values can be several different data structures, each suited to a different real problem:

• String — the simplest value: text or a number under one key. Example: caching a seat's availability status, or a rendered HTML fragment.
• List — an ordered sequence of values, like a queue or a stack. Example: the most recent chat messages in a room, added to one end and read from the front.
• Hash — a key that itself contains multiple field-value pairs, like a small record. Example: a user's profile (name, grade, city) stored under one key instead of three separate keys.
• Set — an unordered collection of unique values, with no duplicates allowed. Example: the set of unique student IDs who have submitted today's quiz.
• Sorted Set (ZSet) — like a set, but every member has a numeric score attached, and Redis keeps members ordered by that score automatically. Example: a live leaderboard, where the score is a player's points.

Each of these is not a separate "feature bolted on" — they are core to why Redis is used for far more than plain caching, including real-time leaderboards, message queues, rate limiters, and session storage, all of which you'll see later in this chapter.

Hands-On: Core Redis Commands

Redis is normally operated through a command-line tool called redis-cli, or through a client library in a programming language like Python. Let's trace through real commands and their exact outputs, the way you'd see them in a terminal.

Strings, expiry, and atomic counters:

> SET seat:12951:B1:45 "available"
OK

> GET seat:12951:B1:45
"available"

> EXPIRE seat:12951:B1:45 30
(integer) 1

> TTL seat:12951:B1:45
(integer) 30

> INCR page:views
(integer) 1

> INCR page:views
(integer) 2

> DEL seat:12951:B1:45
(integer) 1

Trace what happened, line by line. SET stores the string "available" under the key and replies OK. GET reads it back unchanged. EXPIRE attaches a 30-second lifetime to the key and replies with the integer 1, meaning "yes, the expiry was set." TTL (time to live) asks how many seconds remain — here, freshly set, it's still 30. INCR is important: it atomically increases a numeric value by 1, creating the key at 0 first if it doesn't exist yet. Two calls give 1, then 2 — this is exactly how a page-view counter or a "how many people are viewing this train right now" counter is implemented, and because the increment is atomic, thousands of simultaneous requests can safely increment the same counter without corrupting the count. Finally, DEL removes the key and confirms with 1 (one key was deleted).

Lists — order matters:

> LPUSH chat:room1 "Hi"
(integer) 1

> LPUSH chat:room1 "Hello"
(integer) 2

> LRANGE chat:room1 0 -1
1) "Hello"
2) "Hi"

LPUSH pushes a new element onto the left (head) of the list, and each call reports the list's new length. Because "Hello" was pushed after "Hi", it now sits ahead of "Hi" at the head of the list. LRANGE chat:room1 0 -1 reads the entire list from index 0 to index -1 (Redis uses -1 to mean "the last element"), and the output correctly shows the most recently pushed message first — exactly the behaviour you want for a live chat feed showing newest messages on top.

Hashes — a mini-record under one key:

> HSET user:101 name "Ananya" grade "9" city "Pune"
(integer) 3

> HGETALL user:101
1) "name"
2) "Ananya"
3) "grade"
4) "9"
5) "city"
6) "Pune"

HSET sets three fields at once inside the hash stored under user:101, and returns 3 because three new fields were created. HGETALL returns every field and its value, alternating field-name, value, field-name, value — one hash key replacing what would otherwise be three separate string keys (user:101:name, user:101:grade, user:101:city).

Sorted sets — a live leaderboard:

> ZADD leaderboard 950 "Rahul" 870 "Priya" 1020 "Zara"
(integer) 3

> ZREVRANGE leaderboard 0 2 WITHSCORES
1) "Zara"
2) "1020"
3) "Rahul"
4) "950"
5) "Priya"
6) "870"

ZADD adds three members with their scores (950, 870, 1020) in one call. ZREVRANGE asks for ranks 0 through 2 in reverse score order — highest first — with scores included. Redis keeps sorted sets ordered internally at all times, so this rank query, which would need an ORDER BY and a full sort in a relational database, is essentially free in Redis.

The Cache-Aside Pattern: How an App Actually Uses Redis

Knowing individual commands is not the same as knowing how Redis fits into a real application. The most common pattern is called cache-aside (also called "lazy loading"), and it works in five steps whenever the app needs some piece of data:

1. The application receives a request (e.g., "is seat B1-45 available?").
2. It first checks Redis for that key.
3. If the key is found — a cache hit — Redis returns the value immediately, and the main database is never touched.
4. If the key is missing — a cache miss — the application queries the main (disk-based) database, gets the real answer, and then stores a copy of that answer into Redis (usually with an expiry time) before returning it.
5. The next request for the same key becomes a cache hit, and stays fast until the cached copy expires.

The dashed arrow from the database back up to Redis is the crucial step people forget: on a miss, the app doesn't just answer the client and move on — it also writes a copy into Redis (using a command like SETEX key 30 value, which sets a value and its 30-second expiry in one atomic call), so the next request for that same key becomes a hit.

Worked Example: How Much Does Caching Actually Save?

Suppose the main database takes 200 ms to answer a seat-availability query under load, and Redis answers the same query in 1 ms. If a fraction h of requests are cache hits (and the rest, 1 − h, are misses that still need the full 200 ms), the average response time across many requests is:

average_latency = h × cache_latency + (1 − h) × db_latency

This is ordinary weighted-average algebra — the same idea as computing your average marks when different tests carry different weights. Let's plug in numbers for three different hit ratios:

h = 0.50 (50% hit rate):
average = 0.50×1 + 0.50×200 = 0.5 + 100    = 100.5 ms
speedup = 200 / 100.5                       ≈ 2.0×

h = 0.90 (90% hit rate):
average = 0.90×1 + 0.10×200 = 0.9 + 20      = 20.9 ms
speedup = 200 / 20.9                        ≈ 9.6×

h = 0.99 (99% hit rate):
average = 0.99×1 + 0.01×200 = 0.99 + 2      = 2.99 ms
speedup = 200 / 2.99                        ≈ 66.9×

Notice how the speedup does not grow evenly with the hit ratio — going from 90% to 99% hits (a jump of 9 percentage points) roughly seven-times the speedup, while going from 50% to 90% (a jump of 40 percentage points) only gave about 4.8 times the speedup. This is why real caching systems obsess over squeezing the last few percent out of their hit ratio — during a Tatkal rush, where the same few trains are being checked by everyone, the hit ratio for popular trains can realistically sit above 95%, which is exactly the regime where caching delivers its biggest wins.

The hit ratio itself is calculated simply as:

hit_ratio = cache_hits / (cache_hits + cache_misses)

If Redis answered 9,500 requests directly and only 500 requests had to fall through to the database, the hit ratio is 9500 / (9500+500) = 9500/10000 = 0.95, or 95%.

TTL, Staleness, and the Hardest Problem in Caching

Every cached value in this chapter has had an expiry attached — 30 seconds via EXPIRE or SETEX. This is called the TTL (Time To Live), and it exists to solve a problem caching itself creates: the cached copy can become stale — wrong — the moment the real data changes. If seat B1-45 gets booked by someone else one second after you cached "available," anyone reading from the cache for the next 29 seconds will be told a seat is free when it is not.

This is a genuinely hard trade-off, not a minor detail: a short TTL keeps data fresher but means more cache misses (less speed benefit); a long TTL gives you more speed but risks showing outdated information for longer. Computer scientists half-jokingly call cache invalidation — deciding exactly when a cached value should be thrown away or updated — one of the notoriously hard problems in the field, precisely because there is no single correct answer; it depends entirely on how quickly the underlying data changes and how costly a stale answer would be. For a seat availability cache during Tatkal booking, engineers typically choose a very short TTL (a few seconds) specifically because stale seat data can cause double-bookings. For something like a train's static route information (which almost never changes), a TTL of many hours is perfectly safe.

Misconception Corner: "In-Memory Means It Just Disappears, Right?"

A very common misunderstanding is: "Redis stores everything in RAM, and RAM is wiped when the power goes out or the server restarts, so Redis must lose all its data constantly — it's basically unreliable." This is only half true, and the missing half matters.

By default, yes, pure in-memory storage is volatile. But Redis provides two mechanisms specifically to guard against this: RDB snapshots, where Redis periodically saves a compressed copy of its entire dataset to disk (so on restart, it reloads from the last snapshot), and the AOF (Append-Only File), where Redis logs every write operation to a file on disk as it happens, allowing it to reconstruct the exact dataset by replaying that log after a restart. A Redis instance configured with AOF enabled can lose, at most, a fraction of a second of the very latest writes — not "all its data." So Redis is not simply a fragile, forgetful cache; it is a deliberately volatile-and-fast system that has opt-in durability features layered on top for when you need them.

That said, the corrected understanding is nuanced, not a flat "actually it's totally safe": most teams still treat Redis as a fast secondary layer sitting in front of a "real," disk-native relational or document database, rather than as the sole permanent home for critical data like financial transaction records — because even with AOF, a disk-based system built from the ground up for durability and complex querying (like PostgreSQL or MySQL) remains the safer choice for data you absolutely cannot afford to reconstruct or lose.

What Happens When the Cache Fills Up: Eviction Policies

RAM is far more expensive per gigabyte than disk, so a Redis cache is usually much smaller than the full dataset sitting in the main database — it can only hold the "hot," frequently-requested subset. When Redis's memory limit is reached and a new key needs to be stored, it must decide what to remove to make room. This is controlled by a configurable eviction policy, and the most common one is LRU — Least Recently Used: Redis removes whichever key hasn't been accessed for the longest time, on the theory that data nobody has asked for recently is the least likely to be asked for again soon.

Think of a small bookshelf in your room that only holds ten books, but you own two hundred. Whenever you need book #201 and the shelf is full, the natural rule is to return whichever of the ten books on your shelf you haven't opened in the longest time, and put the new one in its place. That is exactly LRU eviction. Redis supports several policy variants — allkeys-lru (evict the least-recently-used key from anywhere), volatile-lru (only evict among keys that have a TTL set), and noeviction (refuse new writes once full, rather than deleting anything) — chosen based on whether losing a cached value is acceptable or must be actively prevented.

Where This Pattern Shows Up in Real Systems

The seat-availability example running through this chapter is exactly the shape of problem large ticket-booking and payment platforms in India face: an enormous number of read requests concentrated on a small set of "hot" records within a short burst of time. The general architecture — an in-memory cache like Redis sitting in front of a disk-based database, with short TTLs on fast-changing data — is the standard solution used across high-traffic e-ticketing systems, food-delivery order-status screens, and UPI-style payment platforms that need to enforce rate limits (for example, using INCR plus EXPIRE together to count how many transaction attempts a device has made in the last sixty seconds, to block fraud, without that counter query ever touching the main transaction database). Sorted sets power exactly the kind of real-time leaderboards used in fantasy-cricket apps during an IPL match, where thousands of users' scores need to be re-ranked continuously as matches progress — a query that would be punishingly slow if it required re-sorting a full disk table on every point update, but is nearly instantaneous when the ranking is maintained continuously in a Redis sorted set.

Check Your Understanding

Q1. A cache answers a request in 2 ms and the database answers it in 250 ms. If the hit ratio is 80%, what is the average latency? Show your working.

Q2. Why does Redis use RAM instead of disk as its primary storage, and what is the trade-off it accepts by doing so?

Q3. Trace this command sequence and write the exact output of each line: RPUSH queue "a", then RPUSH queue "b", then LRANGE queue 0 -1. (Hint: RPUSH adds to the right/tail end, unlike LPUSH.)

Q4. A junior developer says: "I'll just cache everything with no expiry time, forever — that way it's always fast." Explain, in your own words, what could go wrong with this plan.

Q5. True or False, with a one-line justification: "Since Redis is in-memory, all data is permanently lost the instant the server restarts, no matter how it is configured."

Q6. Which Redis data structure would you choose to store a live IPL fantasy-league leaderboard, and which command would you use to fetch the top 5 players?

Answers — Q1: average = 0.8×2 + 0.2×250 = 1.6 + 50 = 51.6 ms. Q2: RAM access is roughly 1,000–100,000 times faster than disk access; the trade-off is that RAM is volatile and loses data on power loss unless Redis's RDB/AOF persistence is explicitly configured. Q3: RPUSH queue "a" → (integer) 1; RPUSH queue "b" → (integer) 2; LRANGE queue 0 -1 → 1) "a" 2) "b" (RPUSH adds to the tail, so order is preserved as inserted, unlike LPUSH). Q4: Without any expiry, the cache will keep returning outdated (stale) data forever once the real value changes underneath it — a booked seat could show "available" indefinitely, and old cached entries would also keep consuming RAM forever, eventually forcing evictions anyway. Q5: False — with RDB snapshots and/or AOF logging enabled, Redis can reload its dataset from disk after a restart, losing at most a small window of the most recent writes rather than everything. Q6: A Sorted Set (ZSet), scored by points, using ZREVRANGE leaderboard 0 4 WITHSCORES to fetch the top 5 in descending order.

Summary

• A cache is a smaller, faster storage layer holding copies of frequently-requested data, so repeated requests skip the slower original source.
• Redis (REmote DIctionary Server, 2009, Salvatore Sanfilippo) is an open-source, in-memory key-value NoSQL store; keeping data in RAM instead of on disk is the entire reason it is fast — RAM is roughly 1,000× faster than SSD and 100,000× faster than a spinning hard disk.
• Redis supports several value types beyond plain strings: Lists (ordered, e.g. chat feeds), Hashes (mini-records, e.g. user profiles), Sets (unique unordered members), and Sorted Sets (scored and auto-ranked, e.g. leaderboards).
• The cache-aside pattern is the standard way apps use Redis: check cache → on hit, return instantly; on miss, query the real database, then write the result into Redis for next time.
• The average latency under caching follows h × cache_latency + (1−h) × db_latency, and small increases in hit ratio near 90–99% produce disproportionately large speedups — which is why high hit ratios matter so much in practice.
• TTL (Time To Live) controls how long a cached value is trusted before it expires, balancing freshness against speed; choosing it wrong causes either stale data or too many cache misses.
• Redis is volatile by default but supports RDB snapshots and AOF logging for persistence — it is not automatically "unreliable," though most systems still keep a disk-based database as the ultimate source of truth.
• When a Redis cache fills up, an eviction policy like LRU (Least Recently Used) decides which keys to discard to make room for new ones.