API Design: Rate Limiting & Pagination

The Hook: One Script, One Afternoon, One Blocked API Key

Suppose you are building a small train-search tool for a school project. You sign up for a free API — call it TrainFinder — that lets any app send a source station and a destination station and get back a list of matching trains as JSON. Your first request works beautifully. You type Delhi and Mumbai, and in under a second you get back a neat list of trains, timings, and seat availability.

Now you get ambitious. You write a loop that checks train availability every second, for every one of the 40 station-pairs your friends asked about, so the page can "auto-refresh" and always show live data. Your script runs for three minutes. Somewhere around request number 150, something changes: every response you get back is no longer a list of trains. It's a short JSON object that says "error": "rate_limit_exceeded", and your key stops working for the next several minutes.

Nothing about your code is wrong in the way a bug is wrong — there's no typo, no crashed server. The API is doing exactly what it was built to do: protecting itself from being hammered. This chapter is about the two design decisions that make that protection — and the earlier problem of TrainFinder sending you a sane amount of data instead of dumping every train in India into one response — actually work: rate limiting and pagination.

Why an API Needs These Rules at All

An API server is a shared, finite resource. The same computer (or cluster of computers) answering your requests is also answering thousands of other apps' requests at the same instant — some students doing a class project, some production app booking real tickets, maybe a bot scraping data every 10 milliseconds. Two very different failure modes can happen if the API places no limits on how it's used:

• Too many requests arrive per second, the server's CPU and network queue overflow, and the API becomes slow or crashes for everyone, not just the one script that misbehaved. This is the problem rate limiting solves.
• A single request asks for too much data at once — say, "every train in India" instead of "trains from Delhi to Mumbai" — and the response becomes so large that it's slow to generate, slow to transmit, and slow for your app to even parse. This is the problem pagination solves.

These are two separate problems with two separate solutions, and a well-designed API needs both. Let's build each one from a concrete example, the way you'd actually reason through it if you were designing the API yourself.

Rate Limiting: How Many Times Can You Knock?

Think of a bank counter that can serve, at most, one customer every few seconds without the queue collapsing into chaos. It doesn't refuse you outright — it just enforces a maximum pace. An API rate limit works the same way: it caps how many requests a single client (usually identified by an API key or an IP address) is allowed to send within a given time window.

A rate limit is stated as two numbers: a count and a time window. For example: "5 requests per 10 seconds." Let's use exactly this limit for TrainFinder and trace through what happens to a burst of requests sent at seconds t = 1, 2, 3, 4, 5, 6, and 7.

The simplest way to implement this is called a fixed window counter. The server picks fixed-size time windows — here, [0s–10s), [10s–20s), [20s–30s), and so on — and keeps a running count of requests inside the current window. When a request arrives, the server checks which window the current time falls into, looks up how many requests it has already counted in that window, and either allows the request (and increments the count) or rejects it.

request_log = {}  # maps window_start -> count of requests so far

def is_allowed(current_time, limit, window_size):
    window_start = (current_time // window_size) * window_size
    count = request_log.get(window_start, 0)
    if count < limit:
        request_log[window_start] = count + 1
        return True      # request goes through -> 200 OK
    return False          # request refused -> 429 Too Many Requests

for t in [1, 2, 3, 4, 5, 6, 7]:
    result = "ALLOWED" if is_allowed(t, limit=5, window_size=10) else "REJECTED (429)"
    print(f"t={t}s -> {result}")

Let's trace it by hand, since tracing is the only way to be sure code does what you think it does. window_size is 10, so for every t from 1 to 9, window_start = (t // 10) * 10 = 0 — all seven requests land in the same window, window 0. At t=1, the log has no entry for window 0 yet, so count defaults to 0, which is less than the limit of 5, so the request is allowed and the log becomes {0: 1}. The same happens at t=2, 3, 4, and 5 — each time the count is below 5, so it's allowed, and the log climbs to {0: 2}, then {0: 3}, {0: 4}, and finally {0: 5}. At t=6, the count is now 5, which is not less than the limit of 5, so the request is rejected — no increment happens. The same rejection happens at t=7. The printed output is:

t=1s -> ALLOWED
t=2s -> ALLOWED
t=3s -> ALLOWED
t=4s -> ALLOWED
t=5s -> ALLOWED
t=6s -> REJECTED (429)
t=7s -> REJECTED (429)

Reading the Server's Response When You're Rate-Limited

A well-designed API doesn't just silently drop your request when you cross the limit — it tells you clearly, using the HTTP status code 429 Too Many Requests. This is different from status codes you may already know, like 200 (success) or 404 (not found). 429 specifically means "your request was valid and understood, but you're sending too many of them, too fast." Good APIs also send back headers describing exactly where you stand:

X-RateLimit-Limit: 5
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 10
Retry-After: 4

Retry-After: 4 tells your program precisely how many seconds to wait before trying again — in our traced example, the 6th request arrived at t=6 inside a window that started at t=0 and resets at t=10, so 10 − 6 = 4 seconds remain. A well-written client reads this header and pauses, rather than immediately retrying and getting rejected again.

Misconception: a lot of students assume a 429 error means they did something wrong — that their code has a bug, or that they've been caught doing something forbidden. Reality: a 429 is not an accusation. It's a resource-protection mechanism doing exactly its job, and it can happen to perfectly correct code — for instance, a script that legitimately needs to fetch many pages of results very quickly. The correct response isn't to "fix" your logic; it's to slow down, respect Retry-After, and design your client to pace its requests.

The Fixed Window's Hidden Flaw — And a Smarter Alternative

Here is a second, more subtle misconception worth naming directly. Misconception: "a limit of 5 requests per 10 seconds guarantees that no more than 5 requests ever arrive within any 10-second stretch." Reality: a naive fixed window counter does not guarantee this. Because the counter resets sharply at each window boundary, a client could send 5 requests at t=9.5–9.9 seconds (all allowed, since they land in window [0,10)) and then another 5 requests at t=10.0–10.4 seconds (also all allowed, since the counter for the new window [10,20) starts fresh at zero). That's 10 requests inside a 1-second stretch straddling the boundary — double the intended limit — even though the server never technically violated its own rule.

Real production APIs typically fix this with smarter algorithms. A sliding window counts requests in the last N seconds relative to now, not relative to a fixed clock boundary, so it can't be gamed by timing requests around the edge. A token bucket takes a related but slightly different approach: imagine a bucket that holds a maximum of 5 tokens and refills at a steady rate (say, one token every 2 seconds); each request consumes one token, and a request is rejected only if the bucket is empty. This naturally smooths out bursts while still allowing a client that's been idle to "spend" several tokens at once. You don't need to implement either of these to understand rate limiting conceptually — but knowing the fixed window has this boundary weakness is what separates a real understanding of the topic from having memorized one formula.

Pagination: Splitting a Mountain of Data into Pages

Now for the second problem. Suppose your rate limit isn't the issue at all — TrainFinder happily accepts your request for "all trains from Delhi to Mumbai." But instead of a manageable list, it turns out there are 47 matching trains once you count every day of the week and every train class. If the API tried to stuff all 47 full train objects — each with a name, number, timings, every intermediate stop, and seat availability per class — into one JSON response, that response could be hundreds of kilobytes, slow to generate on the server, slow to transmit over a mobile network, and slow for your app to parse and render all at once, especially on a budget phone.

The fix is pagination: instead of returning all 47 records in one response, the API returns a fixed-size page of records — say, 10 at a time — along with metadata telling the client how to fetch the next page. Let's work out the arithmetic precisely, because it's simple but easy to get off by one.

If there are 47 total records and each page holds 10, how many pages do we need? We can't just divide: 47 ÷ 10 = 4.7, and there's no such thing as 0.7 of a page. We need the smallest whole number of pages that can hold all 47 records, which means rounding up, not rounding to the nearest integer. The formula is:

total_pages = ceil(total_records / per_page)
            = ceil(47 / 10)
            = ceil(4.7)
            = 5

Those 5 pages hold 10, 10, 10, 10, and 7 records respectively — the last page is simply whatever remains after the first four full pages. A typical paginated response for page 2 looks like this:

{
  "data": [ /* 10 train objects for this page */ ],
  "page": 2,
  "per_page": 10,
  "total_records": 47,
  "total_pages": 5
}

Notice the response tells the client exactly where it stands — which page it just received, and how many pages remain — so the client's code never has to guess. Here's a client that fetches every page until it has all 47 records, tracing correctly against our numbers:

import math

def fetch_all_trains(source, destination, per_page=10):
    all_trains = []
    page = 1
    while True:
        response = call_api(source, destination, page, per_page)
        all_trains.extend(response["data"])
        total_pages = math.ceil(response["total_records"] / per_page)
        if page >= total_pages:
            break
        page += 1
    return all_trains

Tracing this with total_records = 47 and per_page = 10: on the first loop, page = 1, the server returns 10 records, all_trains now holds 10 items, and total_pages = math.ceil(47 / 10) = 5. Since 1 is not ≥ 5, the loop continues with page = 2. This repeats for pages 2, 3, and 4, each adding 10 more records (20, then 30, then 40 total). On the fifth iteration, page = 5, the server returns the remaining 7 records, bringing the total to 47, and now page (5) >= total_pages (5) is true, so the loop breaks. The function correctly returns all 47 trains across exactly 5 requests — no more, no fewer.

Two Ways to Paginate: Offset Pages vs Cursors

The style we've used so far — page=2&per_page=10 — is called offset pagination, because behind the scenes the server is really saying "skip the first (page − 1) × per_page records, then give me the next per_page." It's simple to understand and simple to implement, which is why it's extremely common.

But it has a real weakness. Misconception: "as long as I ask for the right page number, I'm guaranteed to see every record exactly once." Reality: offset pagination can silently skip or duplicate records if the underlying data changes while you're paginating through it. Imagine TrainFinder's 47 trains are sorted by departure time, and while you're fetching page 2, a new train gets added to the schedule with an earlier departure than several trains already on page 1. Every record after that point shifts one position to the right. When you now request page 3 using "skip 20, take 10," you get a window that has shifted — and you either see one train twice (once on page 2 before the insert, again on page 3 after the shift) or miss one entirely, depending on exactly where the insert happened relative to your position.

Cursor-based pagination solves this by anchoring each request to a stable value instead of a shifting numeric position. Instead of "give me page 3," the client says "give me the next 10 records after train ID 1042" (or after a specific timestamp). Because train ID 1042 doesn't move even if new trains are inserted elsewhere in the list, the client's position in the data stays stable no matter what else changes. This is why large, fast-changing datasets — a social media feed, a live order-tracking system — almost always use cursors, while smaller, fairly static datasets (like a fixed reference table of railway stations) can get away with simple offset pages.

Designing Both Together: A School Attendance API

Let's put rate limiting and pagination into a single, realistic design decision. Suppose your school wants an API that any teacher's app can call to fetch a class's attendance records for the year — roughly 220 school days per student, across 40 students in a section, so around 8,800 attendance records per class per year.

You would never return 8,800 records in one response — that's exactly the scenario pagination exists for. A sensible design might cap per_page at, say, 100 records, giving ceil(8800 / 100) = 88 pages. You'd also cap the maximum per_page a client is allowed to request — say, 200 — because otherwise a client could simply ask for per_page=8800 and defeat the entire purpose of pagination in one request. This is worth stating as its own point: real APIs bound both ends — a minimum sensible default page size and a hard maximum a client cannot exceed, not just an upper limit on request frequency.

On top of that, you'd add a rate limit — say, 30 requests per minute per teacher account — generous enough that a teacher scrolling through 88 pages of records over a few minutes never notices it, but tight enough that a runaway script accidentally stuck in an infinite loop (forgetting to increment page, for instance) gets stopped quickly rather than hammering the school's server with the exact same request forever. Notice how the two mechanisms protect against two different failure modes even within this one API: pagination protects against any single request being too expensive; rate limiting protects against too many requests arriving too fast, correctly-sized or not.

Summary

• Rate limiting caps how many requests a client can send in a given time window, protecting the server from being overwhelmed; violating it returns HTTP status 429 Too Many Requests, typically with a Retry-After header telling the client how long to wait.
• A fixed window counter is the simplest rate-limiting algorithm: it counts requests within clock-aligned windows and resets the count at each boundary — but this creates a boundary-burst flaw, which sliding windows and token buckets are designed to fix.
• Pagination splits a large result set into fixed-size pages instead of sending everything at once; the number of pages needed is ceil(total_records / per_page), always rounding up so the last, partially-filled page is still included.
• Offset pagination (page numbers) is simple but can skip or duplicate records if the data changes mid-fetch; cursor pagination anchors to a stable ID instead of a shifting position, avoiding that problem.
• A 429 response is not a punishment for broken code — it's the rate limiter doing its job, and the correct client behavior is to slow down and respect the retry timing, not to treat it as a bug to eliminate.

Practice: Test Yourself

1. An API enforces a fixed window limit of 8 requests per 15 seconds. A client sends one request per second starting at t=1. At which second does the client first receive a 429?
2. A results endpoint has 133 total records and a page size of 25. How many pages are needed, and how many records are on the last page?
3. Explain, in your own words, why a fixed window counter of "10 requests per minute" does not actually prevent 20 requests from arriving within some 1-second stretch of time.
4. A client is paginating through a live leaderboard using offset pagination (page numbers). Between fetching page 1 and page 2, three players get eliminated and removed from the list. Will the client see every remaining player exactly once? Explain why or why not, and name the pagination style that would avoid this problem.
5. Why do well-designed APIs enforce a maximum allowed per_page value, rather than letting a client request an unlimited number of records on a single page?

Answers: (1) With 8 allowed per window, requests at t=1 through t=8 are allowed; the 9th request, at t=9, is the first to receive a 429 — all within the same [0,15) window, since t=9 < 15. (2) ceil(133 / 25) = ceil(5.32) = 6 pages; the first five pages hold 25 records each (125 total), leaving 8 records on the sixth page. (3) A fixed window resets its count sharply at each boundary, so a client can send its full quota right before a boundary and its full quota again right after — those two bursts land in different windows and are each individually legal, but together they can land within a much shorter real-time stretch than the limit was meant to allow. (4) Not necessarily — because three players were removed, the positions of the remaining players shift, so "skip 10, take 10" for page 2 may now point past a player who moved into an earlier position, causing that player to be skipped entirely (or, if players were added, someone could be duplicated); cursor-based pagination, anchored to a stable player ID rather than a numeric offset, avoids this. (5) Without a cap, a client could request an enormous per_page value in a single call and effectively recreate the "send everything in one giant response" problem that pagination exists to prevent, defeating rate limiting too, since one oversized request could do the damage of hundreds of normal ones.