Recommender Systems: Netflix for You

The Feed That Seems to Read Your Mind

Open JioHotstar after finishing a cricket highlights reel and the next row is full of sports documentaries. Open it after a Korean thriller and the row changes to crime dramas you have never searched for. Spotify builds you a Discover Weekly playlist every Monday morning that somehow contains three songs you would have picked yourself. None of this is magic, and none of it involves anyone at these companies watching your account by hand. It is the output of an algorithm solving one very specific mathematical problem: given a huge, mostly empty table of who liked what, predict the empty cells.

That sentence is the entire chapter in miniature. Everything that follows — the vectors, the angles, the formulas — exists to answer one question precisely: out of thousands of movies you have never rated, which few should be pushed to the top of your screen? By the end of this chapter you will be able to compute that answer by hand for a small example, and you will understand exactly what a production recommender system is approximating when it does the same thing for two hundred million users.

The Rating Matrix: What the System Actually Sees

Strip away the UI and a service like Netflix is holding one large table. Rows are users, columns are titles, and each cell is a rating — either explicit (you tapped 4 stars) or implicit (you watched 90% of the episode, so the system infers a high rating). A tiny slice of that table might look like this:

A real streaming catalogue has tens of thousands of columns and tens of millions of rows, and the overwhelming majority of cells are blank — you have rated maybe forty of the fifteen thousand titles on the platform. This is called a sparse matrix, and the entire job of a recommender system is to fill in the blanks in your row with predicted numbers, then sort your unwatched titles by that prediction.

There are two fundamentally different strategies for doing this, and almost every real system blends them.

Two Philosophies: Collaborative vs. Content-Based Filtering

Collaborative filtering ignores what a movie is actually about. It only looks at the rating matrix. The logic is: "find users whose row of numbers looks like yours, then recommend you the things they liked that you have not seen yet." It works even for content the system knows nothing about — no genre tags, no cast list, nothing except numbers. Its weakness is the cold-start problem: a brand-new user with zero ratings has no row to compare, and a brand-new movie with zero ratings has no column to compare. The algorithm has nothing to work with.

Content-based filtering goes the other way. It ignores other users entirely and instead represents each movie as a vector of its own attributes — genre, lead actor, director, language, runtime, keywords from the synopsis. It then recommends titles whose attribute-vector is close to the attribute-vector of things you rated highly before. This handles cold-start for new movies just fine (a new film's genre tags exist immediately), but it tends to trap you in a bubble: if you liked one Aamir Khan drama, you get five more Aamir Khan dramas and nothing genuinely new.

Production systems — Netflix, JioHotstar, Spotify, Amazon — run both simultaneously and blend the scores, often adding a third layer of context (time of day, device, what you searched five minutes ago). This chapter builds the mathematical core of the collaborative approach in full rigour, because it is the part built on genuinely elegant linear algebra, and it is the part that shows up in vector-algebra exam questions.

Turning Taste Into a Vector

Look again at Aisha's row in the table: (5, 4, 1). This is not just a list of three numbers — it is a point in three-dimensional space, or equivalently, an arrow from the origin to that point. Rohan's row (4, 5, 2) is a different arrow. Meera's row (1, 4, 5) is a third arrow. "Similar taste" now has a precise geometric meaning: two users have similar taste if their arrows point in roughly the same direction, regardless of how long the arrows are.

That last clause matters more than it looks. A user who rates everything 4s and 5s and a user who rates the exact same titles 1s and 2s, but in the same relative order, have identical taste — one is just an enthusiastic rater and the other stingy. Their arrows point in the same direction but have very different lengths. This is exactly why the industry-standard similarity measure for ratings is not "how far apart are the two points" but "what is the angle between the two arrows." We build that measure next, from scratch.

Deriving Cosine Similarity from the Dot Product

For two vectors a and b, the dot product is defined component-wise: a·b = a₁b₁ + a₂b₂ + ... + aₙbₙ. We want to connect this algebraic definition to the geometric angle θ between the vectors. Consider the triangle formed by a, b, and the difference vector a − b. The Law of Cosines for this triangle states:

|a − b|² = |a|² + |b|² − 2|a||b|cos θ

Now expand the same quantity |a − b|² using the dot product definition, since |v|² = v·v for any vector:

|a − b|² = (a − b)·(a − b) = a·a − 2(a·b) + b·b = |a|² − 2(a·b) + |b|²

Both expressions equal |a − b|², so set them equal to each other and cancel |a|² + |b|² from both sides:

−2(a·b) = −2|a||b|cos θ  ⟹  a·b = |a||b|cos θ

Rearranging gives the working formula, called cosine similarity:

cos θ = (a·b) / (|a| |b|)

This single number, always between −1 and 1 for non-negative rating vectors it lies between 0 and 1, tells you how aligned two taste-vectors are without caring how long either vector is. A value of 1 means identical direction (θ = 0°, identical taste), a value near 0 means the vectors are nearly perpendicular (unrelated taste).

Worked Example: Who Is Most Like Aisha?

Restrict attention to two shows first — 3 Idiots and Kota Factory — so the vectors live in a plane we can actually draw. From the table: Aisha = (5, 4), Rohan = (4, 5), Meera = (1, 4).

Step 1 — magnitudes. |Aisha| = √(5² + 4²) = √41 ≈ 6.403. |Rohan| = √(4² + 5²) = √41 ≈ 6.403. |Meera| = √(1² + 4²) = √17 ≈ 4.123.

Step 2 — dot products. Aisha·Rohan = (5)(4) + (4)(5) = 20 + 20 = 40. Aisha·Meera = (5)(1) + (4)(4) = 5 + 16 = 21.

Step 3 — cosine similarity.

cos θ(Aisha, Rohan) = 40 / (6.403 × 6.403) = 40 / 41 ≈ 0.9756  ⟹  θ ≈ 12.7°

cos θ(Aisha, Meera) = 21 / (6.403 × 4.123) = 21 / 26.40 ≈ 0.7954  ⟹  θ ≈ 37.3°

A smaller angle means higher similarity, so Aisha and Rohan (12.7°) share far closer taste than Aisha and Meera (37.3°). If Rohan has watched a fourth show that Aisha has not, that show becomes a strong recommendation for Aisha — this is the entire prediction step of user-based collaborative filtering: find the most similar user(s), recommend what they liked that you have not seen, optionally weighting the recommendation by the similarity score itself.

The diagram below plots exactly these three vectors on the (3 Idiots, Kota Factory) plane, with the two angles just computed marked to scale.

You can check the exact numbers with a five-line program — the same computation a real recommender does, just at a scale of millions rather than three:

def cosine_similarity(a, b):
    dot = sum(x * y for x, y in zip(a, b))
    mag_a = sum(x ** 2 for x in a) ** 0.5
    mag_b = sum(x ** 2 for x in b) ** 0.5
    return dot / (mag_a * mag_b)

aisha = (5, 4)   # (3 Idiots, Kota Factory)
rohan = (4, 5)
meera = (1, 4)

print(round(cosine_similarity(aisha, rohan), 4))   # 0.9756
print(round(cosine_similarity(aisha, meera), 4))   # 0.7954

Tracing this by hand: dot for (aisha, rohan) is 5×4 + 4×5 = 40; mag_a is (25+16)**0.5 = 6.4031; mag_b is identical since Rohan's two ratings are Aisha's swapped, giving the same sum of squares; the return value is 40 / (6.4031 × 6.4031) = 40/41 = 0.97561, which rounds to 0.9756 — matching the hand calculation exactly.

Adding a Third Dimension — and Why the Angle Changes

Real catalogues never stop at two titles, and the cosine formula does not care how many dimensions it is given — it works identically for 2, 3, or 15,000 dimensions, even though only 2 or 3 can ever be drawn on paper. Bring back Sacred Games as a third coordinate: Aisha = (5, 4, 1), Rohan = (4, 5, 2).

|Aisha| = √(25+16+1) = √42 ≈ 6.481. |Rohan| = √(16+25+4) = √45 ≈ 6.708. Aisha·Rohan = 20 + 20 + 2 = 42.

cos θ = 42 / (6.481 × 6.708) = 42 / 43.475 ≈ 0.9661  ⟹  θ ≈ 15.0°

Notice this is not the 12.7° from the two-dimensional diagram above — it is a genuinely different angle, computed in a genuinely different (three-dimensional) space, because both Aisha and Rohan rated Sacred Games differently enough (1 versus 2) to pull their vectors slightly further apart once that dimension is included. This is an important caveat and a common source of confusion: the angle you can physically draw for two dimensions is not the same object as the angle in the full-dimensional space once more titles are added — you cannot get the 3-D angle by eyeballing a 2-D picture, you must recompute it with all the coordinates included. A real system with fifteen thousand titles is computing an angle in fifteen-thousand-dimensional space that no diagram could ever show; the formula still works because it only ever needs sums of products and square roots, never an actual picture.

Common Misconception: "Numbers Close Together Means Similar Taste"

It is tempting to think you could skip cosine similarity entirely and just measure how far apart two rating vectors are — the ordinary straight-line distance, called Euclidean distance: dist(a,b) = √(Σ(aᵢ − bᵢ)²). This is a real and useful metric, but it measures something different from cosine similarity, and confusing the two produces wrong recommendations.

Consider two users on the same three shows, rating on the standard 1–5 scale: P = (5, 5, 5) — loved everything — and Q = (1, 1, 1) — rated everything at the bare minimum. Their ratios are identical: P rated every show exactly 5× higher than Q rated it. In direction, these users have identical taste — cos θ(P, Q) = (5+5+5)/(√75 × √3) = 15/(8.660×1.732) = 15/15 = 1.000 exactly, meaning θ = 0°. Cosine similarity correctly identifies them as a perfect match; P is simply a generous rater and Q a harsh one.

But their Euclidean distance is: √((5−1)² + (5−1)² + (5−1)²) = √(16+16+16) = √48 ≈ 6.93. On a 1–5 rating scale with three dimensions, the largest possible gap in any single coordinate is 5 − 1 = 4, so the theoretical maximum Euclidean distance between any two rating vectors is √(3 × 4²) = √48 ≈ 6.93 — and P and Q sit at exactly that maximum. By raw distance, P and Q look like the two most dissimilar users mathematically possible on this scale, even though by direction they are a perfect match. This is precisely why production recommender systems overwhelmingly favour cosine similarity (or a mean-centered variant of it called Pearson correlation) over raw Euclidean distance for rating data: it correctly separates "different taste" from "different generosity," and generosity varies enormously between real users for reasons that have nothing to do with taste — culture, mood, or just how someone personally uses a five-star scale.

def euclidean_distance(a, b):
    return sum((x - y) ** 2 for x, y in zip(a, b)) ** 0.5

p = (5, 5, 5)
q = (1, 1, 1)
print(round(euclidean_distance(p, q), 3))

Tracing this: each term (x − y)² is (5−1)² = 16, summed three times gives 48, and 48 ** 0.5 = 6.9282..., which rounds to 6.928 — the theoretical maximum for three dimensions on this scale, reached exactly because P and Q sit at opposite corners of the rating cube.

Item-Based Filtering and the Cold-Start Problem

Everything above is user-based collaborative filtering: find similar users, borrow their opinions. There is a mirror-image version called item-based collaborative filtering, which instead builds a vector for each movie out of every user's rating of it, then asks "which movies get rated similarly by the same people?" If everyone who rated 3 Idiots highly also rated Chak De India highly, the two films are similar in the item space — regardless of whether they share a genre tag. Amazon's original large-scale recommender ("customers who bought this also bought...") is a classic item-based system, and it tends to be more stable in practice because a catalogue's items change far more slowly than its user base.

Both user-based and item-based collaborative filtering share the same weakness: the cold-start problem. A brand-new user has an empty row — there is no vector to compare against anyone. A brand-new release has an empty column — no one has rated it yet, so it can never surface via collaborative filtering alone. This is exactly the gap content-based filtering fills: a new film can be represented immediately by its genre, cast, and language tags, letting the system make a reasonable first guess before a single rating exists. Netflix's onboarding screen, which asks new users to rate a handful of titles before showing a homepage, exists specifically to shrink this cold-start gap as fast as possible.

From Neighbours to Latent Factors: A Glimpse of Matrix Factorization

Comparing every user to every other user, or every item to every other item, does not scale to hundreds of millions of users and tens of thousands of titles — the number of pairwise comparisons grows too fast. Production-scale systems instead use matrix factorization: they try to approximate the giant, sparse rating matrix R as the product of two much smaller matrices, R ≈ U × Vᵀ, where U is a "users × k factors" matrix and V is an "items × k factors" matrix, with k typically a few dozen to a few hundred — far smaller than the number of actual titles. Each of these k hidden factors does not correspond to a labelled category like "comedy" or "Bollywood"; the algorithm discovers them purely by minimizing prediction error, and they often end up capturing genuinely useful latent structure (one factor might informally correspond to "slow-burn versus fast-paced," another to "critically acclaimed versus mass-audience," without ever being told to look for those things). This is the technique that won the famous 2006–2009 Netflix Prize competition, and it is the direct mathematical descendant of the cosine-similarity idea built in this chapter: instead of comparing raw rating vectors, you first compress every user and item down to a compact vector of learned factors, then measure similarity — often still via a dot product — in that much smaller space.

Where This Shows Up in Your Exams

CBSE's Artificial Intelligence skill subject (code 417, offered in Classes IX–X) introduces vector-based similarity as a hands-on AI project component, and the same ideas resurface in the senior-secondary AI elective's data-handling and case-study units — recommender systems are a standard worked example there precisely because the underlying mathematics is the dot-product-and-magnitude computation you have just done by hand. Outside the AI-specific paper, JEE Main and BITSAT both set numerical problems on vector dot products and direction cosines that are structurally identical to the cosine-similarity calculation in this chapter — the angle between two vectors, computed from cos θ = (a·b)/(|a||b|), is the same formula whether the vectors represent physical displacement or movie ratings. If you go on to study linear algebra formally at the undergraduate level or attempt Olympiad-style problems on inner product spaces, this cosine formula generalizes without any change — the dot product and the Cauchy–Schwarz inequality that guarantees cos θ never exceeds 1 in magnitude work identically in any number of dimensions.

Check Your Understanding

1. Two users have rating vectors A = (3, 6) and B = (6, 12) over two movies. Compute their cosine similarity and explain, in one sentence, what the result reveals about how they used the rating scale. (Work it out: A·B = 18+72 = 90, |A| = √45 ≈ 6.708, |B| = √180 ≈ 13.416, cos θ = 90/(6.708×13.416) = 90/90 = 1.000 — a perfect match; B rated everything exactly double A, same direction, different generosity.)
2. Why does adding a third rated title to a two-dimensional comparison generally change the computed angle between two users, even if neither user's original two ratings changed? Refer to the worked 2-D versus 3-D example above in your answer.
3. A new film has just been released on a platform with no ratings yet. Explain, using the vocabulary of this chapter, why collaborative filtering cannot recommend it to anyone, and name the specific technique that can.
4. Two users rate the same five movies, but User X always rates 1 star lower than User Y on every single title. Predict, without calculating, whether their cosine similarity will be closer to 1 or closer to 0, and separately predict whether their Euclidean distance will be small or large. Then verify with the vectors X = (2,3,4,3,2) and Y = (3,4,5,4,3).

Summary

A recommender system's core task is filling in the blank cells of a sparse user-item rating matrix. Collaborative filtering does this by treating each user's ratings as a vector and finding other users whose vectors point in a similar direction, using cosine similarity — derived directly from the dot product via the Law of Cosines as cos θ = (a·b)/(|a||b|) — because it correctly separates taste (direction) from generosity (magnitude), unlike raw Euclidean distance. Item-based filtering runs the same idea on movie-vectors instead of user-vectors. Both approaches suffer from the cold-start problem, which content-based filtering (comparing item attributes rather than ratings) partly solves. At production scale, matrix factorization compresses the rating matrix into small learned user- and item-factor vectors so that millions of similarity comparisons become computationally feasible — the same cosine-similarity mathematics, just applied in a compressed space.

Think About It

Think about this: How would you explain recommender systems: netflix for you to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.