t-SNE and UMAP: Beautiful Data Visualization

The problem: a picture needs 2 axes, your data has hundreds

Take a single handwritten digit image from the MNIST dataset: 28 pixels by 28 pixels, grayscale. To a neural network, that image is not a picture at all. It is a point in a 784-dimensional space — one axis per pixel, each axis measuring a brightness value from 0 to 255. A dataset of 10,000 such digits is 10,000 points scattered somewhere inside this 784-dimensional cube. If you trained a classifier and it reports 97% accuracy, a reasonable next question is: did the network actually learn to separate the ten digit classes into distinct regions, or did it get lucky? You cannot answer that by staring at 784 numbers per image. You need to see the shape of the data. But you have two eyes and a flat screen, and 784 dimensions do not fit into 2.

You already know one tool for this: Principal Component Analysis. PCA finds the directions along which the data varies the most and projects onto the top two or three of them. It is fast, exact, and mathematically clean — but it is a linear projection. It can only rotate and flatten the cloud of points; it cannot bend it. Picture a rolled-up sheet of paper with a spiral pattern drawn on it, sitting in 3D space (this is the classic "Swiss roll" example in manifold learning). Two points can be close together in 3D coordinates while being far apart if you measured distance along the sheet — and two points can be far apart in 3D while lying right next to each other once the paper is unrolled. PCA looks only at straight-line (Euclidean) distance in the original space, so it cannot tell these cases apart; it will happily project the rolled-up spiral into a flat, meaningless smear, because the directions of maximum variance in 3D do not correspond to the directions of maximum variance along the rolled sheet. What you actually want preserved is the local neighborhood structure — which points are near which other points if you only trust distances over short hops — even if that means distorting the global geometry to make it fit on a page. That is exactly the job t-SNE and UMAP were built for.

The shared idea: preserve neighborhoods, not raw distances

Both algorithms follow the same three-part recipe, even though the mathematics underneath differs:

• Build a graph (or a probability distribution) in the original high-dimensional space that says, for every point, "how much does this point consider each other point to be a neighbor?"
• Place all the points in 2D (or 3D) and build the same kind of graph in that low-dimensional space.
• Move the low-dimensional points around, using gradient descent, until the low-dimensional neighbor graph matches the high-dimensional one as closely as possible.

The key design decision — the one that makes t-SNE and UMAP different from a naive "just try to preserve all pairwise distances" approach — is that they deliberately do not try to preserve large distances accurately. A point that is very far away in 784-dimensional space is uninformative anyway; what matters for understanding structure is which points are close. This is a trade-off, and it has real consequences we will come back to.

Step 1 — Turning distances into probabilities: the Gaussian neighborhood

t-SNE (t-distributed Stochastic Neighbor Embedding, introduced by Laurens van der Maaten and Geoffrey Hinton in 2008) starts by converting every pairwise distance in the high-dimensional data into a conditional probability. For points x_i and x_j, define:

p(j|i) = exp(-||x_i - x_j||^2 / 2*sigma_i^2) / sum over k != i of exp(-||x_i - x_k||^2 / 2*sigma_i^2)

Read this as: "if I stood at point i and threw darts according to a Gaussian bell curve centered on myself, with spread σ_i, what fraction of darts would land closest to j?" Nearby points get high probability, far points get probability that shrinks off exponentially fast, and each point's own row of probabilities sums to 1 (excluding itself, p(i|i) = 0).

Two details matter here. First, every point i gets its own σ_i — a dense cluster needs a small spread to distinguish close neighbors from each other, while an isolated point needs a large spread just to have any real neighbors at all. Second, p(j|i) is not the same number as p(i|j) in general, because the normalizing sum is different at each point. To get a single, symmetric measure of how similar i and j are, t-SNE defines the joint probability as the average of the two conditionals:

p_ij = (p(j|i) + p(i|j)) / (2N)

where N is the number of points, so that the sum of p_ij over every pair adds up to 1 across the whole dataset. This averaging step is not cosmetic. In the earlier method t-SNE improved on (plain SNE), only the asymmetric p(j|i) was used, and a genuinely important outlier point — one whose nearest neighbors are all still comparatively far away — ended up with tiny probabilities in every other point's row, contributing almost nothing to the optimization and getting placed almost at random in the final map. Averaging the two conditionals guarantees every point contributes a non-negligible amount, regardless of local density.

Perplexity: from entropy to "effective neighbor count"

Each σ_i is not chosen by hand — it is calibrated automatically using a user-set parameter called perplexity, and this is where a genuinely useful piece of information theory earns its keep. Define the Shannon entropy of point i's neighbor distribution, measured in bits:

H(P_i) = - sum over j of p(j|i) * log2( p(j|i) )

and then define perplexity as Perp(P_i) = 2^H(P_i). Why is this a sensible "effective neighbor count"? Check the cleanest possible case: suppose i's probability mass is spread exactly uniformly over k neighbors, so p(j|i) = 1/k for each of them. Then:

H = - sum_{j=1}^{k} (1/k) * log2(1/k) = -k * (1/k) * log2(1/k) = -log2(1/k) = log2(k)
Perplexity = 2^H = 2^(log2 k) = k

So in the idealized uniform case, perplexity comes out to exactly k — the number of neighbors. For a real, non-uniform distribution, perplexity interpolates smoothly: a distribution with mass piled almost entirely on one neighbor has entropy near 0 and perplexity near 1; a distribution spread thinly over many points has high entropy and high perplexity. t-SNE runs a binary search on σ_i, separately for every point, until Perp(P_i) hits the target value the user chose (typically between 5 and 50).

Let's compute this by hand for a genuinely asymmetric case so the effect of σ is not just abstract. Take four points on a number line: x₁=0, x₂=1, x₃=2, x₄=10, and look at point x₁'s row.

With σ₁ = 1: squared distances from x₁ are 1, 4, 100. Unnormalized weights exp(-d²/2): exp(-0.5) = 0.6065, exp(-2) = 0.1353, exp(-50) ≈ 0. Sum = 0.7419. So p(2|1) = 0.6065/0.7419 = 0.8176, p(3|1) = 0.1824, p(4|1) ≈ 0. Plugging into the entropy formula: H = -(0.8176 × log2(0.8176) + 0.1824 × log2(0.1824)) ≈ -(0.8176 × (-0.2904) + 0.1824 × (-2.456)) ≈ 0.685 bits, so Perp = 2^0.685 ≈ 1.61. With σ=1, point 1 effectively "sees" only about 1.6 neighbors — almost entirely x₂.

With σ₁ = 5: weights become exp(-d²/50): exp(-0.02) = 0.9802, exp(-0.08) = 0.9231, exp(-2) = 0.1353. Sum = 2.0387, giving p(2|1) = 0.4808, p(3|1) = 0.4528, p(4|1) = 0.0664. Now H ≈ 1.286 bits, so Perp ≈ 2^1.286 ≈ 2.44 — the far-away point x₄ now counts for something, and the effective neighborhood has grown. This is exactly what perplexity controls: it is a knob for "how many points around me should be treated as genuinely close," and t-SNE's binary search finds the σ_i that hits your chosen value for every point individually, regardless of local density.

You can verify the first case in three lines of NumPy:

import numpy as np
x = np.array([0, 1, 2, 10], dtype=float)
i = 0
sigma = 1.0
d2 = (x - x[i])**2      # [0, 1, 4, 100]
d2[i] = np.inf           # exclude the point itself
w = np.exp(-d2 / (2 * sigma**2))
p = w / w.sum()
print(np.round(p, 4))    # approximately [0.0, 0.8176, 0.1824, 0.0]

Tracing it: d2 becomes [inf, 1, 4, 100] after the self-exclusion; w becomes [0, 0.6065, 0.1353, ~1.9e-22]; dividing by the sum (≈ 0.7419) gives the same 0.8176 / 0.1824 / ≈0 split computed by hand above. Re-running with sigma = 5.0 reproduces the second row, ≈ [0, 0.4808, 0.4528, 0.0664].

Step 2 — The crowding problem, and why t-SNE needs a heavy tail

Now place the points in 2D and define a similarity q_ij there too. The naive choice would be to reuse a Gaussian, but this creates a specific, well-diagnosed failure called the crowding problem. Here is the geometric reason it happens. In a D-dimensional space, the volume of a thin shell at radius r grows roughly like r^D-1. For large D, almost all of the volume "available" around a point sits at moderate-to-large radius, not close in — so a point in high dimensions can comfortably have a huge number of neighbors that are all moderately similar to it, at roughly the same distance. But a 2D map has area growing only linearly with radius for a fixed-width ring. There simply is not enough room in 2D to place all of those moderately-similar neighbors at moderate distances from the point and keep the genuinely close neighbors close. If both spaces use the same (Gaussian) notion of similarity, the optimizer is forced to crush moderately-similar points inward just to keep them from looking "too dissimilar," and the whole map collapses toward one dense blob in the middle.

t-SNE's fix is to make the low-dimensional similarity decay much more slowly with distance than the high-dimensional one, using the Student-t distribution with one degree of freedom (mathematically identical to the Cauchy distribution) — this heavy tail is where the "t" in t-SNE comes from:

q_ij = (1 + ||y_i - y_j||^2)^(-1) / sum over k != l of (1 + ||y_k - y_l||^2)^(-1)

Compare how fast the two kernels fall off. Using unnormalized weights g(d) = exp(-d²/2) for Gaussian and c(d) = 1/(1+d²) for the t-kernel:

• At d = 1: g(1) = 0.6065, c(1) = 0.5000 — fairly close to each other.
• At d = 4: g(4) = 0.000335, c(4) = 0.0588 — wildly different.

The Gaussian weight fell by a factor of about 1810× between d=1 and d=4; the t-kernel weight fell by only about 8.5×. That gap is the entire fix in one comparison. A pair of points that are only moderately similar in the original space ends up with a moderate p_ij. To match that with a Gaussian q_ij in 2D, the points would have to sit fairly close together (since q decays so fast). With the heavy-tailed t-kernel, that same moderate p_ij can be matched by placing the points comparatively far apart in the map — the slow decay means a large 2D distance still "costs" only a modest drop in similarity. This frees up space near true, tight clusters, and pushes the moderately-related points outward, which is exactly what produces the crisp, well-separated clusters t-SNE plots are known for.

Step 3 — The cost function: Kullback-Leibler divergence as a tug-of-war

With p_ij fixed once (computed from the original data) and q_ij depending on the current guess for the 2D positions y_i, t-SNE minimizes the Kullback-Leibler divergence between the two joint distributions:

C = sum over i, sum over j of  p_ij * log( p_ij / q_ij )

KL divergence is not symmetric and it is not a true distance, but it has a useful property here: it penalizes q_ij being much smaller than p_ij far more heavily than it penalizes q_ij being much larger. In plain terms — if two points are truly similar in the original space (p_ij large) but end up far apart on the map (q_ij small), the cost is large. If two genuinely dissimilar points (p_ij small) happen to land close together on the map (q_ij larger than it should be), the cost is comparatively small. This asymmetry is exactly why t-SNE is obsessive about getting local neighborhoods right and comparatively careless about global arrangement — a design choice, not an oversight, and the direct cause of the misconception in the next section.

Minimizing C by gradient descent gives (this is a standard, well-verified result, not something we re-derive term by term here — but you can check its behavior is sensible):

dC/dy_i = 4 * sum over j of (p_ij - q_ij) * (y_i - y_j) * (1 + ||y_i - y_j||^2)^(-1)

Check the sign in two situations. Suppose p_ij = 0.30 (the points should be quite similar) but currently q_ij = 0.05 (the map has drawn them far apart) — so p_ij - q_ij = +0.25. Gradient descent updates each position as y_i ← y_i - η × dC/dy_i (η is the learning rate); the positive coefficient in front of (y_i - y_j) means subtracting the gradient moves y_i in the direction of -(y_i - y_j), i.e. toward y_j. Points that should be closer get pulled together — an attractive force. Now suppose p_ij = 0.02 but the map currently has q_ij = 0.20 (too close): p_ij - q_ij = -0.18, the coefficient flips negative, and the update pushes y_i away from y_j instead — a repulsive force. Every pair of points in the dataset exerts one of these two forces on every other pair simultaneously, and gradient descent (in practice, accelerated with momentum, run for hundreds to a few thousand iterations) settles into a local equilibrium where attraction and repulsion roughly balance. This is why t-SNE plots are sometimes described as a physical simulation of springs and charges — that description is not just a metaphor, it falls directly out of this gradient's structure.

UMAP: a different mathematical route to a similar-looking map

UMAP (Uniform Manifold Approximation and Projection, introduced by Leland McInnes, John Healy, and James Melville in 2018) often produces plots that look like t-SNE's — tight, well-separated clusters — but it is built on a different mathematical foundation: it treats the data as samples from a manifold and constructs a weighted neighbor graph using ideas from topology, rather than starting from a Gaussian-probability interpretation. The mechanics, stated precisely (the full topological justification, involving fuzzy simplicial sets, is genuinely graduate-level and out of scope here, but the algorithm itself is exactly this):

• For each point i, find its k nearest neighbors (UMAP's main parameter, n_neighbors, plays a role similar to perplexity — bigger k means more emphasis on global structure, smaller k means more emphasis on fine local detail).
• Let ρ_i be the distance from i to its single nearest neighbor. This is used as a local zero-point, guaranteeing every point is fully connected to at least its closest neighbor regardless of how dense or sparse its neighborhood is — this is UMAP's main defense against the very different local densities that different regions of real data tend to have.
• Calibrate a local scale σ_i (by binary search, in direct analogy to t-SNE's perplexity search) so that the sum over j of exp(-(d_ij - ρ_i) / σ_i) equals log2(k), giving each point a comparable total connectivity budget.
• Symmetrize using the fuzzy set union rather than a simple average: w_ij = w(i→j) + w(j→i) - w(i→j)×w(j→i). This says a strong edge in either direction is enough to make the pair well-connected — it does not get diluted the way a plain average would if only one of the two directions was confident.
• In the low-dimensional map, fit a smooth curve φ(d) = 1 / (1 + a×d^2b) (with a, b found by curve-fitting to the user's min_dist parameter, which controls how tightly points are allowed to pack) and minimize a binary cross-entropy between the high-D fuzzy graph weights and the low-D curve — not KL divergence.

Two practical consequences follow directly from these differences. First, cross-entropy (unlike KL divergence) penalizes both directions of mismatch — placing dissimilar points too close is punished about as much as placing similar points too far apart — so UMAP's optimization keeps a somewhat more honest sense of global layout than t-SNE's, though "somewhat more honest" is still far from "you can read off exact distances." Second, UMAP does not need to compute or store the full q_ij over every pair at every step; it approximates the repulsive term using negative sampling — a small random subset of non-neighbor pairs at each update, borrowed from the same trick used to train word embeddings — which is a major reason UMAP scales to millions of points noticeably faster than the classic t-SNE formulation (modern accelerated implementations of t-SNE, such as Barnes-Hut and FIt-SNE, close much of this speed gap, but the underlying algorithm here is genuinely different, not just re-engineered).

The misconception that ruins more class projects than any other

Here is the single most important thing to internalize, and it is not optional if you are going to put a t-SNE or UMAP plot in a project report: the distance between two clusters on the map, and the size or density of a cluster on the map, are not reliable measurements of anything in the original data.

Trace back why. t-SNE's cost function was built, deliberately, to punish getting local neighbors wrong far more than it punishes getting global arrangement wrong — that was the entire point of using KL divergence with an asymmetric penalty. Two clusters that end up on opposite corners of your plot might be almost equally different from a third cluster sitting in the middle; the 2D distances simply do not carry that information reliably, because nothing in the optimization asked them to. Cluster size on the map is driven largely by how many points are competing for space and by the perplexity you chose, not by how much genuine variance exists within that group in the original 784, or 20,000, dimensions. This is well documented — the interactive analysis "How to Use t-SNE Effectively" (Wattenberg, Viégas, and Johnson, published on Distill in 2016) is the standard reference, and it shows the same underlying dataset producing visibly different cluster spacing, tightness, and even apparent cluster count, purely from changing the perplexity value, with nothing about the data itself changed. UMAP's cross-entropy cost is somewhat less extreme about this than t-SNE's KL divergence, and there is genuine, ongoing research debate about how much of UMAP's apparent global structure is trustworthy — but treat it as "somewhat better, still not literal" rather than "solved."

What you can trust, from either method: which points are near which other points (the thing both algorithms are explicitly optimizing for), and therefore whether a classifier's learned features are actually separating your classes into distinct neighborhoods. What you cannot trust: "cluster A is twice as far from cluster B as it is from cluster C, so A is twice as related to C as to B" — that sentence is not a claim either algorithm's cost function ever tried to make true.

Where this fits in your exams and your toolkit

Be precise about this rather than inflating it: t-SNE and UMAP are not on the JEE Main/Advanced or BITSAT syllabus, which stay within Physics, Chemistry, and Mathematics and do not cover machine learning. Where this genuinely matters for a CBSE student is different but real. CBSE's Artificial Intelligence and Data Science skill subjects at the senior-secondary level include units on data visualization and unsupervised pattern-finding, and any class project that clusters real data — student performance across topics, images, text, sensor readings — is a natural place to apply and, more importantly, to correctly interpret one of these plots (the previous section is the difference between a project that gets full marks for insight and one that gets marked down for over-reading a picture). PCA is the dimensionality-reduction technique most likely to appear by name and require computation in a GATE Data Science and AI paper; t-SNE and UMAP are not always named explicitly in foundational syllabi, but they are the natural nonlinear extension of the same idea and are fair game for conceptual questions in any research-oriented context — KVPY-style fellowship interviews, Informatics Olympiad-adjacent data-analysis rounds, or undergraduate research applications — where being able to explain why a visualization technique works, and what it does not tell you, is what separates a real understanding from a plotted picture.

Outside exams, this pair of techniques shows up constantly in real Indian technical work: genomics groups doing single-cell RNA sequencing routinely use UMAP to visualize which cells cluster by type before any labels are assigned; teams working with hyperspectral or multispectral satellite imagery, where every pixel carries a value in a hundred-plus spectral bands, use dimensionality reduction as a first exploratory step before land-cover classification; and anyone building or debugging a classifier — image, text, or otherwise — uses exactly the MNIST-style sanity check from the opening of this chapter to confirm the model's internal features actually separate the categories it is supposed to distinguish.

Summary

• High-dimensional data cannot be plotted directly; PCA only removes dimensions along straight lines, so it fails on curved (nonlinear) structure. t-SNE and UMAP instead try to preserve local neighborhoods, accepting distortion of global distances as the price.
• t-SNE converts high-D distances into a Gaussian-based conditional probability p(j|i) per point, symmetrizes it into a joint p_ij, and calibrates each point's spread σ_i via a perplexity target — computable, in the uniform case, exactly as "effective number of neighbors," Perplexity = 2^H.
• In 2D, using the same Gaussian shape causes the "crowding problem" because volume in high-D grows much faster with radius than area does in 2D. t-SNE fixes this with a heavy-tailed Student-t (Cauchy) kernel for q_ij, whose weight falls off roughly 200× more slowly than a Gaussian between d=1 and d=4 in the worked comparison above.
• The cost function is KL divergence, minimized by gradient descent whose sign structure produces attractive forces when p_ij > q_ij and repulsive forces when p_ij < q_ij — a genuine spring-and-charge simulation, not just a metaphor.
• UMAP builds a fuzzy neighbor graph from local distances rescaled by nearest-neighbor distance ρ_i and a calibrated σ_i, symmetrizes with a fuzzy-set union, and minimizes cross-entropy against a fitted low-D curve — producing similar-looking plots through a different, generally faster, and marginally more global-structure-preserving route.
• Never read cluster-to-cluster distance or cluster size on either plot as a quantitative measurement — both were explicitly optimized to prioritize local neighbor accuracy over global geometry.

Active recall

• For the point x₁=0 among {0, 1, 2, 10}, recompute p(j|1) and the perplexity using σ₁ = 2. (Hint: squared distances are still 1, 4, 100; divide each by 2σ² = 8 this time before exponentiating, and expect a perplexity between the σ=1 and σ=5 answers worked out above — roughly 2.0.)
• Explain, using the volume-growth argument, why the crowding problem is specifically a consequence of projecting to a low-dimensional target, and would not appear if you were "reducing" 784 dimensions down to, say, 50 instead of 2.
• A classmate shows you a t-SNE plot of exam scores and says, "Group A and Group C are almost touching, so those students must be nearly identical, while Group B is far from both, so it's very different." Identify the specific mathematical property of the KL-divergence cost function that this claim ignores, and explain what you would check instead to compare the groups honestly.
• Using the gradient sign rule from Step 3, work out what happens if p_ij = q_ij exactly for some pair. What does that imply about how gradient descent treats pairs that are already correctly placed?
• UMAP's symmetrization is w_ij = w(i→j) + w(j→i) - w(i→j)×w(j→i). Plug in w(i→j) = 0.9 and w(j→i) = 0.1 and compute w_ij. Then compute what a plain average, (0.9+0.1)/2, would have given instead, and explain in one sentence why UMAP's designers preferred the fuzzy-union formula for this case.
• Why would increasing UMAP's n_neighbors parameter (analogous to raising t-SNE's perplexity) tend to produce a map that better preserves relationships between distant clusters, at the cost of blurring fine local detail within a cluster?

Think About It

Think about this: How would you explain t-sne and umap: beautiful data visualization to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.