t-SNE and UMAP

The big idea

You have 800 handwritten digits, each a point in 64 dimensions, and you want to see whether the 0s, 1s and 7s form separate groups. PCA gives you the best flat 2D shadow of the cloud, but on these digits that shadow mixes the classes badly: in the code, a nearest-neighbour classifier reading the PCA map gets only 51 percent right.

t-SNEt-distributed Stochastic Neighbor Embedding: a non-linear method that places points in 2D or 3D so that each point's nearest neighbours in the original space stay its nearest neighbours in the map. and UMAPUniform Manifold Approximation and Projection: a non-linear embedding method that builds a nearest-neighbour graph in the original space and lays it out in low dimensions; similar in spirit to t-SNE, usually faster. give up on a faithful shadow. They only try to keep each point's neighbours next to it. Think of seating guests at a wedding: you cannot reproduce everyone's exact relationships on a floor plan, but you can make sure friends sit at the same table. Tables end up clearly separate, and that is the point. How far apart two tables are, and how big each one is, tells you little.

That is the whole lesson in one sentence: these maps are excellent for seeing clusters, and unreliable for distances between clusters or cluster sizes.

How it works

Step 1: Turn distances into neighbour probabilities

For each point i, t-SNE puts a Gaussian bell around it and asks: if i picked a neighbour at random, weighted by that bell, how likely is it to pick j? Close points get high probability, far points almost zero. These probabilities, symmetrized, form the matrix P. Only the local neighbourhood matters; the exact distance to a far-away point is thrown away at this step.

Step 2: Choose the bell width with perplexity

Dense regions need a narrow bell and sparse regions a wide one, so each point gets its own width. You control them all with one number, perplexityThe main t-SNE setting: roughly the effective number of neighbours each point pays attention to. Typical values are 5 to 50., which is roughly how many neighbours each point should care about. t-SNE searches each point's width until its bell covers about that many neighbours. Low perplexity (5) focuses on very local structure and can break real groups into fragments; high perplexity (100) looks wider and shows more global arrangement but can blur small groups.

Step 3: Place points in 2D and compare

Start the map from PCA (or random) positions. In the map, similarity uses a Student-t distributionA bell-shaped distribution with heavier tails than the normal distribution. With one degree of freedom it decays like 1 / (1 + d squared). instead of a Gaussian, giving the matrix Q:

The heavy tail matters. A 2D map has far less room than 64 dimensions, so moderately distant points cannot all be placed at the right distance; this is the crowding problem. Heavy tails let dissimilar points sit far apart in the map at little cost, which is what pulls clusters into clean, separated islands.

Step 4: Minimize the mismatch by gradient descent

The loss is the KL divergenceKullback-Leibler divergence: a measure of how one probability distribution differs from another. It is zero only when they match. between P and Q:

It punishes heavily when real neighbours (large p) end up far apart in the map (small q), and only lightly when non-neighbours end up close. Gradient descent moves the 2D points for about 1000 iterations. Nothing here is a formula you can apply to a new point; the map is the result of an optimization over these specific points.

Step 5: UMAP, same goal, different machinery

UMAP builds a weighted k-nearest-neighbour graph in the original space (its n_neighbors plays the role of perplexity), then lays the graph out in 2D with attractive forces along edges and repulsive forces between random non-neighbours. Its min_dist controls how tightly points pack within a cluster. In practice UMAP is much faster on large datasets, often keeps a little more of the global arrangement, and can transform new points with a fitted model. Its maps carry the same warnings as t-SNE's.

Going deeper

Code: perplexity, cluster size and gaps between clusters

import numpy as np
from sklearn.datasets import load_digits
from sklearn.decomposition import PCA
from sklearn.manifold import TSNE, trustworthiness
from sklearn.neighbors import KNeighborsClassifier
from sklearn.model_selection import cross_val_score

# 800 handwritten digits, 64 pixels each, squeezed into 2D
D, y = load_digits(return_X_y=True)
D, y = D[:800], y[:800]

def report(name, Z):
    # trustworthiness: are 2D neighbours also neighbours in 64D? (1.0 = perfect)
    t = trustworthiness(D, Z, n_neighbors=10)
    # can a 5-NN classifier read the digit from the 2D map alone?
    acc = cross_val_score(KNeighborsClassifier(5), Z, y, cv=5).mean()
    print(f"{name:<16} trustworthiness {t:.3f}   5-NN accuracy in 2D {acc:.3f}")

report("PCA", PCA(2).fit_transform(D))
for perp in [5, 30, 100]:
    report(f"t-SNE perp={perp}", TSNE(perplexity=perp, init="pca", random_state=0).fit_transform(D))

# Cluster SIZE is not preserved: a tight blob and a blob 10x wider
rng = np.random.default_rng(0)
A = rng.normal(0, 1, (200, 10))
B = rng.normal(0, 10, (200, 10)) + 100
Z = TSNE(perplexity=30, init="pca", random_state=0).fit_transform(np.vstack([A, B]))
print("\nspread in 10D:  tight", round(A.std(), 2), " wide", round(B.std(), 2))
print("spread in t-SNE: tight", round(Z[:200].std(), 2), " wide", round(Z[200:].std(), 2))

# Distances BETWEEN clusters are not preserved: centres at 0, 10 and 100
C = np.vstack([rng.normal(c, 1, (150, 10)) for c in [0, 10, 100]])
Z = TSNE(perplexity=30, init="pca", random_state=0).fit_transform(C)
m = [Z[i * 150:(i + 1) * 150].mean(0) for i in range(3)]
d = lambda a, b: np.linalg.norm(a - b)
print(f"\ntrue gaps:  A-B {d(np.full(10, 0), np.full(10, 10)):.1f}   B-C {d(np.full(10, 10), np.full(10, 100)):.1f}")
print(f"t-SNE gaps: A-B {d(m[0], m[1]):.1f}   B-C {d(m[1], m[2]):.1f}")

PCA              trustworthiness 0.799   5-NN accuracy in 2D 0.512
t-SNE perp=5     trustworthiness 0.989   5-NN accuracy in 2D 0.946
t-SNE perp=30    trustworthiness 0.993   5-NN accuracy in 2D 0.916
t-SNE perp=100   trustworthiness 0.990   5-NN accuracy in 2D 0.915

spread in 10D:  tight 1.0  wide 9.96
spread in t-SNE: tight 11.37  wide 14.75

true gaps:  A-B 31.6   B-C 284.6
t-SNE gaps: A-B 40.4   B-C 39.6

The first block scores each 2D map by trustworthinessA score from 0 to 1 for an embedding: how well the map's nearest neighbours are also nearest neighbours in the original space. It penalizes points that look close in the map but were far apart. (are map neighbours also true neighbours?) and by how well a 5-nearest-neighbour classifier can read the digit from the map. PCA scores 0.799 and 0.512; t-SNE scores 0.99 and over 0.91 at every perplexity tried. Local structure survives very well. (The classifier number only measures how separated the classes look in this map; t-SNE saw all 800 points, so it is not an honest test accuracy.)

What the map does not tell you

The code builds two more traps. A tight blob (spread 1.0) and a blob ten times wider (spread 9.96) come out of t-SNE at similar sizes, 11.37 and 14.75, because each point's bell adapts to its local density. Three clusters with a 9 times difference in gaps come out evenly spaced (Figure 2). So:

• Cluster sizes in the map do not reflect how spread out the clusters really are.
• Distances between clusters are not meaningful; two islands next to each other may not be more similar than two at opposite corners.
• Small clumps can be artefacts. At low perplexity, random noise can form convincing little clusters. Check that a structure survives several perplexity values and random seeds.
• Axes mean nothing. Unlike PCA, the x and y axes have no interpretation, and the map can be rotated or flipped freely.

Practical recipe

Scale features first. For very wide data (thousands of features), reduce to about 50 dimensions with PCA before running t-SNE; it removes noise and speeds things up. Try perplexity 5, 30 and 50, and keep conclusions that hold for all of them. Use PCA initialization (scikit-learn's default today), which keeps more of the global layout than random starts. And never feed t-SNE coordinates into a downstream model as features: the map is not a stable function of the input.

Trade-offs

Common mistakes

• Reading the distance between two clusters in the map as how different they are.
• Concluding a group is more varied because its island looks bigger.
• Trusting a cluster that appears at one perplexity and vanishes at others.
• Using t-SNE or UMAP coordinates as input features for a model.
• Running t-SNE on raw, unscaled, very wide data without a PCA step first.

In the wild

t-SNE and UMAP are the standard first picture of any embedding space. Single-cell biology uses them to show thousands of cells grouped into cell types. ML teams plot sentence or image embeddings with them to see whether classes separate, to find mislabelled examples sitting inside the wrong island, and to explore what a model has learned. Embedding projector tools in common ML toolkits offer PCA, t-SNE and UMAP side by side for exactly this reason.

Check yourself

Recap