Lie Groups and Symmetries: Continuous Groups in Deep Learning and Geometric Computing

A Photograph That Should Not Need Retraining

An ISRO Earth-observation satellite does not always photograph a riverbed with "north" pointing to the top of the frame. Depending on the orbit pass, the same riverbed can appear in an image rotated by 17 degrees, 63 degrees, or any of infinitely many other angles. A classifier trained to spot riverbeds should say "riverbed" regardless of that angle. A plain convolutional neural network (CNN) trained only by feeding it many rotated copies of the same riverbed will eventually learn to handle most of them, but it is learning the rotations statistically, one example at a time, because nothing in its architecture actually knows that rotation is a symmetry of the problem.

Compare this with translation. A CNN recognizes a cat whether it sits in the top-left or bottom-right corner of a photo, and it does this without ever needing separately labelled examples of "cat in every possible position." The reason is architectural, not statistical: convolution reuses the same small set of weights at every location in the image, so learning what an edge or a whisker looks like once is enough — the network never had to be taught position separately. Translation symmetry is baked into the layer itself.

The natural question is whether rotation, and other continuous symmetries, can be baked into a network the same principled way. Answering it requires the right mathematical object for "a symmetry with infinitely many elements that vary continuously" — that object is called a Lie group. This chapter builds the idea from the ground up: what makes a continuous symmetry different from a finite one, how continuity lets you differentiate a group at its identity element to extract its "generator," and how that machinery, under the modern name equivariance, is now standard toolkit in geometric deep learning — used in systems ranging from satellite image classifiers to protein-structure predictors.

Groups You Already Know Are Discrete

Recall the four group axioms using a concrete, finite example: the symmetries of an equilateral triangle. There are exactly six transformations that map the triangle back onto itself: the identity, rotations by 120 degrees and 240 degrees about the centre, and three reflections through each vertex. Call this set D₃. Composing any two of these six transformations gives another one of the six (closure); doing them in sequence three at a time does not depend on how you group the sequence (associativity); "do nothing" is in the set and changes nothing (identity); and every transformation can be undone by another transformation in the set (inverse). This is a completely ordinary, finite group with a 6-row multiplication table you could write out by hand.

Now replace the triangle with a perfectly round wheel — a chakra with no marked spokes. Every rotation by any angle θ ∈ [0°, 360°) maps the wheel back onto itself. This is still a group under the same composition rule (do one rotation, then another), but something new has appeared: there are uncountably many elements, and they have a notion of closeness. A rotation by 1° is close to a rotation by 1.001° in a way that has no analogue in the six discrete symmetries of the triangle. Because the elements vary continuously and the group operations (composing two rotations, undoing one) are smooth — differentiable — functions of the angle, this rotation group is not just a group, it is simultaneously a smooth curved space, called a manifold. A group that is also a smooth manifold, with multiplication and inversion as smooth maps, is called a Lie group, named after the Norwegian mathematician Sophus Lie, who studied continuous symmetry groups of differential equations in the 1870s.

SO(2): The Rotation Group as a Circle

Write a rotation of the plane by angle θ as a 2×2 matrix acting on column vectors:

R(θ) = [ cosθ   -sinθ ]
        [ sinθ    cosθ ]

Three facts, each provable with tools you already have, show that the set of all such matrices — called SO(2), the "special orthogonal group in 2 dimensions" — is a group under matrix multiplication.

Closure. Multiply R(α) and R(β) directly and use the angle-addition formulas cos(α+β) = cosαcosβ − sinαsinβ and sin(α+β) = sinαcosβ + cosαsinβ from Class 11 trigonometry. The top-left entry of R(α)R(β) works out to cosαcosβ − sinαsinβ, which is exactly cos(α+β). Carrying this through all four entries gives R(α)R(β) = R(α+β) exactly. Composing two rotations is another rotation, so the set is closed.

Identity and inverse. R(0) = I, the identity matrix, and R(θ)R(−θ) = R(0) = I, so R(−θ) is the inverse of R(θ).

Associativity is inherited automatically from matrix multiplication, which is always associative.

Because R(α)R(β) = R(α+β) is literally addition of the real parameter θ (taken modulo 2π), and because the map θ ↦ −θ is smooth, SO(2) is a Lie group — arguably the simplest nontrivial one that exists. Geometrically, since θ only matters modulo 2π, the space of elements of SO(2) is a circle: walking all the way around it by 2π brings you back to the identity. A tiny arc of that circle looks just like a small stretch of the real number line — the same way a small patch of the Earth's curved surface looks locally flat to someone standing on it, even though the whole surface is curved. That "locally flat, globally curved" property is exactly what the word manifold means.

import numpy as np

def R(theta):
    c, s = np.cos(theta), np.sin(theta)
    return np.array([[c, -s],
                      [s,  c]])

a, b = np.pi / 6, np.pi / 4          # 30 degrees, 45 degrees
lhs = R(a) @ R(b)
rhs = R(a + b)
print(np.allclose(lhs, rhs))          # True: closure confirmed, R(a)R(b) = R(a+b)

Differentiating a Group: The Generator and the Exponential Map

Because SO(2) is a smooth manifold, an operation that makes no sense for the discrete group D₃ becomes available here: differentiating the group's own multiplication rule at the identity. This single idea is the technical heart of Lie theory, so it is worth deriving carefully rather than quoting the result.

Start from the closure identity proven above, R(θ+ε) = R(θ)R(ε), and expand R(ε) for small ε using its own Taylor series around ε = 0. Since R(0) = I, write R(ε) ≈ I + εJ, where J = R′(0) is the matrix of derivatives of each entry of R(θ) evaluated at θ = 0. Substituting:

R(θ+ε) ≈ R(θ)(I + εJ) = R(θ) + εR(θ)J

Rearranging and taking ε → 0 turns this into a differential equation:

dR/dθ = R(θ) J,   R(0) = I

This is a matrix version of the scalar equation dy/dx = ky, whose solution is the familiar exponential y = e^kx. By exact analogy, the solution here is defined by the matrix exponential series:

R(θ) = exp(θJ) := I + θJ + (θJ)²/2! + (θJ)³/3! + ...

J is called the generator of SO(2): the entire infinite, curved group can be reconstructed by exponentiating a single fixed matrix scaled by a real number. Differentiating R(θ) directly gives J explicitly: dR/dθ = [[−sinθ, −cosθ], [cosθ, −sinθ]], and evaluating at θ = 0 gives

J = [ 0  -1 ]
    [ 1   0 ]

Now check what J does to itself: J² = J·J. Multiplying out row by row, J² = [[−1, 0], [0, −1]] = −I. This is the crucial fact: J behaves exactly like the imaginary unit i, since i² = −1. Substitute J² = −I into the exponential series and group even and odd powers of θ separately: the even-power terms only ever produce +I or −I with alternating signs, giving the Maclaurin series for cosine; the odd-power terms only ever produce +J or −J with alternating signs, giving the Maclaurin series for sine. So the series collapses to

exp(θJ) = I·cosθ + J·sinθ = [ cosθ  -sinθ ]  =  R(θ)
                                 [ sinθ   cosθ ]

which is exactly the formula for R(θ) written down at the start of this section — now derived, not assumed. This is precisely the structure of Euler's formula e^iθ = cosθ + i sinθ, and it is not a coincidence: SO(2) and the group of unit complex numbers under multiplication (usually written U(1)) are the same Lie group wearing two different notations, one using 2×2 real matrices and one using complex numbers.

The one-dimensional space of all multiples of J, written so(2) in lowercase, is called the Lie algebra of SO(2). In general, the Lie algebra of any Lie group is the tangent space to the group's manifold at its identity element — the "flat, linear approximation" of the curved group right at the point where nothing has happened yet. The diagram below shows this concretely: the vertical green line is the tangent space (the Lie algebra) touching the circle (the group SO(2)) at the identity, and the exponential map is what bends a straight-line motion along that tangent into the correct curved motion along the circle.

def exp_map(theta, terms=15):
    J = np.array([[0., -1.],
                  [1.,  0.]])
    total = np.eye(2)
    term = np.eye(2)                 # (theta*J)^0 / 0!
    for k in range(1, terms):
        term = term @ (theta * J) / k
        total += term
    return total

print(np.round(exp_map(np.pi / 3), 4))
print(np.round(R(np.pi / 3), 4))
# Both print [[ 0.5   -0.866]
#             [ 0.866  0.5  ]]  -- the truncated series matches the closed form

The general theorem, stated here without a full proof since it belongs to a later course, is that near the identity, every element of a connected Lie group can be reached by exponentiating some element of its Lie algebra. This is exactly why researchers usually work with Lie algebras rather than Lie groups directly wherever possible: the algebra is a flat vector space, where addition and scaling behave exactly the way they do in ordinary linear algebra, while the group itself is curved and harder to compute with directly.

SO(3): When Rotations Stop Commuting

Common misconception, named and corrected. Having just seen that SO(2) is commutative — R(α)R(β) = R(β)R(α) because ordinary addition of angles does not care about order — it is tempting to generalize: "continuous symmetry groups are always commutative (abelian)." This is false, and the standard counterexample is the group of rotations of ordinary 3D space, SO(3).

Try this physically with a book on your desk. Rotate it 90° about the vertical axis, then 90° about the axis pointing left-right. Note the final orientation of the cover. Now start over from the original position and do the same two rotations in the opposite order: left-right axis first, then vertical axis. The book ends up facing a different way. Order matters in three dimensions in a way it never did for rotations confined to a single plane.

This is not a fluke of physical fumbling — it is provable with the same rotation-matrix tools already built. Let R_x(t) and R_y(t) rotate by angle t about the x-axis and y-axis respectively:

import numpy as np

def Rx(t):
    c, s = np.cos(t), np.sin(t)
    return np.array([[1, 0,  0],
                      [0, c, -s],
                      [0, s,  c]])

def Ry(t):
    c, s = np.cos(t), np.sin(t)
    return np.array([[ c, 0, s],
                      [ 0, 1, 0],
                      [-s, 0, c]])

t = np.pi / 2
AB = Rx(t) @ Ry(t)
BA = Ry(t) @ Rx(t)
print(np.allclose(AB, BA))            # False

Working the 90° case out by hand confirms the code: R_x(90°) = [[1,0,0],[0,0,−1],[0,1,0]] and R_y(90°) = [[0,0,1],[0,1,0],[−1,0,0]]. Multiplying gives R_xR_y = [[0,0,1],[1,0,0],[0,1,0]], while R_yR_x = [[0,1,0],[0,0,−1],[−1,0,0]]. These are different matrices, so the two orders of rotation genuinely send the book to different final orientations. SO(3) is non-abelian.

The structural reason traces straight back to generators. SO(2) has one generator, J, and any matrix trivially commutes with itself, so nothing can ever fail to commute. SO(3), being a 3-dimensional manifold, has three independent generators — call them L_x, L_y, L_z, one for infinitesimal rotation about each axis — and these do not commute with each other: their commutator (defined as L_xL_y − L_yL_x) works out to L_z, not zero. This commutator structure, called the Lie algebra so(3), is the algebraic signature of "order matters," and it is the same mathematical structure that governs angular momentum in physics. Engineers designing spacecraft attitude-control systems know this non-commutativity by its practical name — it is one of the reasons naive sequential "roll, then pitch, then yaw" (Euler angle) control schemes are fragile and prone to gimbal lock, and why quaternion or exponential-map-based representations of orientation are preferred in ISRO satellite attitude control, robotics, and flight software.

Equivariance: Teaching Neural Networks About Symmetry

The payoff for all this structure is a precise definition that tells an architecture designer exactly what "respecting a symmetry" means. Let a group G act on both the input space X and the output space Y of a function f. Then f is G-equivariant if, for every g ∈ G and every x ∈ X,

f(g·x) = g·f(x)

— transforming the input first and then applying f gives the same result as applying f first and then transforming the output the corresponding way. f is G-invariant when the action on the output side is trivial, i.e. f(g·x) = f(x): the output does not change at all when the input is transformed.

Convolution is precisely a translation-equivariant operation. Let T_a denote shifting a signal by a. For any fixed kernel k, (T_ax) * k = T_a(x * k): convolving a shifted signal gives the same result, shifted, as shifting the convolved signal. This is why a single set of convolutional weights, reused at every spatial position, is enough to detect a feature anywhere in an image — the translation group's symmetry is built structurally into the layer, rather than something the network has to infer statistically from thousands of augmented examples.

def circular_conv(signal, kernel):
    n, k = len(signal), len(kernel)
    pad = k // 2
    out = np.zeros(n)
    for i in range(n):
        s = 0.0
        for j in range(k):
            s += kernel[j] * signal[(i - pad + j) % n]
        out[i] = s
    return out

x = np.array([0., 0., 1., 2., 0., 0., 0., 0.])
kernel = np.array([1., 0., -1.])          # edge-detecting kernel

shifted_x = np.roll(x, 2)                  # translate the input

out1 = np.roll(circular_conv(x, kernel), 2)      # convolve, then shift
out2 = circular_conv(shifted_x, kernel)           # shift, then convolve

print(np.allclose(out1, out2))              # True: translation-equivariance holds exactly

(A technical note for accuracy: on a finite image, convolution is exactly translation-equivariant only under circular or periodic boundary handling, as coded above; with plain zero-padding at fixed edges, equivariance holds in the interior but degrades slightly near the border, since information can shift off the edge. Also, most deep learning frameworks label this operation "convolution" but implement cross-correlation — the kernel is not flipped — which does not affect the equivariance argument at all.)

Once equivariance is stated this precisely, it becomes a design target rather than an accident. Group-equivariant CNNs (Cohen and Welling, 2016) replace the translation group with a larger one that also includes rotations — for instance the four 90° rotations, or the full continuous SO(2) — by convolving with several rotated copies of each kernel instead of one. The result is a network that is equivariant to rotation as well as translation, by construction, not by data augmentation. This has a measurable payoff in domains with a genuine physical rotation symmetry: galaxy morphology classification from telescope images (a spiral galaxy is still a spiral galaxy no matter which way "up" happens to be in the frame) and satellite or aerial imagery (a road, field, or building has no preferred orientation relative to the satellite's flight path) both show accuracy gains from rotation-equivariant architectures compared with an ordinary CNN of the same size trained only with rotation-augmented data.

Geometric Deep Learning: One Blueprint, Many Groups

The broader research program now called geometric deep learning (associated with work by Bronstein, Bruna, Cohen, and Veličković, among others) turns this into a general design blueprint: first identify the symmetry group G that the data and task genuinely possess, then build layers that are provably equivariant to G, rather than hoping a generic architecture learns the symmetry from enough labelled examples. Images and audio have the translation group. 3D point clouds and molecules have SO(3) or the full rigid-motion group SE(3) — rotations combined with translations. Graphs and sets have the permutation group S_n, which is finite and discrete, not a Lie group at all — this is exactly why graph neural networks use permutation-equivariant aggregation functions such as sum, mean, or max over a node's neighbours rather than exponential-map machinery: the right mathematical tool depends on whether the underlying symmetry group is continuous or discrete.

Protein structure prediction systems that must output a 3D shape use SE(3)-equivariant layers as standard practice, because a protein's predicted geometry has to rotate and translate correctly along with however the input coordinate frame happened to be chosen when the atomic coordinates were recorded — the biology has not changed just because the file used a different reference frame, so the model's output must not depend on that arbitrary choice either. This is a direct, real-world descendant of the same generator-and-exponential-map machinery derived by hand for SO(2) earlier in this chapter.

Where This Sits in Your Syllabus

Every prerequisite used above is already on the CBSE Class 11-12 syllabus even though Lie theory itself is not: the NCERT Class 12 Matrices chapter's orthogonal-matrix questions (proving A^TA = I forces det(A) = ±1) are literally asking you to verify membership in O(2) or O(3); the angle-addition formulas used to prove SO(2)'s closure are Class 11 Trigonometric Functions; and Euler's identity e^iθ = cosθ + i sinθ, met in the Complex Numbers chapter, turned out to be the same object as SO(2) in different notation. JEE Advanced and BITSAT regularly test exactly these orthogonal-matrix and rotation-composition properties, and Olympiad-style problems on the symmetry group of a regular polygon (like D₃ above) are the discrete warm-up for precisely the continuous case built in this chapter. None of this makes Lie groups "in syllabus," but it means the material above is a genuine head start into standard first-year linear algebra and geometry content in engineering and computer science programmes, built entirely from tools you already have.

Active Recall

1. Prove SO(2) is abelian directly from the angle-addition formulas, without writing out any matrices — show R(α)R(β) and R(β)R(α) both equal R(α+β).
2. Verify by direct matrix multiplication that J³ = −J and J&sup4; = I for the SO(2) generator J. Explain the parallel with i³ = −i and i⁴ = 1.
3. (JEE-style) If A is a 2×2 real orthogonal matrix (A^TA = I) with det(A) = 1, show that A must equal R(θ) for some θ. What goes wrong if instead det(A) = −1?
4. In one sentence, using the idea of generators, explain why SO(2) is abelian while SO(3) is not.
5. A function f takes an RGB image and returns only its average brightness, discarding all spatial and colour-channel structure. Is f invariant or equivariant with respect to the group that permutes the order of the three colour channels? Justify your answer using the definitions given above.
6. A plain fully connected (dense) layer, with an independent weight for every input-output pixel pair, is applied to an image. Explain why this layer cannot be translation-equivariant in general, unless a large number of its weights are forced to be tied together — and name what that tying is called once it has been imposed.

Summary

• A group is a set with a composition rule satisfying closure, associativity, an identity, and inverses; a finite group like the triangle's symmetry group D₃ has no notion of "closeness" between elements.
• A Lie group is a group whose elements also form a smooth manifold, with multiplication and inversion as smooth functions of the parameters. SO(2), the rotation group of the plane, is the simplest example, with its elements parametrized by an angle θ living on a circle.
• Differentiating a Lie group's multiplication rule at the identity produces its generator; for SO(2) this is the matrix J with J² = −I, and the whole group is recovered from J via the exponential map R(θ) = exp(θJ) = I cosθ + J sinθ, matching Euler's formula.
• The tangent space to a Lie group at its identity is called its Lie algebra; it turns curved-group computations into flat linear-algebra computations near the identity.
• SO(3), the rotation group of 3D space, is non-abelian: its three generators do not commute, which is the algebraic reason rotation order matters physically, and the practical reason engineers avoid naive Euler-angle sequencing in attitude-control systems.
• A function f is G-equivariant if f(g·x) = g·f(x); convolution is exactly translation-equivariant, which is why CNNs generalize across position without needing position-augmented data.
• Group-equivariant CNNs extend this to rotation and other continuous symmetries by construction; geometric deep learning generalizes the idea to any symmetry group a dataset genuinely has — SE(3) for molecules and proteins, permutations for graphs — choosing the right mathematical machinery (Lie theory or finite group theory) to match whether the symmetry is continuous or discrete.