Transfer Learning: Leveraging Pre-Trained Models

Priya, Rahul, and a Bird Neither of Them Has Seen Before

Priya has spent three years birdwatching with her grandfather every Sunday morning. Rahul has never really looked closely at a bird photograph in his life. One day, their biology teacher shows both of them the same unfamiliar photo — a Malabar Trogon, a bird neither has specifically studied before — and gives them twenty minutes to describe what they notice.

Rahul struggles. He has to figure out, from scratch, what counts as a beak versus a claw, how feathers differ from fur, what "typical bird proportions" even look like, and how shadows and lighting change the colour you perceive in a photo. Twenty minutes and one photo is nowhere near enough to build all of that visual understanding from zero.

Priya, on the other hand, is not starting from zero. She already has years of visual knowledge about beak shapes, feather textures, wing postures, and colour patterns, built from looking at hundreds of other birds. She isn't recalling this specific trogon from memory — she has genuinely never seen one — but she reuses everything she already knows about "birdness in general" and only has to figure out the small amount that is new and specific to this one photo. She finishes confidently in five minutes; Rahul is still stuck describing the beak.

That gap between Priya and Rahul is exactly the situation every real AI project faces when it needs to recognise something new from images, text, or sound. Transfer learning is the technique that turns a new AI model into "Priya" instead of "Rahul" — letting it reuse knowledge learned from a large, unrelated dataset instead of learning everything from a handful of new examples.

Why "Just Train From Scratch" Usually Fails

To understand why this matters, look at what training an image-recognition neural network from scratch actually costs. A well-known benchmark dataset called ImageNet contains roughly 1.28 million labelled training photographs spread across 1,000 categories — everyday objects like dogs, chairs, vehicles, and musical instruments. Training a convolutional neural network (CNN) to get good accuracy on this dataset typically takes many hours of computation on powerful graphics hardware, even before you touch your own project.

Now compare that to a typical school or early-startup AI project in India: a student building an app to sort ₹10, ₹20, ₹50, ₹100, and ₹500 currency notes for a visually-impaired user might realistically collect 200–400 photographs of each note using a phone camera — a few hundred images per class, not a few hundred thousand. If you initialise a CNN with random weights and try to train it directly on that small dataset, two things go wrong. First, there simply isn't enough data for the network to discover reliable visual patterns — it has too many adjustable numbers (parameters) and too few examples to pin them down correctly. Second, the network tends to overfit: instead of learning "what a ₹500 note generally looks like," it starts memorising quirks of your specific 300 training photos — a particular smudge, a particular lighting angle — and then fails badly on any new photo that doesn't share those quirks.

Transfer learning solves this by refusing to start from zero. It borrows the "Priya" advantage: begin from a network that has already learned general visual understanding from a large dataset like ImageNet, and only teach it the small, specific part that is new to your task.

What a Neural Network Actually Learns, Layer by Layer

To see why borrowed knowledge is reusable at all, you need to know what the different layers of a trained CNN are actually doing. A CNN processes an image through a stack of layers, and research on these networks has repeatedly shown a consistent pattern in what each part learns.

The earliest layers, right after the input image, learn to detect extremely simple, generic things: edges, colour gradients, and corners. An edge detector doesn't care whether it's looking at a currency note, a mango, or a cricket ball — an edge is an edge. The middle layers combine those edges into slightly more complex, still fairly generic patterns: textures, repeating shapes, simple curves. Only in the later layers does the network start combining these patterns into things that are specific to the original 1,000 ImageNet categories — parts like "a wheel," "an eye," or "a wing," and finally, in the very last layer, a decision about which of the 1,000 categories the whole image belongs to.

The key insight is this: the early and middle layers are not specialised for "dog versus cat." They are a generically useful visual vocabulary that is almost equally useful for recognising currency notes, X-rays, or handwritten Devanagari characters. That reusable vocabulary is what transfer learning actually transfers. Only the last layer or two — the part that maps "shapes and parts" onto a specific set of category names — is genuinely tied to the original task and needs to be replaced.

Defining Transfer Learning

Now the formal idea, built directly from what you just saw: transfer learning is the practice of taking a model whose weights were already learned on one task and dataset (the source task — here, classifying 1,000 ImageNet categories) and reusing part of that learned knowledge as the starting point for a different task and dataset (the target task — here, classifying currency notes), instead of initialising every weight randomly and learning everything from nothing.

The model that supplies the starting knowledge is called a pre-trained model. Well-known pre-trained image models include MobileNetV2, ResNet, and VGG — all trained once, by their original creators, on large datasets like ImageNet, and then published so that anyone can reuse them. You are not expected to retrain them yourself; you download the already-learned weights and build on top.

Two Concrete Strategies

There are two standard ways to actually use a pre-trained model, and the difference between them comes down to one question: how much of the borrowed knowledge do you let your training process change?

Feature extraction is the more cautious strategy. You keep every pre-trained layer exactly as it is — this is called freezing a layer, meaning its weights are marked as not updatable during training — and you attach a small, brand-new set of layers (often just one or two Dense layers) on top, ending in an output layer sized for your specific number of classes. During training, only the new layers' weights change; the frozen trunk just acts as a fixed feature detector. This is fast, needs relatively little data, and is the safer default when your target dataset is small.

Fine-tuning goes a step further. After the new head has already learned something reasonable, you unfreeze some of the later pre-trained layers too, and continue training the whole combination together — but with a much smaller learning rate than you'd normally use. The small learning rate matters enormously: it lets the late layers nudge gently toward your specific task without being yanked far from the well-tuned starting point they arrived with. Fine-tuning can push accuracy higher than pure feature extraction, but it needs somewhat more target data, because you are now asking more parameters to adapt correctly.

Worked Example: Recognising Currency Notes with a Pre-Trained Network

Let's make this concrete with code for the assistive-app scenario: classifying a photo into one of five Indian currency denominations — ₹10, ₹20, ₹50, ₹100, ₹500 — using MobileNetV2 as the pre-trained trunk.

from tensorflow.keras.applications import MobileNetV2
from tensorflow.keras.layers import GlobalAveragePooling2D, Dense
from tensorflow.keras.models import Model

# Load MobileNetV2, pre-trained on ImageNet, WITHOUT its original
# 1000-class classifier head (include_top=False)
base_model = MobileNetV2(weights='imagenet',
                          include_top=False,
                          input_shape=(224, 224, 3))

# Freeze every layer in the trunk: their weights will not change
for layer in base_model.layers:
    layer.trainable = False

# Attach a brand-new head sized for our 5 currency classes
x = base_model.output                       # shape: (None, 7, 7, 1280)
x = GlobalAveragePooling2D()(x)              # shape: (None, 1280)
x = Dense(128, activation='relu')(x)         # shape: (None, 128)
output = Dense(5, activation='softmax')(x)   # shape: (None, 5)

model = Model(inputs=base_model.input, outputs=output)
model.compile(optimizer='adam',
              loss='categorical_crossentropy',
              metrics=['accuracy'])

Trace through what each line produces. MobileNetV2 with include_top=False and a 224×224 input ends its trunk with a 7×7 grid of 1,280 channels — that shape comes from MobileNetV2's own downsampling design, not from anything we chose. GlobalAveragePooling2D averages each of those 1,280 channels across the 7×7 grid, collapsing the shape down to a single vector of 1,280 numbers per image. That vector is then passed through our new, randomly-initialised Dense(128) and Dense(5) layers, ending in five probabilities (thanks to softmax) that sum to 1 — one probability per currency class.

Now count exactly what gets trained. A Dense layer's parameter count is (inputs × outputs) + outputs, one weight per input-output connection plus one bias per output neuron. For Dense(128) taking the 1,280-length vector: 1,280 × 128 + 128 = 163,840 + 128 = 163,968 parameters. For Dense(5) taking that 128-length vector: 128 × 5 + 5 = 640 + 5 = 645 parameters. Add them: 163,968 + 645 = 164,613 trainable parameters in total.

Compare that to the frozen trunk, which contains several million parameters that were learned once from those 1.28 million ImageNet photographs and are now locked. Your 300-photos-per-class dataset never has to teach the network what an edge or a texture is — it only has to teach those 164,613 numbers how to map MobileNetV2's general-purpose visual vocabulary onto five specific currency classes. That is a dramatically more learnable problem than training millions of parameters from a few hundred images, and it is exactly why the feature-extraction approach can produce a usably accurate currency classifier from a dataset that would badly overfit a from-scratch CNN. (In Keras, calling model.summary() will print "Trainable params" and "Non-trainable params" as two separate totals, letting you verify this split directly.)

Fine-Tuning in Practice: Unfreeze Carefully

Suppose feature extraction gets you to a working model, but you want to squeeze out more accuracy by letting the trunk adapt slightly to what currency notes specifically look like — their fine print, security threads, and colour patterns, which are more specific than "generic textures." You would unfreeze a handful of the trunk's later layers and continue training with a small learning rate:

from tensorflow.keras.optimizers import Adam

# Unfreeze only the last 20 layers of the trunk -
# the ones closest to being task-specific
for layer in base_model.layers[-20:]:
    layer.trainable = True

# Recompile with a much smaller learning rate than before
model.compile(optimizer=Adam(learning_rate=1e-5),
              loss='categorical_crossentropy',
              metrics=['accuracy'])

The learning rate here, 0.00001, is typically 100 to 1,000 times smaller than the default learning rate you'd use for training from scratch. Here's why that matters: a large learning rate makes big updates to weights on every training step. If you applied a large learning rate to layers that already encode good, general-purpose knowledge, you would overwrite that knowledge almost immediately with large, noisy updates driven by your small dataset — a problem often called catastrophic forgetting. It is the equivalent of forcing Priya to "relearn" bird identification aggressively from one blurry photo, in a way that damages the three years of solid intuition she started with. A tiny learning rate lets the unfrozen layers make small, careful adjustments instead, preserving most of what they already knew while adapting to your specific data.

When Transfer Learning Works Well — and When It Struggles

Two factors decide how well transfer learning will work for a given project: how much target data you have, and how visually similar your target domain is to the source domain (natural, everyday photographs, in the case of ImageNet).

• Small target dataset, similar domain (our currency-note example — real photographs, just like ImageNet's): feature extraction alone usually works well. This is the safest and most common combination for student and early-startup projects.
• Large target dataset, similar domain: you can afford to fine-tune more of the network, often pushing accuracy noticeably higher than feature extraction alone.
• Small target dataset, very different domain — for example, medical X-rays, which have very different textures, contrast, and structure than everyday photographs: transfer learning still helps, because edges and simple shapes are still somewhat universal, but the benefit is smaller. Practitioners often pull features from an earlier, more generic layer rather than the very last one, and lean more heavily on data augmentation to stretch the small dataset further.
• Large target dataset, very different domain: you have enough data to fine-tune extensively, or in some cases even train major parts of the network from scratch, since data scarcity is no longer the limiting factor.

This is a genuine engineering trade-off, not a rule to memorise blindly — the right choice always depends on how much labelled data you actually have and how far your images are, visually, from natural everyday photographs.

A Common Misconception

A frequent misunderstanding is this: "transfer learning means the pre-trained model already knows my new classes." It does not. ImageNet's 1,000 categories are everyday objects and animals — the base MobileNetV2 model has never once seen a labelled ₹500 note, because that category simply does not exist in its original training data. What transfers is the general visual vocabulary — the ability to detect edges, textures, and shapes — not the final answer for your specific categories.

Skipping the step of attaching and training a new classifier head, on the theory that "the pre-trained model already understands images," leaves you with a network that can describe an image richly in terms of edges and shapes but genuinely has no mechanism for outputting "₹100" or "₹500" — those output neurons don't even exist until you add and train them. You must always supply your own labelled examples of the actual target classes and train at least a new head on them; transfer learning reduces how much data and training you need, but it never eliminates the need for target-task data entirely.

Where This Technique Actually Shows Up

Transfer learning is the default starting point, not the exception, in most practical computer-vision work today — squarely inside the Computer Vision domain you encounter in the AI project-cycle portion of your CBSE syllabus. Crop-disease detection apps built for Indian farmers typically start from a network pre-trained on general photographs and fine-tune it on a smaller set of labelled leaf-disease images, because collecting hundreds of thousands of diseased-leaf photographs for every crop is not realistic. Satellite land-cover classification work, of the kind used with ISRO's Bhuvan imagery, often begins from networks pre-trained on natural photographs and adapts them to multi-band satellite data, since building a satellite-scale labelled dataset from zero is far more expensive than adapting an existing one. Optical character recognition for Indic scripts such as Devanagari commonly reuses low-level stroke-and-edge features learned on other handwriting or printed-text datasets, fine-tuning only the higher layers on script-specific samples. In every one of these cases, the underlying reason is the same one you worked through above: collecting a million labelled examples for a narrow, specific problem is rarely realistic, but reusing a general visual vocabulary learned once, elsewhere, usually is.

Practice: Test What You've Learned

1. In your own words, map each part of the Priya-and-Rahul story onto the vocabulary of this chapter: what plays the role of the "source task," the "target task," and the "generic features"?
2. Which layers of a CNN would you expect to transfer well to a completely unrelated visual task — the early layers or the very last layer? Justify your answer using what each part actually detects.
3. Suppose a base model's last layer produces 512 channels (instead of 1,280), and you attach Dense(64, activation='relu') followed by Dense(10, activation='softmax') after a GlobalAveragePooling2D. Calculate the exact number of trainable parameters in just these two new layers, showing your working. (Answer: Dense(64) has 512×64 + 64 = 32,832 parameters; Dense(10) has 64×10 + 10 = 650 parameters; total = 33,482.)
4. CBSE-style short answer: Explain the difference between feature extraction and fine-tuning as transfer-learning strategies, and state one situation in which you would prefer each.
5. A classmate wants to fine-tune every layer of a pre-trained ImageNet model using only 40 labelled X-ray images and a learning rate of 0.01. Identify two things likely to go wrong, and suggest a better approach based on the decision framework in this chapter.

Summary

Transfer learning reuses a model already trained on a large source dataset (commonly ImageNet's ~1.28 million images) as the starting point for a new, usually much smaller, target task — instead of learning every weight from random initialisation. It works because convolutional networks learn in a hierarchy: early and middle layers detect generic, broadly reusable patterns like edges and textures, while only the last layers are specific to the original categories. This lets you freeze the reusable trunk and train only a small new classifier head (feature extraction), or additionally unfreeze some later layers and continue training everything with a small learning rate to adapt further (fine-tuning), choosing between the two based on how much target data you have and how visually similar your target domain is to the source domain. Crucially, a pre-trained model never already knows your specific target classes — you must always train a new head on your own labelled data; what transfer learning saves you is the enormous cost of relearning basic visual understanding from scratch.

Think About It

Think about this: How would you explain transfer learning: leveraging pre-trained models to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.