Convolutional Neural Networks

A fully-connected layer on a 224×224 RGB image has 224·224·3 = 150,528 inputs. A single hidden layer of 1024 units is 154 million parameters. You can't train it, and you wouldn't want to — it ignores the most important fact about images: nearby pixels are correlated, and the same feature (an edge, a corner) can appear anywhere in the frame.

CNNs are the architecture that exploits both. A small filter slides across the image, computing the same operation at every location — parameter sharing across space, translation equivariance for free. From LeNet's 1998 demo on MNIST to ConvNeXt rivalling vision transformers in 2022, the CNN has been the backbone of modern computer vision. This lesson is the engineer's tour: the maths, the architecture family tree, and the modern tricks that matter in production.

1. Convolution — The Maths, Shape by Shape

A 2D convolution takes an input feature map $X \in \mathbb{R}^{H \times W \times C_{\text{in}}}$ and a bank of $C_{\text{out}}$ filters, each of shape $k \times k \times C_{\text{in}}$. For each output channel $c$ and spatial position $(i, j)$:

$$
Y_{i,j,c} = \sum_{u=0}^{k-1} \sum_{v=0}^{k-1} \sum_{d=0}^{C_{\text{in}}-1} X_{i+u,\, j+v,\, d} \cdot K^{(c)}_{u,v,d} + b_c
$$

Concretely: every output channel is a learned linear combination of all input channels, computed by a $k \times k$ window sliding across the spatial dimensions.

Output shape formula

Given input spatial size $H_{\text{in}}$, kernel size $k$, padding $p$, stride $s$, dilation $d$:

$$
H_{\text{out}} = \left\lfloor \frac{H_{\text{in}} + 2p - d(k-1) - 1}{s} \right\rfloor + 1
$$

For the common case of $d = 1$:

$$
H_{\text{out}} = \left\lfloor \frac{H_{\text{in}} - k + 2p}{s} \right\rfloor + 1
$$

Parameter count

A Conv2D layer with $C_{\text{in}}$ input channels, $C_{\text{out}}$ output channels, kernel size $k$:

$$
\text{params} = (k \cdot k \cdot C_{\text{in}} + 1) \cdot C_{\text{out}}
$$

That +1 is the bias per output channel. Independent of spatial size — the same filter scans the whole image. A $3\times3$ conv from 64 to 128 channels is $(9 \cdot 64 + 1) \cdot 128 = 73,856$ parameters, no matter whether your input is $32\times32$ or $1024\times1024$.

2. Receptive Field — How Much Each Output Sees

A single $3\times3$ conv neuron sees a $3\times3$ patch of its input. Stack two of them and the second neuron sees a $5\times5$ patch of the original input. Stack three and it's $7\times7$. The receptive field grows linearly with depth for stride-1 stacks:

$$
\text{RF}_{\ell} = \text{RF}_{\ell-1} + (k_\ell - 1) \prod_{i < \ell} s_i
$$

When you add stride-2 layers, the receptive field grows much faster — each stride-2 layer doubles the effective step size of every subsequent layer.

Why this matters: a classification head at the top of the network needs to see the whole object. Three $3\times3$ convs see a $7\times7$ window; four with a stride-2 in the middle see $\sim15\times15$. To classify a 224×224 image, you need enough depth (or stride, or pooling) to grow the receptive field to roughly the size of the whole image — that's why classification CNNs are deep.

Two stacked $3\times3$ convs have the same receptive field as one $5\times5$ conv but use $2 \cdot 9 = 18$ params (per channel pair) instead of $25$, with two non-linearities instead of one. Decomposing big kernels into stacks of small ones is the central trick of VGG and everything since.

3. The 1×1 Convolution

A $1\times1$ conv is not "no convolution" — it's a per-pixel linear projection across channels. It doesn't change the spatial dimensions, but it changes the channel count.

layers.Conv2D(filters=64, kernel_size=1)(x)    # (H, W, 256) -> (H, W, 64)

# Lightweight mock for objects whose attributes/methods aren't critical
class _AutoMock:
    def __init__(self, name='mock'): self._name = name
    def __getattr__(self, k): return _AutoMock(self._name + '.' + k)
    def __call__(self, *a, **kw):
        print('-> ' + self._name + '() called')
        return _AutoMock(self._name + '()')
    def __repr__(self): return '<mock ' + self._name + '>'
    def __str__(self): return '<mock ' + self._name + '>'
    def __bool__(self): return True
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __getitem__(self, k): return _AutoMock(self._name + '[...]')
    def __setitem__(self, k, v): pass
    def __enter__(self): return self
    def __exit__(self, *a): return False
    async def __aenter__(self): return self
    async def __aexit__(self, *a): return False
    def __add__(self, o): return self
    def __radd__(self, o): return self
    def __sub__(self, o): return self
    def __mul__(self, o): return self
    def __rmul__(self, o): return self
    def __truediv__(self, o): return self
    def __eq__(self, o): return isinstance(o, _AutoMock)
    def __hash__(self): return hash(self._name)
    def __lt__(self, o): return True
    def __le__(self, o): return True
    def __gt__(self, o): return False
    def __ge__(self, o): return False
    def __mro_entries__(self, bases): return (object,)

x = _AutoMock('x')
layers = _AutoMock('layers')

Used for:

• Channel reduction before an expensive conv ("bottleneck"). Inception and ResNet do this constantly: drop channels with a $1\times1$, do the big $3\times3$ on the reduced channels, expand back with another $1\times1$.
• Channel mixing without spatial change — equivalent to a fully-connected layer applied independently at each spatial position.
• Network-in-Network: stack $1\times1$ convs to build per-pixel MLPs.

The $1\times1$ conv is the unsung workhorse — almost every modern architecture has them sprinkled throughout for cheap channel manipulation.

4. Pooling — And Why Modern Networks Use Less of It

Pooling layers downsample the spatial dimensions with a fixed (non-learned) operation:

• Max pooling: take the max over a $k \times k$ window. The classic; emphasises strong activations, throws away the rest.
• Average pooling: take the mean. Smoother, less aggressive.
• Global average pooling (GAP): take the mean over the entire spatial map → one scalar per channel. Replaces the Flatten + Dense(N) + Dense(num_classes) head with a single dense layer on top of GAP. This is the modern practice — fewer parameters, much less overfitting, and built into ResNet, EfficientNet, and ConvNeXt.

# Classic head — millions of params, prone to overfit
x = layers.Flatten()(feature_map)
x = layers.Dense(4096, activation="relu")(x)         # 4096 * H * W * C params
x = layers.Dense(num_classes, activation="softmax")(x)

# Modern head — GAP, then one Dense
x = layers.GlobalAveragePooling2D()(feature_map)     # (B, C)
x = layers.Dense(num_classes, activation="softmax")(x)   # C * num_classes params

# Lightweight mock for objects whose attributes/methods aren't critical
class _AutoMock:
    def __init__(self, name='mock'): self._name = name
    def __getattr__(self, k): return _AutoMock(self._name + '.' + k)
    def __call__(self, *a, **kw):
        print('-> ' + self._name + '() called')
        return _AutoMock(self._name + '()')
    def __repr__(self): return '<mock ' + self._name + '>'
    def __str__(self): return '<mock ' + self._name + '>'
    def __bool__(self): return True
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __getitem__(self, k): return _AutoMock(self._name + '[...]')
    def __setitem__(self, k, v): pass
    def __enter__(self): return self
    def __exit__(self, *a): return False
    async def __aenter__(self): return self
    async def __aexit__(self, *a): return False
    def __add__(self, o): return self
    def __radd__(self, o): return self
    def __sub__(self, o): return self
    def __mul__(self, o): return self
    def __rmul__(self, o): return self
    def __truediv__(self, o): return self
    def __eq__(self, o): return isinstance(o, _AutoMock)
    def __hash__(self): return hash(self._name)
    def __lt__(self, o): return True
    def __le__(self, o): return True
    def __gt__(self, o): return False
    def __ge__(self, o): return False
    def __mro_entries__(self, bases): return (object,)

feature_map = _AutoMock('feature_map')
num_classes = _AutoMock('num_classes')
layers = _AutoMock('layers')

The GAP head also makes the network input-size agnostic — the dense layer's input size depends only on the channel count, not the feature map's spatial size.

Modern architectures often skip explicit pooling altogether, using stride-2 convolutions for downsampling — same effect (halve spatial dims) but learnable.

5. The Architecture Family Tree

The history of CNN architectures is a sequence of "why didn't I think of that" ideas.

LeNet-5 (1998)

LeCun's hand-digit classifier. Two conv layers, two pooling, two FC. About 60k parameters. Demonstrated that conv + pool + FC works. Trained on MNIST in the era before GPUs.

AlexNet (2012)

The shot heard round the world — won ImageNet 2012 by a huge margin, kicked off the deep learning boom. 5 conv layers + 3 FC, ~60M params, ReLU activations (first time), dropout, GPU training. Used $11\times11$ and $5\times5$ kernels — replaced in everything since.

VGG (2014)

Simonyan & Zisserman: "what if we just stack a lot of $3\times3$ convs?" 16 or 19 layers, all $3\times3$, all stride 1, with $2\times2$ max-pool between blocks. Roughly 138M params. The architecture is simple enough that VGG-16 is still used as a feature extractor for things like perceptual loss in image generation. The big lesson: depth + small kernels > shallow + big kernels.

Inception / GoogLeNet (2014)

Szegedy et al.: parallel branches of different kernel sizes ($1\times1$, $3\times3$, $5\times5$, pool) concatenated together — the "Inception module". Captures features at multiple scales in one layer. Used $1\times1$ convs aggressively for dimension reduction. ~6M params despite 22 layers.

ResNet (2015)

He et al.: skip connections, $y = F(x) + x$. Trained 152-layer (and even 1000-layer) networks. The first architecture where adding more layers actually helped. Every modern architecture borrows the residual idea — transformers, vision transformers, U-Nets, ConvNeXt, all of them. ResNet is the most influential CV architecture of the decade.

DenseNet (2016)

Huang et al.: every layer takes inputs from all previous layers in the block, concatenated. Massive feature reuse, very parameter-efficient. Less common in production than ResNet, but the idea (dense connectivity) shows up in U-Net++ and other segmentation models.

EfficientNet (2019)

Tan & Le: "compound scaling" — instead of separately scaling depth, width, or input resolution, scale all three together using a single coefficient. EfficientNet-B0 through B7 form a Pareto frontier of accuracy vs FLOPs. Used Swish activations and depthwise-separable convs.

ConvNeXt (2022)

Liu et al.: "what if we apply the design choices that made ViT (Vision Transformer) win, back to CNNs?" Larger kernels ($7\times7$), LayerNorm instead of BN, GELU instead of ReLU, inverted bottleneck, no max pool. Result: a pure CNN that matches Swin Transformer on ImageNet. Vindicates the CNN paradigm in the transformer era.

6. A Residual Block in Code

The most useful architecture pattern to internalise. Two convs with batchnorm and ReLU, plus a skip connection:

from tensorflow.keras import layers, Input, Model

def residual_block(x, filters, stride=1):
    shortcut = x

    # 3x3 conv -> BN -> ReLU
    y = layers.Conv2D(filters, 3, strides=stride, padding="same", use_bias=False)(x)
    y = layers.BatchNormalization()(y)
    y = layers.ReLU()(y)

    # 3x3 conv -> BN  (no activation yet)
    y = layers.Conv2D(filters, 3, strides=1, padding="same", use_bias=False)(y)
    y = layers.BatchNormalization()(y)

    # If the shortcut's shape doesn't match (different filters or stride),
    # project it with a 1x1 conv. Otherwise just add.
    if stride != 1 or shortcut.shape[-1] != filters:
        shortcut = layers.Conv2D(filters, 1, strides=stride, use_bias=False)(shortcut)
        shortcut = layers.BatchNormalization()(shortcut)

    y = layers.Add()([y, shortcut])
    y = layers.ReLU()(y)         # activation AFTER the addition
    return y

Key design points:

• use_bias=False on conv before BN — BN has its own learnable shift, the conv bias is redundant.
• ReLU after the add, not before — this is the "post-activation" ResNet design. The 2016 "pre-activation" variant (BN → ReLU → Conv) trains a bit better at very high depth but is otherwise equivalent.
• Shortcut projection when shapes mismatch — required when stride != 1 (you've downsampled) or when channel count changes.

You stack these blocks. ResNet-18 is 8 of them, ResNet-50 uses a deeper bottleneck variant (1×1 → 3×3 → 1×1) for parameter efficiency.

7. Depthwise-Separable Convolutions — The MobileNet Trick

A regular $3\times3$ conv with $C_{\text{in}} = C_{\text{out}} = C$ costs $9 C^2$ params and $9 C^2 H W$ multiply-adds. A depthwise-separable conv factors this into two cheaper steps:

1. Depthwise conv: a separate $3\times3$ kernel per input channel, no mixing across channels. Cost: $9 C$ params.
2. Pointwise conv ($1\times1$): mix channels. Cost: $C^2$ params.

Total: $9C + C^2$ vs $9 C^2$. For $C = 256$, that's $65{,}792$ vs $589{,}824$ — about 9× cheaper. The accuracy hit is small. Used in MobileNet, EfficientNet, Xception.

# Regular conv
x = layers.Conv2D(256, 3, padding="same")(x)

# Depthwise-separable equivalent
x = layers.DepthwiseConv2D(3, padding="same")(x)    # depthwise step
x = layers.Conv2D(256, 1)(x)                        # pointwise step

# Lightweight mock for objects whose attributes/methods aren't critical
class _AutoMock:
    def __init__(self, name='mock'): self._name = name
    def __getattr__(self, k): return _AutoMock(self._name + '.' + k)
    def __call__(self, *a, **kw):
        print('-> ' + self._name + '() called')
        return _AutoMock(self._name + '()')
    def __repr__(self): return '<mock ' + self._name + '>'
    def __str__(self): return '<mock ' + self._name + '>'
    def __bool__(self): return True
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __getitem__(self, k): return _AutoMock(self._name + '[...]')
    def __setitem__(self, k, v): pass
    def __enter__(self): return self
    def __exit__(self, *a): return False
    async def __aenter__(self): return self
    async def __aexit__(self, *a): return False
    def __add__(self, o): return self
    def __radd__(self, o): return self
    def __sub__(self, o): return self
    def __mul__(self, o): return self
    def __rmul__(self, o): return self
    def __truediv__(self, o): return self
    def __eq__(self, o): return isinstance(o, _AutoMock)
    def __hash__(self): return hash(self._name)
    def __lt__(self, o): return True
    def __le__(self, o): return True
    def __gt__(self, o): return False
    def __ge__(self, o): return False
    def __mro_entries__(self, bases): return (object,)

layers = _AutoMock('layers')

When you want a fast mobile model, depthwise-separable is the first optimisation.

8. Dilated (Atrous) Convolutions

A dilated conv with dilation $d$ inserts $d-1$ "holes" between weights. A $3\times3$ kernel with $d = 2$ has the same parameter count as a regular $3\times3$ but covers a $5\times5$ receptive field. Stacked dilated convs grow receptive field exponentially.

layers.Conv2D(64, 3, dilation_rate=2, padding="same")(x)

# Lightweight mock for objects whose attributes/methods aren't critical
class _AutoMock:
    def __init__(self, name='mock'): self._name = name
    def __getattr__(self, k): return _AutoMock(self._name + '.' + k)
    def __call__(self, *a, **kw):
        print('-> ' + self._name + '() called')
        return _AutoMock(self._name + '()')
    def __repr__(self): return '<mock ' + self._name + '>'
    def __str__(self): return '<mock ' + self._name + '>'
    def __bool__(self): return True
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __getitem__(self, k): return _AutoMock(self._name + '[...]')
    def __setitem__(self, k, v): pass
    def __enter__(self): return self
    def __exit__(self, *a): return False
    async def __aenter__(self): return self
    async def __aexit__(self, *a): return False
    def __add__(self, o): return self
    def __radd__(self, o): return self
    def __sub__(self, o): return self
    def __mul__(self, o): return self
    def __rmul__(self, o): return self
    def __truediv__(self, o): return self
    def __eq__(self, o): return isinstance(o, _AutoMock)
    def __hash__(self): return hash(self._name)
    def __lt__(self, o): return True
    def __le__(self, o): return True
    def __gt__(self, o): return False
    def __ge__(self, o): return False
    def __mro_entries__(self, bases): return (object,)

x = _AutoMock('x')
layers = _AutoMock('layers')

Used in segmentation (DeepLab) and audio (WaveNet): you want a large receptive field without downsampling the spatial dimensions. Segmentation needs per-pixel output, so you can't aggressively pool away spatial detail; dilation grows context without sacrificing resolution.

9. Beyond Classification

CNNs aren't just for "what's in this image" — they're the backbone of every vision task.

Object detection — "what's there and where"

• R-CNN family (Girshick, 2014 → Faster R-CNN, 2015): two-stage — propose regions, then classify each. Accurate, slower.
• YOLO (Redmon, 2016 → YOLOv8+, 2023): single-stage — predict boxes and classes in one forward pass. Fast, real-time.
• SSD (Liu, 2016): single-shot multi-box detector. Between YOLO and Faster R-CNN.

All use a CNN backbone (ResNet, EfficientNet, ConvNeXt) followed by a detection head.

Segmentation — "label every pixel"

• U-Net (Ronneberger, 2015): encoder-decoder with skip connections from encoder to decoder. The dominant architecture for medical imaging and the backbone of every diffusion model.
• Mask R-CNN (He, 2017): adds a segmentation head to Faster R-CNN for instance segmentation (separate masks per object).
• DeepLab (Chen, 2018): atrous convs for dense semantic segmentation.

Image generation

• GANs — generator + discriminator adversarial training. See GANs.
• Diffusion models — iterative denoising. The current SOTA for image generation. U-Net is the standard denoiser backbone, though transformers (DiT) are taking over.

10. Data Augmentation — Free Generalisation

Most of the gap between an overfit model and a well-generalising one comes from data augmentation. Keras has GPU-runnable augmentation layers — apply them in-graph so they don't bottleneck the CPU pipeline:

augment = keras.Sequential([
    layers.RandomFlip("horizontal"),
    layers.RandomRotation(0.05),
    layers.RandomZoom(0.1),
    layers.RandomContrast(0.1),
    layers.RandomBrightness(0.1),
])

# Apply only during training
inputs = keras.Input(shape=(224, 224, 3))
x = augment(inputs)                       # in-graph, GPU-accelerated
x = backbone(x)
outputs = head(x)
model = keras.Model(inputs, outputs)

# Lightweight mock for objects whose attributes/methods aren't critical
class _AutoMock:
    def __init__(self, name='mock'): self._name = name
    def __getattr__(self, k): return _AutoMock(self._name + '.' + k)
    def __call__(self, *a, **kw):
        print('-> ' + self._name + '() called')
        return _AutoMock(self._name + '()')
    def __repr__(self): return '<mock ' + self._name + '>'
    def __str__(self): return '<mock ' + self._name + '>'
    def __bool__(self): return True
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __getitem__(self, k): return _AutoMock(self._name + '[...]')
    def __setitem__(self, k, v): pass
    def __enter__(self): return self
    def __exit__(self, *a): return False
    async def __aenter__(self): return self
    async def __aexit__(self, *a): return False
    def __add__(self, o): return self
    def __radd__(self, o): return self
    def __sub__(self, o): return self
    def __mul__(self, o): return self
    def __rmul__(self, o): return self
    def __truediv__(self, o): return self
    def __eq__(self, o): return isinstance(o, _AutoMock)
    def __hash__(self): return hash(self._name)
    def __lt__(self, o): return True
    def __le__(self, o): return True
    def __gt__(self, o): return False
    def __ge__(self, o): return False
    def __mro_entries__(self, bases): return (object,)

def backbone(*_a, **_kw):
    print('-> backbone() called')
    return _AutoMock('backbone()')
def head(*_a, **_kw):
    print('-> head() called')
    return _AutoMock('head()')
keras = _AutoMock('keras')
layers = _AutoMock('layers')

These layers automatically no-op at inference time (they only augment during training=True).

Heavier augmentations worth knowing:

• Mixup: linearly blend two images and their one-hot labels. Strong regulariser.
• CutMix: paste a random patch of one image onto another; blend labels by the patch area.
• RandAugment / AutoAugment: searchable augmentation policies that beat hand-tuned ones.

For a small dataset (<10k images), augmentation is often the difference between 60% and 90% test accuracy. Don't skip it.

11. Inference Optimisations

Training in fp32 and shipping in fp32 is leaving performance on the table. Three standard tricks:

Quantisation

Map fp32 weights and activations to int8 (or even int4). 4× smaller, 2-4× faster inference, modest accuracy hit (often <1% on a well-trained model).

• Post-training quantisation: quantise after training using a small calibration set. Quick, no retraining.
• Quantisation-aware training (QAT): simulate quantisation noise during training so the model adapts. Smaller accuracy hit.

TFLite handles both with a few lines of converter config.

Pruning

Set small-magnitude weights to zero and store the sparse model. With structured pruning (entire channels), the model is genuinely faster. With unstructured pruning, you need sparse-matmul hardware to see speedup.

Knowledge distillation

Train a small "student" network to mimic a large "teacher" network's outputs. The student often beats training from scratch — soft targets carry information that hard labels don't. DistilBERT (~40% smaller, 60% faster, 97% of BERT's accuracy) is the textbook example.

These three compose. A distilled-then-quantised-then-pruned ResNet can be 20× smaller and 10× faster than the original with a few percent accuracy hit — the difference between a model that runs on your phone and one that needs a GPU.

Common Mistakes

1. BatchNorm momentum issue when fine-tuning

When you fine-tune a pretrained model with a small dataset and small batch, BN's running stats can drift catastrophically. Either freeze the BN layers (layer.trainable = False — but read the Keras docs; the semantics are subtle) or switch to GroupNorm. Many "fine-tuning failures" are actually BN failures in disguise.

2. Very deep without residuals

A 30-layer plain CNN won't train. Without skip connections, gradients vanish through the depth. You either add residual connections, drop depth, or use a normalisation scheme designed for the depth (transformer-style pre-norm). Plain stacks past ~15 layers are a historical artefact.

3. No augmentation on a small dataset

If your training accuracy is 99% and validation is 70%, the cure is almost never a bigger model. It's augmentation, dropout, weight decay, and early stopping. Augmentation is the cheapest of these — turn it on first.

4. Wrong padding mode for downsampling

padding="valid" with stride=2 and kernel_size=3 on a 224×224 input gives 111×111 (not 112×112), because $(224 - 3)/2 + 1 = 111.5 \to 111$. The lost half-pixel propagates. Use "same" padding when you want predictable halving.

5. Mixing channels-first and channels-last formats

Keras defaults to channels_last (NHWC). Some pretrained models (especially imported from PyTorch) ship in channels_first (NCHW). Mixing them silently misaligns the channel dimension and gives garbage outputs. Always check data_format when loading external weights.

6. Forgetting to normalise inputs to match the pretrained model

ImageNet pretrained models expect inputs normalised with the ImageNet mean/std (or the model's specific preprocessing). Feeding raw [0, 1] images to a model that expects mean-subtracted inputs gives terrible accuracy — Keras's preprocess_input function for each model family does the right thing.

🎯 Your Turn — Build a Residual Block as a Custom Keras Layer

Implement a ResidualBlock subclass of keras.layers.Layer. The block should:

• Accept filters and stride as constructor arguments.
• Apply: Conv2D(3×3) → BN → ReLU → Conv2D(3×3) → BN, add the skip connection, then ReLU.
• Project the skip with a 1×1 conv + BN when stride != 1 or when the channel count of the input doesn't match filters.
• Use use_bias=False on convs followed by BN.
• Use "same" padding so spatial size is preserved (except when stride > 1, which halves it).
• Override get_config for serialisation.

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers

class ResidualBlock(layers.Layer):
    def __init__(self, filters, stride=1, **kwargs):
        super().__init__(**kwargs)
        # TODO 1: store filters and stride
        # TODO 2: create conv1 (3x3 stride=stride), bn1, conv2 (3x3 stride=1), bn2
        # TODO 3: prepare placeholders for the projection conv + bn (created in build)
        ...

    def build(self, input_shape):
        # TODO 4: if input channels != filters OR stride != 1, build a 1x1 projection
        #         with matching stride and bn; otherwise mark projection as None
        ...

    def call(self, x, training=False):
        # TODO 5: forward pass — conv1 -> bn1 -> relu -> conv2 -> bn2
        #         project shortcut if needed, add, relu, return
        ...

    def get_config(self):
        return {**super().get_config(), "filters": self.filters, "stride": self.stride}


# Sanity check
inp = keras.Input(shape=(32, 32, 64))
x   = ResidualBlock(filters=128, stride=2)(inp)   # downsample + channel change
x   = ResidualBlock(filters=128, stride=1)(x)     # same shape
model = keras.Model(inp, x)
model.summary()                                   # expect (None, 16, 16, 128)

# Lightweight mock for objects whose attributes/methods aren't critical
class _AutoMock:
    def __init__(self, name='mock'): self._name = name
    def __getattr__(self, k): return _AutoMock(self._name + '.' + k)
    def __call__(self, *a, **kw):
        print('-> ' + self._name + '() called')
        return _AutoMock(self._name + '()')
    def __repr__(self): return '<mock ' + self._name + '>'
    def __str__(self): return '<mock ' + self._name + '>'
    def __bool__(self): return True
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __getitem__(self, k): return _AutoMock(self._name + '[...]')
    def __setitem__(self, k, v): pass
    def __enter__(self): return self
    def __exit__(self, *a): return False
    async def __aenter__(self): return self
    async def __aexit__(self, *a): return False
    def __add__(self, o): return self
    def __radd__(self, o): return self
    def __sub__(self, o): return self
    def __mul__(self, o): return self
    def __rmul__(self, o): return self
    def __truediv__(self, o): return self
    def __eq__(self, o): return isinstance(o, _AutoMock)
    def __hash__(self): return hash(self._name)
    def __lt__(self, o): return True
    def __le__(self, o): return True
    def __gt__(self, o): return False
    def __ge__(self, o): return False
    def __mro_entries__(self, bases): return (object,)

kwargs = _AutoMock('kwargs')

Run in Colab or locally with pip install tensorflow. Expected outputs in comments.

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers

class ResidualBlock(layers.Layer):
    """Standard ResNet basic block: Conv-BN-ReLU-Conv-BN + skip → ReLU."""

    def __init__(self, filters, stride=1, **kwargs):
        super().__init__(**kwargs)
        self.filters = filters
        self.stride  = stride

        # Main path
        self.conv1 = layers.Conv2D(filters, 3, strides=stride,
                                   padding="same", use_bias=False)
        self.bn1   = layers.BatchNormalization()
        self.conv2 = layers.Conv2D(filters, 3, strides=1,
                                   padding="same", use_bias=False)
        self.bn2   = layers.BatchNormalization()

        # Will be created in build() if needed
        self.proj_conv = None
        self.proj_bn   = None

    def build(self, input_shape):
        in_channels = input_shape[-1]
        if self.stride != 1 or in_channels != self.filters:
            self.proj_conv = layers.Conv2D(self.filters, 1,
                                           strides=self.stride, use_bias=False)
            self.proj_bn   = layers.BatchNormalization()
        super().build(input_shape)

    def call(self, x, training=False):
        shortcut = x

        y = self.conv1(x)
        y = self.bn1(y, training=training)
        y = tf.nn.relu(y)

        y = self.conv2(y)
        y = self.bn2(y, training=training)

        if self.proj_conv is not None:
            shortcut = self.proj_conv(shortcut)
            shortcut = self.proj_bn(shortcut, training=training)

        y = y + shortcut
        y = tf.nn.relu(y)               # activation AFTER addition
        return y

    def get_config(self):
        return {**super().get_config(),
                "filters": self.filters, "stride": self.stride}


# Sanity check
inp = keras.Input(shape=(32, 32, 64))
x   = ResidualBlock(filters=128, stride=2)(inp)     # (None, 16, 16, 128)
x   = ResidualBlock(filters=128, stride=1)(x)       # (None, 16, 16, 128)
model = keras.Model(inp, x)
model.summary()
# Layer (type)                Output Shape              Param #
# input (InputLayer)          (None, 32, 32, 64)        0
# residual_block_0            (None, 16, 16, 128)       ~230k
# residual_block_1            (None, 16, 16, 128)       ~296k

# Lightweight mock for objects whose attributes/methods aren't critical
class _AutoMock:
    def __init__(self, name='mock'): self._name = name
    def __getattr__(self, k): return _AutoMock(self._name + '.' + k)
    def __call__(self, *a, **kw):
        print('-> ' + self._name + '() called')
        return _AutoMock(self._name + '()')
    def __repr__(self): return '<mock ' + self._name + '>'
    def __str__(self): return '<mock ' + self._name + '>'
    def __bool__(self): return True
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __getitem__(self, k): return _AutoMock(self._name + '[...]')
    def __setitem__(self, k, v): pass
    def __enter__(self): return self
    def __exit__(self, *a): return False
    async def __aenter__(self): return self
    async def __aexit__(self, *a): return False
    def __add__(self, o): return self
    def __radd__(self, o): return self
    def __sub__(self, o): return self
    def __mul__(self, o): return self
    def __rmul__(self, o): return self
    def __truediv__(self, o): return self
    def __eq__(self, o): return isinstance(o, _AutoMock)
    def __hash__(self): return hash(self._name)
    def __lt__(self, o): return True
    def __le__(self, o): return True
    def __gt__(self, o): return False
    def __ge__(self, o): return False
    def __mro_entries__(self, bases): return (object,)

kwargs = _AutoMock('kwargs')

What You Learned

• A conv layer's output shape is $\lfloor (H - k + 2p) / s \rfloor + 1$. Params are $(k^2 C_{\text{in}} + 1) C_{\text{out}}$ — independent of spatial size.
• Receptive field grows with depth and stride. Stacked $3\times3$ convs beat single $5\times5$ on both params and expressiveness.
• $1\times1$ convs mix channels per-pixel. Used everywhere for cheap channel manipulation and bottlenecks.
• Global average pooling + one Dense replaces the old Flatten + huge Dense head. Modern practice — fewer params, less overfitting, input-size agnostic.
• The CNN family tree: LeNet → AlexNet → VGG (stack small kernels) → Inception (parallel paths) → ResNet (skip connections) → DenseNet → MobileNet (depthwise-separable) → EfficientNet (compound scaling) → ConvNeXt (transformer-era tricks for CNNs).
• Residual blocks are the LEGO brick. Internalise the shape: two convs + BN + ReLU + skip + ReLU.
• Depthwise-separable convs are ~9× cheaper than regular convs for similar accuracy — the mobile-deployment trick.
• Dilated convs grow receptive field without downsampling — used in segmentation and audio.
• Beyond classification: detection (YOLO, R-CNN), segmentation (U-Net, Mask R-CNN), generation (GANs, diffusion U-Net).
• Augmentation is non-optional on small datasets. GPU-runnable Keras layers (RandomFlip, RandomRotation...) are the cheap path; Mixup/CutMix/RandAugment for the strong path.
• Inference optimisations: quantisation (int8, 4× smaller), pruning (sparse), distillation (small student). Compose for huge speedups with small accuracy hits.

Next: RNNs & LSTMs: Sequential Architectures — when your input has order, the architecture that learns from sequence one token at a time.