Convolutional Neural Network (CNN)
First Principles Question
How do you recognize a cat in an image regardless of where in the image the cat appears?
The Core Idea
A convolution slides a small filter (kernel) across an image, computing a dot product at each position. The filter detects a local pattern — an edge, a texture, a shape. The same filter is applied everywhere (weight sharing). Stack enough of these layers and the network builds increasingly abstract representations: edges → textures → parts → objects.
The Math
Convolution: (f * g)(t) = Σ f(τ)·g(t - τ)
For 2D images:
Output[i,j] = Σₘ Σₙ Input[i+m, j+n] · Kernel[m,n]
After conv: apply ReLU activation → non-linearity
Pooling: downsample spatial dimensions, preserve channels
Flatten → Fully Connected → Softmax → class probabilities
Key Properties
- Local connectivity: each neuron only sees a small region
- Weight sharing: same kernel applied everywhere → fewer parameters
- Translation invariance: can detect patterns regardless of position (pooling helps)
- Depth: early layers = low-level features, deep layers = high-level semantics
Training
Backpropagation through convolution layers using chain rule from Calculus. Loss: cross-entropy (from Information Theory). Optimizer: Adam / SGD (from Optimization & Gradient Descent).
Connection to Your Work
You fine-tuned SAM (built on ViT, a transformer cousin of CNN) on satellite building footprint segmentation. The vision encoder in SAM encodes images into feature embeddings using this same mechanism.
Prerequisites
Linear Algebra · Calculus · Optimization & Gradient Descent · Distance & Similarity
Builds To
Transformers, Vision Transformers (ViT), SAM fine-tuning
Content Ideas
Obsidian note: “CNNs don’t understand images. They understand local patterns. Here’s what a convolution actually computes.”
X post: “A CNN is just a bunch of sliding dot products stacked on top of each other. Here’s why that’s enough to recognize anything.”
GitHub: sam-building-footprints — your existing SAM fine-tuning pipeline