Calculus

First Principles Question

How do you find the bottom of a bowl when you can only feel the slope beneath your feet?

The Core Idea

A derivative tells you the slope of a function at a point — which direction it’s going up, and how steeply. If you want to minimize something (like prediction error), you follow the slope downhill. That’s the entire engine of modern ML.

Key Concepts to Cover

• Derivative as instantaneous rate of change — the limit definition
• Partial derivatives — slope in one direction while holding others fixed
• Chain rule — how to differentiate composed functions (critical for backprop)
• Gradient — the vector of all partial derivatives; points uphill
• Second derivative — is the slope increasing or decreasing? (curvature)
• Minima: where derivative = 0, and second derivative > 0

Why It Matters for ML

• Linear Regression: set derivative of loss to zero → closed form solution
• Gradient Descent: move opposite to gradient to reduce loss
• Backpropagation: chain rule applied repeatedly through network layers
• GBT: each new tree fits the negative gradient of the loss

Prerequisites

None. This is Layer 0.

Builds To

Optimization & Gradient Descent · Linear Regression · Gradient Boosting Tree · Convolutional Neural Network

Content Ideas

Obsidian note: “The derivative is just a slope. Here’s why that’s enough to train any ML model.” X post: “Backpropagation is just the chain rule. Applied 50 times. That’s it.” GitHub: Animate gradient descent on a 2D loss surface