Skip to content
learn
  1. Learn
  2. Machine Learning
  3. Neural Networks and Deep Learning Foundations

ML-M08

Neural Networks and Deep Learning Foundations

How neural networks compute and learn: neurons and layers, activations and losses, backpropagation, optimizers, and the tricks that make deep training work.

13 lessonsabout 160 min

Neural networks power almost every headline result in modern machine learning, from image recognition to language models. Underneath, they are built from one simple part, a neuron that weighs its inputs and fires, stacked into layers and trained by following gradients downhill.

You will start with the perceptron, build multi-layer networks, compute a forward pass by hand, derive backpropagation and write your first PyTorch training loop. Then you will learn the practical toolkit: optimizers such as Adam, learning rate schedules, initialization and normalization, regularization, residual connections, and a checklist for debugging training that refuses to work.

Lessons

  1. ML-08.01 The perceptron and the artificial neuron An artificial neuron multiplies each input by a weight, sums them with a bias and passes the result through an activation function. Beginner 12 min
  2. ML-08.02 Multi-layer networks A multi-layer network stacks layers of neurons, where each layer transforms the previous layer's output. Beginner 14 min
  3. ML-08.03 Activation functions Activation functions add non-linearity; without them, any stack of layers collapses into a single linear function. Beginner 10 min
  4. ML-08.04 Loss functions The loss function defines what good means. Beginner 10 min
  5. ML-08.05 Forward pass by hand In a forward pass, inputs flow through the network layer by layer: each layer multiplies by its weights, adds biases and applies an activation, until the output layer produces a prediction, which is compared with the label to compute the loss. Beginner 12 min
  6. ML-08.06 Backpropagation Backpropagation computes the gradient of the loss with respect to every weight efficiently. Intermediate 20 min
  7. ML-08.07 Autograd and your first PyTorch training loop In PyTorch, tensors with requires_grad record operations so calling loss.backward() computes gradients automatically. Intermediate 15 min
  8. ML-08.08 Optimizers: SGD, momentum, RMSProp, Adam, AdamW Optimizers decide how gradients update weights. Intermediate 14 min
  9. ML-08.09 Learning rate schedules and warmup A fixed learning rate is rarely best. Intermediate 8 min
  10. ML-08.10 Weight initialization and normalization Good initialization keeps signals and gradients at a healthy scale through deep networks; Xavier and He initialization scale random weights by layer size. Advanced 12 min
  11. ML-08.11 Regularization for deep nets Deep networks can memorize training data, so we regularize them. Intermediate 10 min
  12. ML-08.12 Vanishing and exploding gradients, residual connections In very deep networks, gradients can shrink toward zero or grow out of control as they are multiplied through many layers. Advanced 12 min
  13. ML-08.13 Debugging neural network training When training fails, first check that the model can overfit a single small batch; if it cannot, there is a bug. Intermediate 12 min

Start with lesson 1