Skip to content
learn
  1. Learn
  2. Machine Learning
  3. Trees and Ensembles

ML-M06

Trees and Ensembles

Decision trees and the ensembles built from them: random forests and gradient boosting, the strongest models for tabular data.

6 lessonsabout 75 min

A single decision tree is a flowchart of yes-or-no questions: easy to read, quick to train and badly prone to overfitting. Combine many trees, though, and you get the models that win most real-world problems on tables of numbers and categories.

You will learn how a tree chooses its splits, how bagging and random forests average away variance, how boosting fixes errors one small tree at a time, how to use XGBoost, LightGBM and CatBoost in practice, how stacking blends different models, and how to explain what a tree ensemble has learned with permutation importance and SHAP.

Lessons

  1. ML-06.01 Decision trees A decision tree splits data with a sequence of yes-or-no questions on features, choosing each split to make the resulting groups as pure as possible, measured with Gini impurity or entropy. Beginner 14 min
  2. ML-06.02 Bagging and random forests Bagging trains many models on random bootstrap samples of the data and averages their predictions, which reduces variance. Intermediate 12 min
  3. ML-06.03 Boosting: AdaBoost to gradient boosting Boosting builds models sequentially, each one focusing on the errors of the ones before. Intermediate 16 min
  4. ML-06.04 XGBoost, LightGBM and CatBoost in practice XGBoost, LightGBM and CatBoost are fast, production-grade gradient boosting libraries. Intermediate 14 min
  5. ML-06.05 Stacking and blending Stacking trains a meta-model that learns how to combine the predictions of several different base models. Advanced 8 min
  6. ML-06.06 Feature importance and SHAP Feature importance explains which inputs a model relies on. Intermediate 12 min

Start with lesson 1