Skip to main content

Course

Machine Learning with Tree-Based Models in Python

Intermediate

4.8+

Updated 05/2025

In this course, you'll learn how to use tree-based models and ensembles for regression and classification using scikit-learn.

Start Course for Free

Included withPremium or Teams

PythonMachine Learning5 hours15 videos57 Exercises4,650 XP101,606Statement of Accomplishment

Create Your Free Account

or

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.

Training 2 or more people?

Try DataCamp for Business

Loved by learners at thousands of companies

Course Description

Decision trees are supervised learning models used for problems involving classification and regression. Tree models present a high flexibility that comes at a price: on one hand, trees are able to capture complex non-linear relationships; on the other hand, they are prone to memorizing the noise present in a dataset. By aggregating the predictions of trees that are trained differently, ensemble methods take advantage of the flexibility of trees while reducing their tendency to memorize noise. Ensemble methods are used across a variety of fields and have a proven track record of winning many machine learning competitions. In this course, you'll learn how to use Python to train decision trees and tree-based models with the user-friendly scikit-learn machine learning library. You'll understand the advantages and shortcomings of trees and demonstrate how ensembling can alleviate these shortcomings, all while practicing on real-world datasets. Finally, you'll also understand how to tune the most influential hyperparameters in order to get the most out of your models.

Prerequisites

Supervised Learning with scikit-learn

1

Classification and Regression Trees

Decision tree for classification

Train your first classification tree

Evaluate the classification tree

Logistic regression vs classification tree

Classification tree Learning

Growing a classification tree

Using entropy as a criterion

Entropy vs Gini index

Decision tree for regression

Train your first regression tree

Evaluate the regression tree

Linear regression vs regression tree

2

The Bias-Variance Tradeoff

Generalization Error

Complexity, bias and variance

Overfitting and underfitting

Diagnose bias and variance problems

Instantiate the model

Evaluate the 10-fold CV error

Evaluate the training error

High bias or high variance?

Ensemble Learning

Define the ensemble

Evaluate individual classifiers

Better performance with a Voting Classifier

3

Bagging and Random Forests

Define the bagging classifier

Evaluate Bagging performance

Out of Bag Evaluation

Prepare the ground

OOB Score vs Test Set Score

Random Forests (RF)

Train an RF regressor

Evaluate the RF regressor

Visualizing features importances

4

Boosting

Define the AdaBoost classifier

Train the AdaBoost classifier

Evaluate the AdaBoost classifier

Gradient Boosting (GB)

Define the GB regressor

Train the GB regressor

Evaluate the GB regressor

Stochastic Gradient Boosting (SGB)

Regression with SGB

Train the SGB regressor

Evaluate the SGB regressor

5

Model Tuning

Tuning a CART's Hyperparameters

Tree hyperparameters

Set the tree's hyperparameter grid

Search for the optimal tree

Evaluate the optimal tree

Tuning a RF's Hyperparameters

Random forests hyperparameters

Set the hyperparameter grid of RF

Search for the optimal forest

Evaluate the optimal forest

Congratulations!

Machine Learning with Tree-Based Models in Python

Course
Complete

Earn Statement of Accomplishment

Add this credential to your LinkedIn profile, resume, or CV
Share it on social media and in your performance review

Included withPremium or Teams

Don’t just take our word for it

*4.8

from 155 reviews

88%

12%

1%

0%

0%

Highest to Lowest
Lowest to Highest
Most recent
Top reviews

Uduak

about 8 hours

Noah

about 19 hours

This is one of the greatest course I have seen in DataCamp. Whether you are a beginner or an expert, there is something to be learned about Tree based Models from this course.

Miguel Ángel

2 days

Winners

3 days

Great!

Luis Alejandro

3 days

Bence

3 days

Uduak

"This is one of the greatest course I have seen in DataCamp. Whether you are a beginner or an expert, there is something to be learned about Tree based Models from this course."

Noah

Miguel Ángel

FAQs

Join over 16 million learners and start Machine Learning with Tree-Based Models in Python today!

Create Your Free Account

or

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.