Decision Tree

A decision tree splits the data into increasingly pure groups. We first work out how those splits are chosen by hand on the golf dataset, then fit a tree with scikit-learn and submit predictions to Kaggle.

Topics

• Decision Tree Basics — purity and entropy, information gain, and the ID3 algorithm worked through by hand; split points for numeric features, overfitting, and pruning.
• Decision Tree Prediction — fit a DecisionTreeClassifier, predict on the test set, build the Kaggle submission file, visualize the tree, and copy the complete script from the end of the page.

Datasets and Notebooks

• golf-id3.csv.zip — the 14-row golf dataset used throughout Decision Tree Basics.
• id3-simple-starter.ipynb.zip — starter notebook for computing entropy and information gain on that dataset in code.