Tree ensembles combine multiple decision trees that recursively partition the input space and assign predictions to each region. Since these models can contain hundreds of trees and tens of thousands of nodes, improving interpretability and efficiency calls for removing unnecessary nodes/trees (pruning) and selecting a subset of features/rules (sparsity). Our proposal merges FIRE’s fusion and non-convex sparsity penalties with ForestPrune’s binary depth-pruning into a unified framework. We will select smaller ensembles using fused-LASSO with node- and depth-weighted matrices to encourage smaller and fewer trees, respectively. We will also explore combining subtrees and post-processing to prioritize a small set of input features, yielding compact, transparent ensembles improving interpretability.

I hope to gain experience working with classical statistical models through this SuperUROP. In addition, I want to further my knowledge on commonly used optimization techniques and tools.