Deep neural networks have revolutionized a number of important tasks in AI, including computer vision and machine translation. However, these models are often overly complex, limiting their practical deployment. Pruning away unimportant neural network weights is one way to reduce complexity, and can often be done without affecting performance. However, effective pruning itself currently relies on inefficient training techniques. In this project, we develop methods to speed up iterative model pruning by parallelizing across hardware. We simultaneously train multiple models with equivalent architecture, identify prunable weights in each network, and aggregate this information to prune more weights at each iteration. We study whether this strategy allows us to achieve more sparse and efficient models than traditional single-network pruning methods.

During SuperUROP, I’ll try to figure out the type and style of research I enjoy. Hopefully this year will bring some clarity to my post-undergrad plans. Content-wise, the experience will differ quite a bit from my previous work in computational biology, so I’m looking forward to that freshness. Also, I’m pretty excited to work with Jonathan Frankle and Prof. Carbin. I’m a big fan of their work.