Rajiv  Movva

Rajiv Movva

Scholar Title

MIT EECS Undergraduate Research and Innovation Scholar

Research Title

Parallel Pruning for Faster Discovery of Neural Network Lottery Tickets

Cohort

2020–2021

Department

EECS

Research Areas
  • Artificial Intelligence & Machine Learning
Supervisor

Michael Carbin

Abstract

Deep neural networks have revolutionized a number of important tasks in AI, including computer vision and machine translation. However, these models are often overly complex, limiting their practical deployment. Pruning away unimportant neural network weights is one way to reduce complexity, and can often be done without affecting performance. However, effective pruning itself currently relies on inefficient training techniques. In this project, we develop methods to speed up iterative model pruning by parallelizing across hardware. We simultaneously train multiple models with equivalent architecture, identify prunable weights in each network, and aggregate this information to prune more weights at each iteration. We study whether this strategy allows us to achieve more sparse and efficient models than traditional single-network pruning methods.

Quote

During SuperUROP, I’ll try to figure out the type and style of research I enjoy. Hopefully this year will bring some clarity to my post-undergrad plans. Content-wise, the experience will differ quite a bit from my previous work in computational biology, so I’m looking forward to that freshness. Also, I’m pretty excited to work with Jonathan Frankle and Prof. Carbin. I’m a big fan of their work.

Back to Scholars