Rohan Sundar Kodialam

Rohan Sundar Kodialam
Undergraduate Research and Innovation Scholar
Advisor: Alexander Rakhlin
Department: Physics
Years: 2018-2019
Research Project Title:

Long-Memory Recurrent Models through Regularization

abstract:Recurrent Neural Nets (RNNs) are increasingly popular tools for a variety of machine learning applications. However, past research has exposed numerous drawbacks to setting up and training RNNs using the standard back-propagation algorithm using gradient descent. In particular, the exploding and vanishing gradient problems make RNNs hard to train. It has been hypothesized that because of these problems RNNs do not work better in practice than much simpler feed-forward neural networks, and that feed-forward networks are sufficient for modelling tasks. We plan to outline a method by which a RNN can be regularized in order to mitigate the exploding and vanishing gradient problems. We further plan to empirically show that these RNNs can solve benchmark problems that cannot be solved by a feed-forward network of comparable size. and that these RNNs are able to maintain longer-term memory than un-regularized models.