The sparse reward problem is a problem that arises in many RL problems (e.g. robotic planning) where the lack of reward signal hinders learning. One way to address this issue is to use skills–temporal abstractions that each consist of a policy of low-level actions for achieving a subtask. They allow the agent to act at a higher level, which boosts RL performance in certain domains. However, skills have not enjoyed widespread use due to their failure to improve RL in many other domains. It is not well understood when we expect skills to be useful and, in cases where they are, why. To address this gap in our understanding of RL skills, my project theoretically analyzes the effect of skills on RL performance and elucidates the exact mechanisms by which they improve RL learning.

Through my UROP projects, I discovered that I enjoy doing research and want to learn more about it, and I think SuperUROP provides great resources for a more structured learning experience of how research works. In addition, I discovered neurosymbolic learning through one of my UROP projects and find it to be an exciting emerging field of research, and I hope to use SuperUROP as an opportunity to further explore this field.