Jerry  Mao

Jerry Mao

Scholar Title

MIT EECS | CS+HASS Undergraduate Research and Innovation Scholar

Research Title

How to use language to control robots: Language-conditioned imitation learning for SE(3) manipulation

Cohort

2021–2022

Department

EECS

Research Areas
  • CS+HASS
  • Robotics
Supervisor

Pulkit Agrawal

Abstract

Recent advances in robotic research have increasingly focused on systems for complex manipulation tasks. State-of-the-art systems are already able to perform language-conditioned table-top object manipulation, as well as imitate pick-and-place actions in 3D. However, there is not yet an agent capable of efficiently learning to manipulate objects in 3D in response to language prompts. We propose an imitation learning agent to learn language-conditioned out-of-plane manipulation, imbuing a robotic system with semantic awareness to augment its physical manipulation capabilities.

Quote

I subsist on the allure of challenges. From my experiences as a UROP with the Improbable AI lab, and theoretical grounding in classes such as 6.867 and 6.884, I am excited to pursue the challenge of probing the boundaries of what RL can achieve.

Back to Scholars