Dexterous manipulation of objects requires estimating object and manipulator states, such as the position and orientation. Utilizing RGBD cameras in realistic environments leads to occluded point clouds, objects appearing visually similar in different orientations, partially observable environments, and noisy depth readings which all lead to inaccurate state estimation and manipulation policies. Methods with several cameras and motion capture systems can be tedious and expensive, and may be unrealistic when deployed in real world settings. Leveraging tactile sensing will improve the performance of manipulation tasks and reduce the error that is currently seen with purely vision-based systems. In this work, we propose a framework leveraging tactile sensing identification and manipulation of various objects. This consists of multiple stages, where the manipulator is initially trained using privileged information in simulation to generate grasping policies that are deployed on the hardware. These policies are then used to collect data, which is used to train a policy for object classification with blind tactile sensing.

MIT provides so many opportunities for students to work on research alongside professionals and leaders. I have been able to work on projects in several labs at MIT in past UROPs and lab work. SuperUROP now gives me the opportunity to take these past experiences and refine my research skills, thoroughly investigate a problem, and produce my own research to make an impact on the world.