Recognizing and grounding actions from observations is a popular problem in deep learning. In last project, we learned first-order logic formulas over primitive predicates for action grounding in videos. Specifically, we designed learnable primitive predicates on physical properties, such as DQThe distance between object A and object B is larger than θDQ where θ is the parameter to learn. Then we model actions with first-order logic formulas over these predicates. In the SuperUROP project, we will first try to replace human-designed predicates with neural-based predicates, which will improve the generalizability. Then, we will apply our network on modeling tasks involving multi-step, object-centric goals so it can be used for action recognition and task planning.

I am participating in SuperUROP because I want to engage more in the most frontier research being done at MIT and also fulfill the EECS CIM-requirement. Before the SuperUROP program, I have been doing UROP for two semesters, and the SuperUROP project is some kind of extension of the previous project. I hope this project will improve my understanding of this area and my research skills.