Towards the Development of a Robotic System with Conversational Audio-Visual Localization Capabilities

Robotic systems that can interact with humans have the potential to fill an important niche in situations that are inherently dangerous or tedious for humans such as healthcare and disaster relief. One component of the human-robot interaction problem involves robotic participation in human spoken conversation where a robot would effectively respond to verbal instructions and non-verbal cues. In this project we aim to demonstrate the feasibility of a simple robotic system that can orient itself in the direction of a speaking subject. In particular we will implement audio and visual localization modules on top of existing hardware and conduct experiments to gauge the effectiveness of the integrated system under different environmental and subject configurations.

The SuperUROP program will give me the opportunity to apply the skills acquired from my artificial intelligence work on the UAV Team and my coursework in machine learning and signal processing to a challenging research project. I feel that my project has the potential to advance the progress of human-computer interaction hopefully bringing us closer to more intelligent robotic systems.