Exploring object deformation provides insights into materials, and being able to predict an object’s deformation reveals significant information as to its composition. We use the Greatest Hits video data set, where a drumstick is scratches or hits various objects, and apply computer vision techniques to predict the deformation. Optical flow is used to determine whether there is deformation and characterize that deformation. The first component of this project involves training a convolutional neural network (CNN) to predict from video, whether deformation will occur. We build off this to then predict the area of the deformation and the deformation itself through optical flow. Lastly, audio input is added as an extra feature set into the CNN to achieve higher accuracy predictions.

I became interested in computer vision and computational photography after taking the computational photography class. This class along with a machine learning class and other background research I have done over the summer have prepared me for my SuperUROP. I hope to learn how to effectively apply machine learning to perform novel computer vision tasks.