Adriano Hernandez

Adriano  Hernandez
MIT EECS | Quick Undergraduate Research and Innovation Scholar
Advisor: Marin Soljačić
Department: EECS
Areas of Research: Representation Learning
Years: 2021-2022
Research Project Title:

Exploring the Efficacy and Properties of Model Stitching

abstract:We expand model stitching (Lenc \& Vedaldi 2015) as a methodology to compare neural networks. Previously, Bansal, Nakkiran & Barak used it to compare the representations learned by differently seeded and/or trained neural networks. We use it to compare the representations learned by neural networks with different architectures. This gives us insight into the properties of stitching. Namely, we find that stitching can reach unintuitively high accuracy for intuitively different layers if those layers come earlier in the first (sender) network than in the second (receiver). This leads us to hypothesize that stitches are not in fact learning to match the representations expected by receiver layers, but instead to find representations which yield similar results.

I am doing SuperUROP because I'm curious to explore machine learning research, especially as a way to understand our own inductive biases.