Adriano  Hernandez

Adriano Hernandez

Scholar Title

MIT EECS | Quick Undergraduate Research and Innovation Scholar

Research Title

Exploring the Efficacy and Properties of Model Stitching

Cohort

2021–2022

Department

EECS

Research Areas
  • Artificial Intelligence & Machine Learning
Supervisor

Marin Soljacic

Abstract

We expand model stitching (Lenc & Vedaldi 2015) as a methodology to compare neural networks. Previously, Bansal, Nakkiran & Barak used it to compare the representations learned by differently seeded and/or trained neural networks.

We use it to compare the representations learned by neural networks with different architectures.
This gives us insight into the properties of stitching. Namely, we find that stitching can reach unintuitively
high accuracy for intuitively different layers if those layers come earlier in the first (sender) network than in
the second (receiver). This leads us to hypothesize that stitches are not in fact learning to match the
representations expected by receiver layers, but instead to find representations which yield similar results.

Quote

I am doing SuperUROP because I’m curious to explore machine learning research, especially as a way to understand our own inductive biases.

Back to Scholars