Despite significant improvements on many benchmarks, image classification models fail to match human performance in real world conditions. This implies that accuracy on classification benchmarks misrepresents human object recognition ability. In this project, we leverage human psychophysics to characterize the recognition difficulty of images from two widely used datasets (ImageNet and ObjectNet). We find that these datasets vastly oversample images that humans solve very easily, while undersampling difficult images. By conditioning model performance on image difficulty, we hope to reveal an important and systematic shortcoming of classification benchmarks and propose analyses to improve model evaluation.

I am fascinated by intelligence. What it is, how it emerges in different organisms, and how to reproduce it in silica. I want to leverage our increasingly deep understanding of the human brain to improve AI models.