Simultaneous EEG-fMRI combines the high temporal resolution of EEG and the high spatial resolution of fMRI, allowing us to investigate the spatiotemporal dynamics of electrophysiological neural rhythms that underlie cognition and vigilance. However, the complex relationship between fMRI and EEG signals and the substantial inter-subject variability in how these signals are related make jointly analyzing the two challenging. As a novel approach for analyzing EEG-fMRI, we developed a machine learning framework to decode EEG spectrograms (0-17.1 Hz) from brainwide fMRI data. By predicting spectrograms, our model leverages the covarying structure of neural oscillations. Our model consists of an RNN encoder and a nonlinear fully-connected decoder. We created a Subject Generalization model that predicts EEG spectrograms for unseen subjects, and a Subject Specific model that learns individualized EEG features. We applied this approach to an EEG-fMRI dataset of 27 subjects transitioning through wakefulness and non-REM sleep stages, brain states which are defined by neural rhythms. We found that the Subject Generalization model accurately decoded EEG spectrograms. The Subject Specific model significantly improved decoding for all frequencies above 6 Hz, demonstrating that inter-subject variability in the EEG and fMRI relationship is smaller in slower frequencies. To determine which brain areas contain predictive information, we trained models on each gray matter region. We found that certain networks, such as arousal systems, broadly decoded all frequencies, whereas others were selectively predictive of specific rhythms, such as the visual network predicting alpha (8.5-12 Hz). However, decoding benefited from learning simultaneously from all brain regions, demonstrating that they contain complementary information. Our results show that fMRI contains rich information about neural rhythms that allow decoding EEG dynamics, and that this information is distributed across large-scale brain networks. Additionally, this approach enables investigation of brainwide dynamics underlying oscillatory activity in a variety of experimental paradigms.

I am participating in SuperUROP because it is an amazing opportunity to bolster my research experience, specifically learning how to solve problems that have not been solved before. Additionally, I am interested in the intersection of machine learning and neuroscience, specifically how we can use machine learning to learn more about the brain and cognitive processes. I believe this SuperUROP will help me start answering this question.