AI decision-making methods often lack the critical ability to generalize to unseen challenges. In contrast, large foundation models have achieved strong out-of-distribution generalization by ingesting broad amounts of semi-structured data. We argue that a key step in towards generalizable decision-making is the creation of a rich data-ingestion architecture for training agents. We present the Sponge model, a singular model trained on many objectives. This research specifically answers questions on the *data* side — what should be trained on to form strong decision-making capabilities? We develop and release a solid framework capable of solving increasingly difficult gridworld tasks, conduct a thorough study on various data sources, and show that data-sponge models display a strong capability of generalization.

I am participating in SuperUROP because I am interested in doing research, and the program provides a structured way of doing so. I have acquired various background knowledge about AI, open-endedness, design, etc., and I am excited to apply it to unsolved problems.