This research proposes the development of a novel multimodal AI system aimed at being able to learn general reasoning capabilities by integrating domain-specific (DSL) program synthesis with machine learning model training. The system will dynamically create and refine new modules, combining multimodal machine learning-based experts with DSL to tackle a wide range of reasoning tasks. The system aims to both use pre-trained experts while training auxiliary models using both real data and synthetic generated data to allow generalisation across diverse reasoning challenges. The system’s performance will be evaluated on multiple benchmarks, including the Abstraction and Reasoning Corpus (ARC), to validate its generalisation and abstraction capabilities.

SuperUROP allows me to leverage my machine learning skills, gained from experience with large language models, machine learning courses, and previous UROP work, in a longer term research project. I aim to advance my knowledge of multimodal AI and develop strong research skills, hoping to publish my results. I believe enhancing generalized reasoning is key to advancing artificial general intelligence, and my project is a step in that direction.