Finding an optimal intervention through sequential experimental design is a problem with many important applications in medicine and other fields. In sequential experimental design, one is repeatedly allowed to apply an intervention and observe its effects. The choice of the next intervention should depend on our observations from previous interventions – the goal is to most effectively choose the intervention at each step to help us converge towards the optimal intervention.

To help us choose interventions, an estimate of how good an intervention will perform is very useful. To this end, we construct a doubly-robust estimator. This estimator has been used in reinforcement learning settings. We extend it to a causal model, where the intervention can be arbitrarily “upstream.” We then construct high-probability bounds for this estimator, which will inform us on how to choose successive interventions in sequential experimental design. To design a procedure, we draw inspiration from algorithms used for multi-armed bandit problems.

This project excites me for two reasons: it is strongly motivated by applications, and I get to use advanced mathematical concepts from my coursework. I am really excited to continue working on this project this year as a SuperUROP!