Description:

Intelligent Adaptive Experimentation: Integrating Statistics & Machine Learning to Accelerate Science and Help People

Joseph Jay Williams, Assistant Professor, Computer Science, University of Toronto

Abstract: Randomized 'A/B' experiments are both: (a) A fundamental scientific tool in research on people – how they learn, make decisions, interact with each other; and increasingly (b) A practical tool for testing which versions of websites, apps, messages best help people. To integrate scientific discovery and practical impact, we explore the use of machine learning algorithms and Bayesian & Frequentist statistics for Adaptively Randomized Experiments, in which the probability of assigning arms/conditions to people is adapted as data is being collected.

(1) We empirically investigate the multi-armed bandit algorithm "Posterior/Thompson Sampling", used in many applications, which aims to assign an arm in proportion to the posterior probability it is optimal (based on the data collected). The results quantify how and why outcome-maximizing bandit algorithms can increase false positives (incorrectly concluding differences in arms that do not exist) and reduce power (correctly detecting actual differences in arms), as well as bias posterior distributions.

(2) We therefore introduce a more "Statistically Considerate" algorithm, which incorporates a sampling step for traditional experimentation, which is proportional to the probability arm differences are "small". This allows adaptive and efficient reduction in false positives and increased power, while maximizing outcomes for larger differences, better than extant approaches.

(3) We further investigate statistical tests that take into account the properties of how algorithms collect data: Inverse-probability weighting; an "Algorithm-Induced" Hypothesis Test; and a novel Assignment Probability test statistic, which is a more discerning function of the posterior probabilities the algorithm uses to assign arms. These address the inference challenges to varying degrees.

This work collectively clarifies opportunities and challenges in adaptive experimentation: Pursuing applications, models, and algorithms that can accelerate scientific discoveries while directly helping people.