Description:

Understanding the Universality Phenomenon in High-Dimensional Estimation and Learning: Some Recent Progress

Yue M. Lu, Gordon McKay Professor of Electrical Engineering and of Applied Mathematics, John A. Paulson School of Engineering and Applied Sciences, Harvard University

Abstract: Universality is a fascinating high-dimensional phenomenon. It points to the existence of universal laws that govern the macroscopic behavior of wide classes of large and complex systems, despite their differences in microscopic details. The notion of universality originated in statistical mechanics, especially in the study of phase transitions. Similar phenomena have been observed in probability theory, dynamical systems, random matrix theory, and number theory.

In this talk, I will present some recent progress in rigorously understanding and exploiting the universality phenomenon in the context of statistical estimation and learning on high-dimensional data. Examples include spectral methods for high-dimensional projection pursuit, statistical learning based on kernel and random feature models, approximate message passing algorithms, structured random dimension reduction maps for efficient sketching, and regularized linear regression on highly structured, strongly correlated, and even (nearly) deterministic design matrices. Together, they demonstrate the robustness and wide applicability of the universality phenomenon.

Based on joint work with Rishabh Dudeja, Hong Hu, Subhabrata Sen, and Horng-Tzer Yau.
(arXiv:2208.02753, arXiv:2205.06308, arXiv:2205.06798, arXiv:2009.07669)