How do neural networks learn features from data?
Adityanarayanan Radhakrishnan, Assistant Professor of Mathematics, Massachusetts Institute of Technology
Abstract: The ability of neural networks to learn patterns from data, or features, has been central to their success. In this talk, I will present a unifying mechanism that characterizes feature learning across neural network architectures. Namely, features learned by neural networks are captured by a statistical operator known as the average gradient outer product (AGOP). More generally, the AGOP enables feature learning in machine learning models that have no built-in feature learning mechanism (e.g., kernel methods). I will present two applications of this line of work. First, I will show how AGOP can be used to steer LLMs and vision-language models, guiding them towards specified concepts and shedding light on vulnerabilities in these models. I will then discuss how AGOP connects feature learning with independence testing and how we used AGOP to develop a scalable, nonlinear measure of dependence known as the InterDependence Score (IDS). I will conclude with an application of IDS to million-scale text and genomics datasets, where we use it to identify subpopulations of interest.
Cost: free