Description:

Title: Introduction to Mathematics of Artificial Neural Networks (ANNs)

Speaker: Leonid Berlyand, The Pennsylvania State University

Abstract: We begin by addressing the image classification problem, where the goal is to map images x to classes α using an exact classifier Φ(x,α). Since an exact classifier is often unfeasible, Artificial Neural Networks (ANNs) F(x,α) are used to approximate Φ(x,α), with α representing tunable parameters. Unlike classical methods that use coefficients in expansions, ANN parameters are inspired by the structure of the human brain. The process of optimizing these parameters is called training.

We highlight two advancements: First, a pruning technique using the Marchenko-Pastur spectral approach from Random Matrix Theory (RMT), which reduces computational complexity without sacrificing accuracy. Second, we examine autoencoders, a special type of ANN used for image-to-image transformations. The focus is on the fixed points of the autoencoder function F(x,α), crucial for distinguishing real images from fakes. Using the Banach Fixed Point Theorem, we show that with light-tailed distributions (e.g. Gaussian), there is a unique stable fixed point, while with heavy-tailed distributions (e.g. Cauchy), there are multiple fixed points N. The number of fixed points N depends non-monotonically on the number of layers L, suggesting an optimal number of layers L_0 for best performance. These results are vital for improving autoencoder design.

Zoom: https://northwestern.zoom.us/j/92383067097

-----

To subscribe to the Applied Mathematics Colloquia List send a message to LISTSERV@LISTSERV.IT.NORTHWESTERN.EDU with the command:

SUBSCRIBE esam-seminar FirstName LastName