Dear Colleagues, we would like to invite you to the following seminar by Enrico Malatesta (Bocconi) to be held Wednesday, May 25th, at Dipartimento di Matematica in Pisa and online via Google Meets.
The organizers, A. Agazzi and F. Grotto
--------------------------------------------
Location: Sala Seminari, Dipartimento di Matematica, Pisa Google Meet Link: https://meet.google.com/gji-phwo-vbg
Time: May 25th, 2022, 14:00-15:00 CET Speaker: Enrico Malatesta Title: Phase transitions in the landscape of solutions of overparametrized neural networks. Abstract: Current deep neural networks are nonlinear devices composed of a number of parameters that far exceed the number of data points. Understanding how these systems can fit the data almost perfectly through variants of gradient descent algorithms and achieve exceptional levels of prediction accuracy without overfitting are key conceptual challenges. In this talk I will show how common techniques used in machine learning (e.g. the choice of the activation function or the loss) deeply affect the loss landscape, tending to mild its roughness. Then we shed light on the role of overparameterization in non-convex neural networks. By analytically studying a non-convex model of random features, we identify a novel (non-equilibrium) phase transition, that we call “Local Entropy” transition, controlled by the degree of overparameterization. In non-convex models this transition is strictly different to the SAT/UNSAT threshold and it coincides with the appearance of highly entropic minima of the error loss function. Those minima, in turn, are found to be highly attractive to the learning algorithms currently used in deep learning.