Dear Colleagues,
we would like to invite you to the
following seminar by Enrico Malatesta (Bocconi) to be
held Wednesday, May 25th, at Dipartimento di Matematica in Pisa
and online via Google Meets.
The organizers,
A. Agazzi and F. Grotto
--------------------------------------------
Time: May 25th, 2022, 14:00-15:00 CET
Speaker: Enrico Malatesta
Title: Phase transitions in the landscape of solutions of overparametrized neural networks.
Abstract: Current deep neural networks are nonlinear devices composed of a number
of parameters that far exceed the number of data points. Understanding
how these systems can fit the data almost perfectly through variants of
gradient descent algorithms and achieve exceptional levels of prediction
accuracy without overfitting are key conceptual challenges. In this
talk I will show how common techniques used in machine learning (e.g.
the choice of the activation function or the loss) deeply affect the
loss landscape, tending to mild its roughness. Then we shed light on the
role of overparameterization in non-convex neural networks. By
analytically studying a non-convex model of random features, we identify
a novel (non-equilibrium) phase transition, that we call “Local
Entropy” transition, controlled by the degree of overparameterization.
In non-convex models this transition is strictly different to the
SAT/UNSAT threshold and it coincides with the appearance of highly
entropic minima of the error loss function. Those minima, in turn, are
found to be highly attractive to the learning algorithms currently used
in deep learning.