Dear all, you are all invited to participate to the following seminar.
Speaker : Domenico Marinucci (https://sites.google.com/view/domenicomarinucci/home) Affiliation: Dipartimento di Matematica - Università Roma Tor Vergata Title: Spectral complexity of deep neural networks Date: Monday, May 29, 2024 at 11.30 Place: Aula 704 (7th floor) , Dipartimento di Matematica at University of Genova, Via Dodencaneso 35,
Abstract: It is well-known that randomly initialized, push-forward, fully-connected neural networks weakly converge to isotropic Gaussian processes, in the limit where the width of all layers goes to infinity. In this paper, we propose to use the angular power spectrum of the limiting fields to characterize the complexity of the network architecture. In particular, we define sequences of random variables associated with the angular power spectrum, and provide a full characterization of the network complexity in terms of the asymptotic distribution of these sequences as the depth diverges. On this basis, we classify neural networks as low-disorder, sparse, or high-disorder; we show how this classification highlights a number of distinct features for standard activation functions, and in particular, sparsity properties of ReLU networks. Our theoretical results are also validated by numerical simulations.
Joint work with Simmaco Di Lillo, Michele Salvi e Stefano Vigogna
Best wishes,
Ernesto
--------------------- Ernesto De Vito DIMA - Dipartimento di Matematica MaLGa - Machine Learning Genoa center Via Dodecaneso 35 16146 Genova Italy
e-mail: ernesto.devito@unige.it tel: +390103536783