Mini-course by Dario Trevisan at Tor Vergata, September 24-25
Dear all, We are glad to announce that next week RoMaDS <https://www.mat.uniroma2.it/~rds/events.php> will host at the University of Rome Tor Vergata the following mini-course: September 24-25 (10h00-12h00 and 13h30-15h30 on both days) Department of Mathematics, Aula Dal Passo Dario Trevisan (Università di Pisa) From Gaussian Limits to Feature Learning in Wide Neural Networks Abstract: Wide neural networks can be seen as random functions whose behavior simplifies in several different large-width regimes. At the scale of typical fluctuations, central-limit phenomena lead to Gaussian-process limits with recursively defined covariance kernels, while the parameter-space tangent geometry converges to the neural tangent kernel and yields a linearized description of training. These Gaussian and kernel limits provide a remarkably tractable theory of random initialization and learning, but they capture only part of the finite-width behavior. The course develops a probabilistic view of random wide neural networks across these different scales. We will study Gaussian and quantitative finite-width approximations, neural tangent kernel limits, large deviations of outputs and empirical layer kernels, and regimes in which feature learning survives at large width. Large-deviation theory provides a complementary description of rare, non-Gaussian behavior: atypical outputs can be realized through atypical internal kernels, leading to variational representations of the output rate function. Under suitable Bayesian scalings, the same viewpoint yields posterior variational problems over both predictors and kernels, providing one probabilistic mechanism for data-dependent feature learning. We will compare this with dynamical feature-learning limits such as mean-field and maximal-update regimes, emphasizing throughout how different scalings reveal different aspects of neural networks. We encourage in-person partecipation. Should you be unable to come, here is the link to the Teams streaming: Mini-corso Trevisan | Join meeting in Teams | Microsoft Teams <https://teams.microsoft.com/meet/35005768873253?p=IK9DCp8bgD0QL9P7PK> See you soon in Tor Vergata! This events are part of the Excellence Project MatMod@TOV.
participants (1)
-
Michele Salvi