Cari tutti,
giovedì 4 aprile, alle ore 14:30, presso la Sala Riunioni del Dipartimento di Matematica (primo piano) dell'Università degli studi di Pisa si terrà il seguente seminario:
(Dear all, on Thursday, April 4th, 2:30 PM, the following talk will take place in Sala Riunioni, Dipartimento di Matematica, University of Pisa:)
SPEAKER: Giacomo De Palma (University of Copenhagen)
TITLE: Random deep neural networks are biased towards simple functions
ABSTRACT: We prove that the binary classifiers of bit strings generated by random wide deep neural networks are biased towards simple functions. The simplicity is captured by the following two properties. For any given input bit string, the average Hamming distance of the closest input bit string with a different classification is at least sqrt(n/(2πlog(n))), where n is the length of the string. Moreover, if the bits of the initial string are flipped randomly, the average number of flips required to change the classification grows linearly with n. On the contrary, for a uniformly random binary classifier, the average Hamming distance of the closest input bit string with a different classification is one, and the average number of random flips required to change the classification is two. Our proof is based on the recent breakthrough in machine learning stating that random deep neural networks behave as Gaussian processes. Our results are confirmed by numerical experiments on deep ne ural networks with two hidden layers, and settle the conjecture stating that random deep neural networks are biased towards simple functions. The conjecture that random deep neural networks are biased towards simple functions was proposed and numerically explored in [Valle Pérez et al., arXiv:1805.08522] to explain the unreasonably good generalization properties of deep learning algorithms. By providing a precise characterization of the form of this bias towards simplicity, our results open the way to a rigorous proof of the generalization properties of deep learning algorithms in real-world scenarios.