Dear all, It is not time to go to the seaside yet! On Tuesday, June 30th, at 14h00 in Aula Dal Passo in the Math Department of the University of Rome Tor Vergata, RoMaDS will host Alberto Bietti (Flatiron Institute, NYC), with the mini-mini-course (about 3 hours long) “Learning in Transformers with Associative Memories” Abstract: Large language models based on transformers have achieved great empirical successes. However, as they are deployed more widely, there is a growing need to better understand their internal mechanisms in order to make them more reliable. These models appear to store vast amounts of knowledge from their training data, and to adapt quickly to new information provided in their context or prompt. Through toy tasks for reasoning and factual recall, we highlight the role of weight matrices as associative memories, and provide theoretical results on how gradients enable their learning during training, how over-parameterization affects their storage capacity, and sample complexity. We encourage in-person partecipation. Should you be unable to come, here is the link to the Teams streaming: https://teams.microsoft.com/meet/366597487320635?p=lk1ffPDon4Quh9ygMt The seminar is part of the Excellence Project MatMod@TOV.