The DeepRiemann project aims at the design and analysis of novel training algorithms for Neural Networks in Deep Learning, by applying notions of Riemannian optimization and differential geometry. The task of the training a Neural Network is studied by employing tools from Optimization over Manifolds and Information Geometry, by casting the learning process to an optimization problem defined over a statistical manifold, i.e., a set of probability distributions. The project is highly interdisciplinary, with competences spanning from Machine Learning to Optimization, Deep Learning, Statistics, and Differential Geometry. The objectives of the project are multiple and include both theoretical and applied research, together with industrial activities oriented to transfer knowledge, from the institute to a startup or spin-off of the research group.
The positions will be part of the new Machine Learning and Optimization group, which will be performing research at the intersection of Machine Learning, Stochastic Optimization, Deep Learning, and Optimization over Manifolds, including nature-inspired optimization algorithms, from the unifying perspective of Information Geometry. The group is one of two newly-formed groups in Machine Learning at RIST, where about 20 new postdoctoral research associates and research software developers will be hired in the next year.
The official job announcement can be seen here:
# Jobs Description #
The three open positions will focus on different and overlapping aspects of the project:
1) Optimization Algorithms over Statistical Manifolds with Applications to Deep Learning
The postdoctoral researcher will conduct research on the design and implementation of novel first- and second-order methods for the optimization of functions defined over statistical models, such as exponential families, the Gaussian distribution, and Markov Random Fields. In particular the research will focus on the optimization problem associated to the training of a Neural Network, which will be studied from the perspective of Optimization over Manifolds with non-Euclidean geometries. The postdoctoral candidate will develop optimization algorithms which explicitly take into account the Riemannian and dual-affine Hessian geometries of the search space, given by the Fisher information metric. Such algorithms should be efficient, scalable, largely parallelizable, and suitable for the large scale and big data settings.
Keywords: stochastic gradient descent, Riemannian manifolds, dually-flat Hessian geometries, optimization over manifolds, second-order optimization methods, large scale optimization, nature-inspired optimization algorithms
2) Theory of Neural Networks
The postdoctoral researcher will contribute towards the definition of a probabilistic and geometric framework for the study of deep Neural Networks aimed at a better understanding of the working mechanisms behind the success of Deep Learning. The analysis is focused towards a better understanding of what enables Deep Learning systems to achieve remarkable successes, and of its limits, in classification and predictive analysis, with a particular focus on the learning task, in order to guide the development of novel algorithms. The research will include the study of the representational power of various Neural Networks topologies, by taking into consideration the impact of hierarchical architectures with respect to the expressiveness of the network; and the analysis of the number of local minima and of saddle points, which appear during the training and which strongly affect convergence of the optimization algorithms, depending on the nature of the network.
Keywords: theory of neural networks, deep learning, learning representations, expressive power of neural networks.
3) Information Geometry of dually-flat Hessian Manifolds
The postdoctoral researcher will focus on the characterization of first and second-order Riemannian and affine geometries of statistical models, or more in general Hessian manifolds, aimed at the study of optimization methods over statistical manifolds. Neural Networks can be modeled as statistical manifolds, or manifolds of Neural Networks, and the training of the network can be studied as an optimization problem over a non-linear manifold, where the weights of the network are the parameters of the model and thus the variables of the function to be optimized. The purpose of the analysis is to introduce alternative second-order approximations of a function over a statistical manifold, depending on the choice of the connection and thus how parallel transport is implemented, which are at the basis of the design of second-order optimization methods.
Keywords: information geometry, geometry of statistical models, natural gradient, dually-flat manifolds, Hessian geometries.
# Desired Qualifications #
- PhD in machine learning, theoretical computer science, stochastic optimization, statistics, applied mathematics, including fields such as manifold optimization, differential geometry, statistical mechanics, and related fields
- Strong publication record
- Strong analytical skills, such as problem solving and logical thinking
- Enthusiasm to work in a multidisciplinary and international research environment
- Good written and oral communication skills in English
Knowledge in machine learning is helpful, but not strictly mandatory. Doctoral students close to the competition of their thesis will also be considered.
The positions are to start as early as December 2016 or at any agreed later date. Applications will be reviewed as they are received.
RIST offers competitive salaries and top-level working conditions. The net salary for these positions will be around 2.000 euro per month, with a possible increase up to 2.200 euro, based on new tax incentives for research and development activities in Romania. Positions are endowed with travel resources. The cost of living in Cluj is significantly lower than in Western Europe or the USA (e.g., it is 1/3 of the cost of living in London, UK).
# How to Apply #
In order to apply to this position, the candidate should send an email to
deepriemann.jobs@rist.ro, mentioning in the subject of the email "DeepRiemann Postdoc Application". The email should include among the attachments:
- A cover letter
- A complete CV with full list of publications
- A short research statement (max 3 pages), which describes your research interests and explain why your skills, knowledge and experience makes you a suitable candidate for one or more of the open positions
- The pdfs of 2 selected publications
- Name and contact information of up to 3 referees, which will be contacted directly by the institute for a reference letter
Informal inquiries can be sent to Dr. Luigi Malagò <
malago@rist.ro>, Principal Investigator of the DeepRiemann project.
# About the Institute #
The Romanian Institute of Science and Technology is a non-governmental, not-for-profit, independent research institute, founded in 2009, with the purpose of offering scientists a place to conduct research in Romania with top working conditions, comparable to those you can find in Western Europe. RIST currently performs research on computational and experimental neuroscience, computational intelligence, machine learning, and dynamical systems. The institute is currently undergoing a phase of significant and sustained growth supported by European and Romanian structural funds, with plans for hiring around 20 new scientists and researchers in the next year. The institute has close contacts with the Babeș-Bolyai University, which is the largest university in Romania, and with the Technical University of Cluj-Napoca.
The institute is located in Cluj-Napoca, in the heart of Transylvania, which has been named by Lonely Planet as the top region to visit in 2016. Cluj is a welcoming and innovative city, recently listed as one of the major tech hubs (with a 9% growth per year for IT industry). Cluj has 12 universities, over 70k students every year, and an extremely vibrant startup scene, which has recently named the Silicon Valley of Transylvania.
Cluj is Europe's friendliest city for foreigners, according to a study by the UK Office of National Statistics, and Romania is no. 16 in the Internations best expat destinations of 2016. About 1100 French citizens, 800 Italian citizens and 500 German citizens live in Cluj (source). The city has an international airport, only 8km away from the city center, with flights to more than 30 European destinations.
More information about Cluj: