Study of the generalization abilities of affine neural networks used in regression tasks trained with gradient flow

Details

Supervisors

Legat, Benoît ; Jacques, Laurent

Faculty

Ecole polytechnique de Louvain

Degree label

Master [120] : ingénieur civil en science des données, à finalité spécialisée: analyse de données

Abstract

Multi-layer perceptrons and other neural networks have been used very successfully in many supervised learning tasks. These performances remains largely unexplained to this day. There exist a large literature who study the properties of linear neural networks, a more simplified form of multi-layer perceptrons, to discover insights on the reasons of the performances of multi-layer perceptrons. Building on these works, the author selects one paper in this literature and analyses it through the length of a concept called implicit regularisation. He then introduces affine neural networks, a new type of neural network which is an intermediate step in terms of complexity between linear neural networks and multi-layer perceptrons. He shows how the analysis of the generalisation of the properties of linear neural networks to affine neural networks gives new insights on multi-layer perceptrons that linear neural networks do not give.