Deep Learning

Slides

• Gabriel Peyré (CNRS and ENS): Parametric Model Fitting with Automatic Differentiation.
• Alexandre Allauzen (Paris-Sud): Deep-Learning : the basics.
• Guillaume Charpiat, (INRIA): Part 1 (Which Architectures, for which Purposes), Part 2 (Gradient descents and inner products).

Computational resources

Autodiff and more specific deep-learning softwares (Python):

• AutoGrad: directly differentiate basic python codes.
• Chainer: similar purpose.
• Theano: more advanced (symbolic computation).
• TensorFlow: similar.
• PyTorch: similar.
• Keras: deep learning library (on top of Theano or TensorFlow).
• Natural gradient in Torch by Gaetan Marceau.

Bibliography

Here is a (biased) bibliography related to the lectures:

• Deep learning, Ian Goodfellow and Yoshua Bengio and Aaron Courville.
• Reference article on natural gradient: Riemannian metrics for neural networks I: feedforward networks, Yann Ollivier, Information and Inference.
• Another article: Practical Riemannian Neural Networks, Gaétan Marceau-Caron and Yann Ollivier.