Advanced non-smooth optimization

Slides

• Gabriel Peyré, CNRS and ENS, First Order Splitting Schemes.
• Jalal Fadili, ENSICaen, Primal Dual Methods.

Computational resources

• Scikit-Learn: machine learning in Python, contains implementations of many basic proximal splitting methods.
• The numerical Tours of Signal Processing (in Matlab, Python and Julia) contains a section dedicated to convex optimization, with applications.
• CVX: interface for several conic program for small/medium size linear/quadratic/psd optimization problems. Also in Python.
• TFOCS: similar to CVX, but using first order (proximal) splitting scheme instead of interior point methods.
• SPAMS: state of the art sparse solver, including homotopy/LARS.

Bibliography

Here is a (biased) bibliography related to the lectures.

• Convex Optimization, Boyd and Vandenberghe: basics of convex programming, including interior point methods.
• Convex Analysis and Monotone Operator Theory in Hilbert Spaces, Heinz H. Bauschke, Patrick L. Combettes: the definitive theory-oriented book on proximal methods.
• Optimization with Sparsity-Inducing Penalties, Francis Bach, Rodolphe Jenatton, Julien Mairal and Guillaume Obozinski: a review of proximal splitting schemes for sparsity enforcing regularizations.
• Proximal Algorithms, N. Parikh and S. Boyd: another review article (see also this review for ADMM/DR methods).