Convex Optimization

Slides

• Gabriel Peyré (CNRS and ENS): Convex Optimization.
• Alexandre Gramfort (INRIA): A practical tour of optimization algorithms for the Lasso, (additional material).
• Matthieu Kowalski (Paris-Sud): *-Lasso Therapy: a sparse synthesis approach, (additional material).

Computational resources

• Scikit-Learn: machine learning in Python, contains implementations of many basic proximal splitting methods.
• The numerical Tours of Signal Processing (in Matlab, Python and Julia) contains a section dedicated to convex optimization, with applications.
• CVX: interface for several conic program for small/medium size linear/quadratic/psd optimization problems. Also in Python.
• TFOCS: similar to CVX, but using first order (proximal) splitting scheme instead of interior point methods.
• SPAMS: state of the art sparse solver, including homotopy/LARS.

Bibliography

Here is a (biased) bibliography related to the lectures.

• Convex Optimization, Boyd and Vandenberghe: basics of convex programming, including interior point methods.
• Convex Analysis and Monotone Operator Theory in Hilbert Spaces, Heinz H. Bauschke, Patrick L. Combettes: the definitive theory-oriented book on proximal methods.
• Optimization with Sparsity-Inducing Penalties, Francis Bach, Rodolphe Jenatton, Julien Mairal and Guillaume Obozinski: a review of proximal splitting schemes for sparsity enforcing regularizations.
• Proximal Algorithms, N. Parikh and S. Boyd: another review article (see also this review for ADMM/DR methods).