Numerical Algorithms for Optimization

Syllabus

These classes are intended to help acquire a technical toolbox for Machine Learning and Deep Learning for example. They explain what happens under the hood when applying certain Machine Learning algorithms that rely on numerical optimization.

• A short introduction to Neural Networks
• Linear Least Squares: a method relying on Cholesky’s decomposition / on the QR decomposition
• The one dimensional Newton method for equation solving and minimization
• Solving equations of multiple variables with the Newton methods
• Unconstrained Optimization in $\mathbb R^d$ with the Newton method and gradient descent
• Constrained Optimization
• Automated Differentiation