A two-part crash course covering numerical methods for ODEs (relevant to gradient flow and momentum) and numerical linear algebra (for conditioning, iterative solvers, and preconditioning) essential for modern optimization.
A crash course on tensor calculus, focusing on definitions, notation, and operations essential for understanding advanced machine learning and optimization techniques in high-dimensional spaces.
An introduction to the core concepts of functional analysis essential for understanding optimization theory in machine learning.
A crash course starting from the geometric viewpoint
A concise review of essential multivariable calculus concepts vital for understanding mathematical optimization, including partial derivatives, gradients, Hessians, and Taylor series.
A crash course on the calculus of functionals, exploring how to optimize entire functions. Covers the Euler-Lagrange equation, connections to physics (Lagrangian/Hamiltonian), the Legendre transform, and applications to classic optimization...