Tags Abstract Vector Spaces1 AdaGrad1 Adam2 Adam Optimizer1 Adaptive Learning Rates1 Algebra1 Algebraic Methods1 Automatic Differentiation1 Batch Learning1 Beltrami Identity1 Boolean Algebra1 Bra-Ket Notation2 Brachistochrone1 Bregman Divergence1 Calculus of Variations2 Catenary1 Cheat Sheet2 Cheatsheet1 Combinatorics1 Condition Number2 Conditional Expectation1 Conditional Probability1 Conjugate Gradient2 Contravariance2 Convergence1 Convexity1 Coordinate Invariance1 Covariance2 Crash Course6 Deep Learning1 Determinants1 Diffusion Processes1 Discretization2 Dual Numbers1 Dual Spaces2 Duality1 Efficiency1 Eigenvalues1 Euler Method2 Euler-Lagrange Equation7 Euler-Poisson Equation1 Expectation2 Exponential Moving Average1 Fokker–Planck Equation1 Formulas1 Foundations1 Francesco Orabona1 FTRL1 Functional Analysis2 Functional Optimization1 Functionals4 Fundamental Lemma1 Generalization1 Generating Functions1 Gradient3 Gradient Descent2 Gradient Flow1 Gradient-Based Optimization1 Gradient-Free Optimization1 Gradients2 Hamiltonian Mechanics3 Hessian3 Hessian Spectrum1 Hilbert Spaces2 Identities1 Incremental Learning1 Indicator Functions1 Integration by Parts1 Intuition2 Invariance1 Isoperimetric Problems2 Iterative Methods3 Ito Calculus2 Ito Lemma2 Kolmogorov Axioms1 Lagrange Multipliers1 Lagrangian Mechanics3 Legendre Transform3 Linear Algebra3 Linear Multi-step Methods1 Linear Regression1 Loss Function1 Loss Functions1 Matrices1 Matrix Algebra1 Matrix Decompositions1 Measure Theory1 Momentum2 Multivariable Calculus3 Nesterov Accelerated Gradient1 Newton's Method1 Non-convex Optimization1 Non-Convex Optimization1 Normed Spaces2 Objective Function1 ODE Discretization1 Online Learning2 Optimal Control1 Optimization7 Optimization Algorithms2 Optimization Problems1 Optimization Theory4 Optimizer Properties1 Optimizers1 Ordinary Differential Equations2 Orthogonality1 Parameters1 Partial Derivatives3 Partial Differential Equations1 Physics2 Polyak Heavy Ball1 Preconditioning2 Probability Theory1 Quasi-Newton Methods1 Quick Reference1 Random Matrix Theory1 Random Variables1 RMSProp2 Robustness1 Saddle Points1 Score-Based Generative Modeling1 Set Theory1 SGD1 Spectral Theory2 Stability1 Stochastic Differential Equations2 Stratonovich Calculus2 SVD1 Taylor Series3 Theorems1 Time-Reversal1 Transformations1 Variational Calculus7 Variations1 Vector Algebra1 Vectors1