Tags Abstract Vector Spaces1 Accelerated Methods1 AdaGrad2 Adaptive Algorithms1 Adaptive Optimization1 Affine Sets1 Arc Length1 Atlases1 Automatic Differentiation1 Banach Spaces1 Bandit Algorithms1 Basis Vectors1 Beltrami Identity1 Benchmarks1 Best Practices1 Binomial Coefficient1 Bra-Ket Notation1 Brachistochrone1 Bregman Divergence2 Calculus of Variations2 Calculus Rules1 Catenary1 Charts1 Cheat Sheet8 Cheatsheet1 Christoffel Symbols2 Completeness2 Complex Numbers1 Computational Cost1 Condition Number3 Cones1 Conjugate Gradient2 Connections2 Contraction1 Contravariance3 Contravariant2 Convex Functions1 Convex Optimization1 Convex Sets1 Convexity1 Coordinate Invariance1 Coordinate Transformations2 Covariance3 Covariant2 Covariant Derivative4 Covectors1 Crash Course29 Cross-Entropy1 Curvature3 Deep Learning1 Definitions1 Determinants1 Differential Geometry4 Diffusion Processes1 Discretization2 Distance1 Dual Connections1 Dual Numbers1 Dual Spaces2 Duality4 Eigenvalues1 Einstein Notation3 Entropy1 Epigraph1 Euler Method2 Euler's Formula1 Euler-Lagrange Equation7 Euler-Poisson Equation1 Fenchel Conjugate1 Finite Difference1 First-Order Convexity1 Fisher Information1 Fisher Information Metric3 Fixed Point Theorems1 Fokker–Planck Equation1 Formulas2 Forward Difference1 Fourier Analysis1 FTL1 FTRL5 Functional Analysis6 Functional Optimization1 Functionals4 Fundamental Lemma1 Generalization1 Generating Function1 Geodesics2 Gradient4 Gradient Descent2 Gradients2 Hahn-Banach Theorem1 Hamiltonian Mechanics3 Hessian4 Hilbert Spaces3 Hyperplanes1 Identities1 Imaginary Unit1 Information Geometry4 Information Theory2 Inner Product Spaces1 Integration by Parts1 Interior-Point Methods1 Isoperimetric Problems2 Iterative Methods3 Ito Calculus2 Ito Lemma2 Jacobian1 Jensen's Inequality1 KKT Conditions1 KL Divergence1 L2 Spaces1 Lagrange Multipliers1 Lagrangian1 Lagrangian Mechanics3 Legendre Transform3 Linear Algebra5 Linear Multistep1 Linear Programming1 Lp Spaces1 Machine Learning3 Machine Learning Optimization1 Manifolds2 Matrices1 Matrix Algebra1 Matrix Decompositions1 Matrix Norms2 Metric Tensor3 Mirror Descent4 MLE1 Multivariable Calculus3 Natural Gradient3 Newton's Method3 Non-convex Optimization1 Non-Euclidean Geometry1 Non-smooth Optimization1 Normed Spaces2 Norms1 Numerical Linear Algebra1 OCO8 ODE Solvers1 OGD1 OMD4 Online Gradient Descent1 Online Learning11 Online-to-Batch1 Optimal Control1 Optimality1 Optimality Conditions1 Optimization7 Optimization Algorithms1 Optimization Methods1 Optimization Problems1 Optimization Theory3 Ordinary Differential Equations2 Orthogonality2 Outer Product1 Parallel Transport1 Partial Derivatives3 Partial Differential Equations1 Physics1 Polyhedra1 Preconditioning3 Prerequisite1 Probability2 Probability Distributions2 Problem Formulation1 Proximal Algorithms1 Quadratic Programming1 Quasi-Newton Methods1 Quick Reference1 Random Variables1 Regret2 Regret Analysis1 Regret Minimization2 Regularization1 Reinforcement Learning1 Riemann Tensor1 Riemannian Metrics2 RMS Norm1 Runge-Kutta1 Score-Based Generative Modeling1 SDEs1 SDP1 Second-Order Convexity1 Separation Theorems1 Sequence1 Sequential Decision Making1 Slater's Condition1 Smooth Manifolds1 SOCP1 Spectral Theory1 Stability1 Statistical Manifolds2 Statistics2 Stiffness1 Stirling Number1 Stochastic Differential Equations2 Stochastic Gradient Descent1 Stochastic Optimization1 Stratonovich Calculus2 Subdifferential1 Subgradient1 Subgradient Method1 Sum1 Summary2 Supremum Norm1 SVD1 Symplectic1 Tangent Spaces1 Taylor Series3 Tensor Algebra2 Tensor Calculus2 Tensors4 Theorems1 Time-Reversal1 Transformations1 Variational Calculus7 Variations1 Vector Algebra1 Vector Norms1 Vectors1