Cheat Sheet: Variational Calculus for Optimization
A concise summary of key concepts, equations, and examples from the Variational Calculus crash course, including functionals, the Euler-Lagrange equation, Lagrangian/Hamiltonian mechanics, and the Legendre transform.
This post serves as a concise cheat sheet for the “Variational Calculus for Optimization” crash course. It summarizes the key definitions, equations, and examples discussed throughout the series.
1. Core Concepts
| Concept | Description | Notation Example | | :———————————————- | :—————————————————————————————————————————— | :————————————————————————— | | Functional | A mapping from a set of functions to the real numbers. |
or
1
| | **Variation** | A small, arbitrary perturbation to a function,
. |
1
| | **First Variation** | The principal linear part of the change in a functional due to a variation. Necessary condition for extremum:
. |
| | Admissible Variation | A variation
that respects the boundary conditions of the problem (e.g.,
for fixed endpoints). | | | Fundamental Lemma of Calculus of Variations | If
for all admissible
, then
for all
. | |
2. The Euler-Lagrange Equation (and its forms)
The Euler-Lagrange equation provides a necessary condition for a function to extremize a functional. For a general functional whose integrand is
:
| Type of Functional / Condition | Integrand
1
| Euler-Lagrange Equation(s) | Resulting Equation Type | | :--------------------------------------------------------------------------------- | :------------------------------------------------------------------ | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | :---------------------- | | **Standard Case** <br/> (1 dep. var
, 1st order deriv.
) |
1
|
1
| ODE (2nd order) | | **Special Case:
independent of
**
(
) |
1
| **Beltrami Identity:**
(constant) | ODE (1st order) | | **Special Case:
independent of
**
(
) |
1
|
(constant) | ODE (1st order) |
| Higher-Order Derivatives
(up to
) |
1
|
(Euler-Poisson Equation) | ODE (2n-th order) |
| Multiple Dependent Variables
(
,
) |
1
|
, for each
1
| System of ODEs | | **Multiple Independent Variables** <br/> (e.g.,
) |
(
, etc.) |
1
| PDE |
3. Lagrangian and Hamiltonian Mechanics
A prime application of variational calculus in physics.
| Concept | Definition / Key Equation | Variables Used | Notes | | :——————————- | :————————————————————————————————————– | :—————- | :————————————————————————— | | **Action
** |
1
|
| Principle of Stationary Action:
yields equations of motion. | | **Lagrangian
** |
(Kinetic Energy
minus Potential Energy
for classical systems) |
| The integrand of the action. | | **Generalized Momentum
** |
1
|
| Momentum conjugate to the generalized coordinate
. | | **Hamiltonian
** |
(Legendre transform of
w.r.t.
) |
1
| Often represents the total energy (
) of the system. | | Hamilton’s Equations |
,
1
|
1
| A set of first-order ODEs describing motion. |
4. Legendre Transform
A mathematical tool for changing variables, crucial for moving from Lagrangian to Hamiltonian mechanics and foundational for convex duality.
- Purpose: Transforms a function
into a new function
where the new variable
is related to the derivative of
.
- Definition: Given
and defining
, the Legendre transform is:
1
2
3
4
5
<div class="math-block" markdown="0"> \[ f^\ast(p) = px - f(x) \]
</div>
where
on the right-hand side must be expressed as a function of
by inverting
.
- Invertibility Requirement: For the inversion
to be well-defined,
must be monotonic, which is guaranteed if
is strictly convex (
) or strictly concave (
). Standardly, strict convexity is assumed.
- Symmetric Derivative Property: If
, then the original variable is recovered by
.
- Relationship between Hessians: The Hessian of
with respect to
is the inverse of the Hessian of
with respect to
:
1
2
3
4
5
6
7
8
9
<div class="math-block" markdown="0"> \[ \left( \frac{\partial^2 f^\ast}{\partial p^2} \right) = \left( \frac{\partial^2 f}{\partial x^2} \right)^{-1} \]
</div>
This implies that convexity is preserved under the Legendre transform. - **Connection to Convex Conjugate (Legendre-Fenchel Transform):**
<div class="math-block" markdown="0"> \[ f^\ast(p) = \sup_x (px - f(x)) \]
</div>
5. Constrained Variational Problems (Isoperimetric Problems)
Problems where we extremize a functional
subject to an integral constraint
(a constant).
- Method of Lagrange Multipliers:
- Form an auxiliary functional
using a Lagrange multiplier
:
1
2
3
4
5
<div class="math-block" markdown="0"> \[ J^\ast[y] = J[y] + \lambda K[y] = \int_a^b (F(x, y, y') + \lambda G(x, y, y')) \, dx = \int_a^b H_\lambda(x, y, y', \lambda) \, dx \]
</div>
where
is the integrand of
,
is the integrand of
, and
. 2. Apply the Euler-Lagrange equation to the new integrand
(treating
as a constant for this step):
1
2
3
4
5
<div class="math-block" markdown="0"> \[ \frac{\partial H_\lambda}{\partial y} - \frac{d}{dx}\left(\frac{\partial H_\lambda}{\partial y'}\right) = 0 \]
</div>
3. Solve the resulting differential equation. The solution
will depend on
. 4. Substitute this solution back into the original constraint equation
to determine the value of the Lagrange multiplier
.
6. Classic Problems and Their Solutions
| Problem | Functional’s Integrand
(or similar) | Solution Curve / Shape | | :———————————————— | :—————————————————————— | :——————————- | | Shortest Path (in a plane) |
1
| Straight Line | | **Brachistochrone** (fastest descent) |
1
| Cycloid | | **Catenary** (hanging chain, min. PE) |
(integrand proportional to this) | Catenary (
) | | Minimal Surface (e.g., soap film) |
(for a surface
) | Surface with zero mean curvature | | Dido’s Problem (max area for fixed perimeter) | Maximize
subject to
| Circular Arc |
This cheat sheet provides a quick reference to the fundamental tools and results of variational calculus discussed in this series. For detailed derivations and explanations, please refer to the individual posts.