https://jiha-kim.github.io/Ji-Ha Kim's blog for sharing things. 2026-04-11T01:11:46+00:00 Ji-Ha Kim https://jiha-kim.github.io/ Jekyll © 2026 Ji-Ha Kim /assets/img/favicons/favicon.ico /assets/img/favicons/favicon-96x96.png 2026-04-05T00:00:00+00:00 2026-04-10T12:41:38+00:00 https://jiha-kim.github.io/posts/rational-polar-decomposition/ jiha

A hardware-aware hybrid polar decomposition for ML: one Dynamic Weighted Halley (rational) step to handle the hard early regime, then two Polar Express (polynomial) cleanup steps once the spectrum is easy. The result is exactly two rectangular GEMMs, no eigendecomposition or power iteration, and robust convergence from condition numbers up to 1000.

2026-03-31T19:00:00+00:00 2026-03-31T20:49:40+00:00 https://jiha-kim.github.io/posts/second-order-residual-networks/ jiha

Residual networks are first-order depth dynamics that can only produce exponential modes. Upgrading to second-order by adding a velocity state unlocks bounded oscillatory modes, separates content from transport, and connects depth to the same mathematical framework as LSTMs and state-space models.

2026-03-23T11:00:00+00:00 2026-03-27T23:58:31+00:00 https://jiha-kim.github.io/posts/lion-k-ccwd/ jiha

Derivation of the Lion-K optimizer with Corrected Cautious Weight Decay (CCWD) and transformation rules for hyperparameter transfer.

2026-03-18T23:38:00+00:00 2026-03-19T17:00:46+00:00 https://jiha-kim.github.io/posts/transformers-as-constrained-optimization/ jiha

Rewriting a pre-norm decoder-only transformer as a mixed-geometry constrained splitting scheme: RMSNorm as radial gauge fixing, attention as an entropy- or KL-constrained simplex solve, and residual branches as Euclidean trust-region steps.

2026-03-15T22:59:00+00:00 2026-03-16T01:49:49+00:00 https://jiha-kim.github.io/posts/optimizers-and-odes/ jiha

A continuous-time view of gradient-based optimization: starting from the observation that integrator choice matters in physics simulation, and transferring that insight to understand modern optimizers.