3 problems

Let $H$ be the loss Hessian, with $H_{ii}$ its $(i,i)$-th entry, and let $g_i$ be the $i$-th component of the gradient. Gradient–Hessian alignment conjecture. When…

Near-optimality conjecture. Those negative eigenvalues are caused by the fact that the model is close to an optimum $w_*$, but not exactly at that point.

Let $T\geq2$, let $\theta^*$ be the unique minimum of $\mathrm{KL}(\mathrm{q}(\mathbf{x};\theta)\mathbin{\|}\mathrm{p}(\mathbf{x}))$, and define the local Hessian mismatch ratio ……