2 problems

0 votes0 replies1 view
Conjecture on the interrelations and plateau timings of SGD sharpness metrics
Let stochastic gradient descent (SGD) denote the mini-batch optimization dynamics, and consider the metrics $\lambda_{\max}$ , Batch Sharpness, and Interaction-Aware Sharpness (IAS)…
Conjectureopenoptimization sharpness stochastic-gradient-descentarXiv posted 2 days ago
0 votes0 replies0 views
Conjecture on batch sharpness reaching the GNI threshold through gradient-noise variation
The conjecture. Batch Sharpness reaches $GNI$ at $2/\eta$ by means of increasing $\mathbb{E}[\lVert \mathcal{H}(L_B)-\mathcal{H}\rVert^2_2]$ more than the alignment of gradients wi…
Conjectureopenoptimization sharpness stochastic-gradient-descentarXiv posted 2 days ago

Conjecture on the interrelations and plateau timings of SGD sharpness metrics