3 problems

The summarized scaling law theorem concerns the scaling behavior established earlier in the paper for the considered momentum algorithms and their dimension-dependent hyperparamete…

Let $\nu$ be the momentum parameter in quasi-hyperbolic momentum (QHM), and let $\alpha$ and $\beta$ denote its learning-rate and momentum parameters. For each fixed $\nu$, choose…

Let NAG denote Nesterov's accelerated gradient method in the stochastic optimization setting considered above. Lower-bound conjecture for NAG. A lower bound for NAG can be establis…