7 problems

Learning-rate-annealing conjecture. The mismatch between similar loss values and generative quality under ScheduleFree is partially due to missing learning-rate annealing: adding a…

In the quadratic case, large fixed step-sizes achieve optimal convergence rates. Asymptotic quadratic-behavior conjecture. The success of large learning rates may be attributed to…

A Gibbs posterior uses a learning rate to control the contribution of the loss relative to the prior; in the calibration approach of Syring and Martin, the learning rate is obtaine…

Learning-rate trade-off conjecture. Even in much broader settings, the choice of learning rate $gamma$ dictates a trade-off between accuracy and initial convergence.

Learning-rate relaxation conjecture. This condition can be relaxed. The question concerns whether convergence guarantees under a bounded variance assumption can hold with a larger…

The algorithm of Allen-Zhu works with iterate averaging, rather than the final iterate, and uses an exponentially decaying learning-rate scheme. Final-iterate convergence conjectur…

Adaptive learning-rate conjecture. Choosing the learning rate adaptively according to the size of the current gradient might improve the convergence requirement to