1 problem

0 votes0 replies1 view
The scheduling conjecture for reducing signSGD noise
A learning-rate schedule has the form … for a suitable function $f$ , and the associated signSGD noise term is the contribution to the loss arising from stochastic gradient noise. S…
Conjectureopenlearning-rate-schedules optimization signsgdarXiv posted 3 days ago

The scheduling conjecture for reducing signSGD noise