2 problems

Let $\beta_2$ denote Adam's second-moment decay parameter, and let the scaling law with signSGD refer to the compute-scaling exponents identified in the paper's signSGD analysis. A…

Adam is an optimization algorithm whose behavior depends on its hyperparameters $(\beta_1,\beta_2)$. In convergence analyses, one may choose a problem after fixing these hyperparam…