3 problems

Consider mean-squared-error (MSE) training of a neural network with depth denoted by $\operatorname{depth}$, learning rate $\eta$, and third derivative of the loss denoted by…

Let mini-batch SGD without replacement be compared with an algorithm whose steps are centered and independent, such as full-batch gradient descent or SGD with replacement. The depe…

The paper studies nonconvex, nonsmooth optimization through oracle complexity. In its smoothing results, the algorithm is assumed to be translation invariant with respect to consta…