14 problems

Let $X=UV^\top$ be a full-dimensional Burer--Monteiro factorization, with gradient descent applied using sufficiently small step sizes and initialization sufficiently close to the…

The hypothesis space consists of functions represented by the single-hidden-layer equivariant networks studied in the paper, and norm-bounded functions form a specified subset of t…

Adam's $\ell_\infty$-geometry conjecture. $\mathtt{Adam}$ outperforms $\mathtt{GD}$ due to its utilization of $\ell_\infty$ geometry, under which the loss function could have bette…

Consider deep models with depth parameter $h>2$. Collapse conjecture. We conjecture that such deep models cannot be "collapsed". The source gives only a heuristic argument and no f…

Consider the radially symmetric setting studied for the network interpolation problem, and let a minimum norm interpolant be a function that fits the prescribed data while minimizi…

The setting is a radially symmetric interpolation problem for a ReLU network, with minimum norm interpolants sought among functions fitting the prescribed data. In the one-dimensio…

Consider neural-network training objectives with good or bad regularity, trained using sufficiently large learning rates. The relevant large-learning-rate phenomena include edge of…

Consider the function class analyzed in the paper, whose global minimizers are fixed at $xy=1$, with the parameter restriction that excludes even values of the relevant exponent. E…

Let a function be studied through large-learning-rate phenomena such as edge of stability and balancing, and distinguish global regularity properties from regularity observed only…

Consider objective functions trained by gradient descent with large learning rates, and measure their regularity by an appropriate regularity notion. The phenomena of interest incl…

Large learning rates are considered in the context of gradient-based training, where a flat minimum is a minimum associated with favorable generalization properties. Flat-minima co…

Let $\beta$ and $\rho$ denote the momentum and second-moment decay parameters, respectively, in Adam. Consider a stable regime of training, and let the test error be the resulting…

Let $G$ be a matrix-factorization parametrization, such as $G(U)=UU^\top$ or $G(U,V)=UV^\top$, and distinguish diagonal measurements, whose coordinate parametrizations commute, fro…

In the setting where training and test data points are drawn from the same distribution, the most salient features are predictive in both sets. Gradient Starvation's overfitting co…