54 problems

Higher-moment rate conjecture. The rate $O(\sqrt{\eta_n})$ can be improved by assuming that $\xi^i_k$ has a high-order moment.

Yun–Sra–Jadbabaie conjecture. There exists a constant $\eta_{n,K}\in(0,1]$ such that, whenever $(1-\eta_{n,K})I\preceq A_i\preceq I$ for every $i$, one has

Jain et al.'s anytime optimality conjecture. In the absence of a priori information about $T$, no stepsize sequence can ensure the information-theoretically optimal error rate for…

Bounded-gains conjecture. Lookahead can provide modest but bounded gains in more general settings.

Let $f:cmathbb{R}ctocmathbb{R}$ be convex and of class $C^h(cmathbb{R})$ for an even integer $h>0$, with unique minimizer $x^*$, satisfying $\lim_{x\to x^*}f^{(k)}(x)=0$ for…

The setting is the two-class stochastic-gradient-descent model discussed above, with power-law covariance and mean parameters whose exponents include $\alpha$; the relevant transit…

Logarithm-free last-iterate conjecture. It should be possible to eliminate the $ln(T)$ term from the best possible last-iterate convergence bound for convex smooth problems; in par…

The discussion concerns sieve stochastic gradient descent estimators with tuning parameters $a$, $\tau_1$, and $\tau_2$, where the current stability result requires a condition of…

Lyapunov-framework conjecture. Within this Lyapunov framework, it is not possible to obtain a bound for SGD with a bias term of order

The summarized scaling law theorem concerns the scaling behavior established earlier in the paper for the considered momentum algorithms and their dimension-dependent hyperparamete…

Consider stochastic gradient descent with exponential moving average, and let the bias error be decomposed into components associated with the eigenspaces of the data-feature covar…

Let $L$ have local minima $x_1,\dots,x_M$, and let $u_i$ be the solution of the localized equation around the local minimum $x_i$, for $1\leq i\leq M$. Let $\rho(t,x)$ denote the s…

Let stochastic gradient descent (SGD) denote the mini-batch optimization dynamics, and consider the metrics $\lambda_{\max}$, Batch Sharpness, and Interaction-Aware Sharpness (IAS)…

The conjecture. Batch Sharpness reaches $GNI$ at $2/\eta$ by means of increasing $\mathbb{E}[\lVert \mathcal{H}(L_B)-\mathcal{H}\rVert^2_2]$ more than the alignment of gradients wi…

The hypothesis space consists of functions represented by the single-hidden-layer equivariant networks studied in the paper, and norm-bounded functions form a specified subset of t…

The framework uses exponentially increasing weights, corresponding to a constant $c_t=\gamma$, and these weights achieve optimal worst-case convergence guarantees. Instance-depende…

Let $f^* \coloneqq \sum_{S \in \mathcal{S}}\alpha_S\chi_S$ be a Boolean function, and let its leap complexity be the minimum value $l$ for which there is an ordering of its coeffic…

Computational-cost conjecture. More computational costs are essentially required under the regime $a>2b$.

In an infinite-dimensional linear regression model, suppose only $M$-dimensional sketched covariates are observed, a linear predictor with $M$ trainable parameters is trained by on…

The stochastic-gradient methods and their SDE approximations are considered under the smoothness and convexity assumptions stated for the weak approximation theorem above. In parti…

Deterministic-equivalent conjecture. For all admissible $v$ and $d$, with probability tending to $1$ as $d\to\infty$,

Let mini-batch SGD without replacement be compared with an algorithm whose steps are centered and independent, such as full-batch gradient descent or SGD with replacement. The depe…

The setting concerns stochastic gradient descent (SGD) without replacement, in which the data batches used at successive steps are dependent because they are disjoint, and saddle e…

Let $f(\mathbf{x};\mathbf{W}^{(t)})$ be the model output at iteration $t$, let $y$ be the label, and let $\bar{T}_k$ and \underaccent{\bar}{T}_k denote the corresponding stopping…

Let $p\in [1,+\infty[$, and define the sliced Wasserstein energy by … Here $w_\theta^{(p)}(X,Y)$ is the one-dimensional projected Wasserstein loss between $X$ and $Y$, and path dif…