116 problems

Gradient accumulation uses large-batch samples for gradient evaluation. Gradient-accumulation generalization conjecture. Although gradient accumulation can help during optimization…

Countability necessity conjecture. Neither the countability of $I$ nor the countability of the languages can be dispensed with without additional hypotheses.

Let $Y$ be the output of the binary hypercube Gaussian-location channel, let $V$ denote its binary-hypercube input, and let $P_Y$ and $P_{V\mid Y}$ be the corresponding distributio…

Rate-optimality conjecture. The $\alpha$-dependent price in the convergence rate, namely $\mathcal{O}(\varepsilon^{-(4+\alpha)})$ for $\alpha\in(0,1)$ and…

General-distribution conjecture. Similar results hold for other data distributions.

Consider mini-batching with a batch size that remains of constant order as the problem scales. Let the batch size $1$ case be the reference setting, and let compute-optimal exponen…

The nonsmooth stochastic minimax setting considered here has objective complexity ${\mathcal{O}}(\varepsilon^{-7})$ in the extreme case $\theta=1$, under a restricted concavity con…

The SAG rate in … . Rate-comparison conjecture. This difference arises from the fact that the former is an in-expectation guarantee, while the latter is a high-probability bound. T…

Stochastic exp-concave optimization (SXO) concerns minimizing the population objective from stochastic gradient-oracle queries in dimension $d$, to accuracy $\epsilon$. Optimal-run…

Let the global objective function be the sum of the agents' local objective functions, and suppose that it satisfies the Polyak–Łojasiewicz (PL) inequality, without necessarily bei…

The stochastic online feedback optimization framework considers agents that may be non-compliant with commands issued by a central controller or multiple local controllers. In the…

Convergence of SVS-SPRING for LLQ. Under these assumptions and the SVS$(J,k,\lambda)$ sampling distribution, SPRING should satisfy the displayed convergence bound, where the hidden…

In stochastic minimax optimization, let $F(x,\xi)=\max_{y\in\mathcal{Y}(x,\xi)}\hat{F}(x,y,\xi)$, where $\mathcal{Y}(x,\xi)\subset\mathbb{R}^m$ is the feasible set of the adversari…

Logarithm-free last-iterate conjecture. It should be possible to eliminate the $ln(T)$ term from the best possible last-iterate convergence bound for convex smooth problems; in par…

EFS has a forward optimization step whose time complexity is quadratic in the number of training samples. Stochastic optimization conjecture. Stochastic optimization techniques cou…

Consider the nonconvex-component setting for incremental gradient descent, with random reshuffling (RR) as the permutation-based stochastic optimization method. Robust convergence…

Let a general first-order stochastic optimization algorithm optimize an objective under the assumptions associated with the stated SGD lower bound, and measure gradients by the…

Conjecture. The unsatisfactory performance of the PO methods comes from two aspects: it is generally hard to learn a global prediction model with high accuracy, and the optimizatio…

Smoothing conjecture. The convolution will reduce the role of $\epsilon(x)$, and the stochastic gradient descent step will be mainly induced by $\bar{f}(x)$.

subMIDAS convergence conjecture. The subsampling algorithm subMIDAS satisfies similar convergence properties to those described in Theorem, depending on the choice of $\lambda_n$,…

Let SG denote stochastic gradient and SGM denote stochastic gradient method with momentum. For deep neural networks (DNNs), consider objective functions, their ravines, the expecte…

The stochastic conjugate subgradient (SCS) algorithm is an online method for stochastic convex optimization, while stochastic gradient descent (SGD) methods are stochastic first-or…

An almost-supermartingale is a stochastic process satisfying the relevant approximate supermartingale conditions; the paper also considers the associated quantitative complexity of…

Consider AdaGrad applied to smooth convex optimization problems with stochastic gradient noise whose bounded $\alpha$-th moment satisfies $\alpha<2$, and let $b_t$ denote the adapt…

Stochastic optimization methods such as Adam and Clip-SGD use adaptive stepsizes, and Clip-SGD is known to converge in expectation when the gradient noise has a bounded $\alpha$-th…