Strengthened minimax optimal constant-stepsize conjecture for gradient descent

About 2 years old · traced to

Let FL,D\mathcal{F}_{L,D} be the class of pairs (f,x0)(f,x_0) where ff is LL-smooth and convex, has a minimizer x⋆x_\star with ∥x0−x⋆∥≤D\|x_0-x_\star\|\leq D, and xNx_N is obtained by NN steps of gradient descent with constant stepsize. For any N,L,DN,L,D, let α(N)≥1\alpha(N)\geq1 be the unique solution of

12(2Nα+1)=12(1−α)2N,\frac{1}{2(2N\alpha+1)}=\frac{1}{2}(1-\alpha)^{2N},

and let r(N)r(N) be their common value. Strengthened minimax conjecture. The stepsize α(N)\alpha(N) is the unique minimizer of

min⁡α~∈Rmax⁡(f,x0)∈FL,Df(xN)−inf⁡f,\min_{\tilde\alpha\in\mathbb{R}}\max_{(f,x_0)\in\mathcal{F}_{L,D}} f(x_N)-\inf f,

and the optimal value is

min⁡α~∈Rmax⁡(f,x0)∈FL,Df(xN)−inf⁡f=r(N)LD2.\min_{\tilde\alpha\in\mathbb{R}}\max_{(f,x_0)\in\mathcal{F}_{L,D}} f(x_N)-\inf f=r(N)LD^2.

This strengthens the balancing conjecture by asserting both uniqueness and the exact minimax value for every N,L,DN,L,D; the paper presents it as an open problem.

References

Primary source

Benjamin Grimmer, Kevin Shu and Alex L. Wang, “A Strengthened Conjecture on the Minimax Optimal Constant Stepsize for Gradient Descent”, arXiv:2407.11739 (2024).

Progress summary

Never refreshed

Nothing recorded yet. Refresh searches the literature and the public web for attempts on this problem, and writes the first summary here.

Solutions 0

No solutions have been posted yet.