Strengthened minimax optimal constant-stepsize conjecture for gradient descent

Let FL,D\mathcal{F}_{L,D} be the class of pairs (f,x0)(f,x_0) where ff is LL-smooth and convex, has a minimizer xx_\star with x0xD\|x_0-x_\star\|\leq D, and xNx_N is obtained by NN steps of gradient descent with constant stepsize. For any N,L,DN,L,D, let α(N)1\alpha(N)\geq1 be the unique solution of

12(2Nα+1)=12(1α)2N,\frac{1}{2(2N\alpha+1)}=\frac{1}{2}(1-\alpha)^{2N},

and let r(N)r(N) be their common value. Strengthened minimax conjecture. The stepsize α(N)\alpha(N) is the unique minimizer of

minα~Rmax(f,x0)FL,Df(xN)inff,\min_{\tilde\alpha\in\mathbb{R}}\max_{(f,x_0)\in\mathcal{F}_{L,D}} f(x_N)-\inf f,

and the optimal value is

minα~Rmax(f,x0)FL,Df(xN)inff=r(N)LD2.\min_{\tilde\alpha\in\mathbb{R}}\max_{(f,x_0)\in\mathcal{F}_{L,D}} f(x_N)-\inf f=r(N)LD^2.

This strengthens the balancing conjecture by asserting both uniqueness and the exact minimax value for every N,L,DN,L,D; the paper presents it as an open problem.

Sources & referencesView supporting material

Primary source

Benjamin Grimmer, Kevin Shu and Alex L. Wang, “A Strengthened Conjecture on the Minimax Optimal Constant Stepsize for Gradient Descent”, arXiv:2407.11739 (2024).

Progress summary

Never refreshed

Nothing recorded yet. Refresh searches the literature and the public web for attempts on this problem, and writes the first summary here.

Solutions 0

No solutions have been posted yet.