The n1n^{-1} convergence-rate conjecture for deep-limit minimisers

Let θ(n)\theta^{(n)} be minimisers of the finite-depth energies cEncE_n, and suppose the assumptions of the convergence theorem for cEncE_n and cEcE_\infty are satisfied. Let cthetaΘctheta\in\Theta be a limit point of {θ(n)}nN\{\theta^{(n)}\}_{n\in\mathbb N}, and let {θ(nk)}kN\{\theta^{(n_k)}\}_{k\in\mathbb N} be a subsequence converging to cthetactheta. With dd denoting the distance on the parameter space, the conjecture asserts that there exists C>0C>0 such that

d(θ(nk),θ)Cnk1.d(\theta^{(n_k)},\theta)\leq C n_k^{-1}.

The n1n^{-1} convergence-rate conjecture. Under these assumptions, the distance between the convergent subsequence of finite-depth minimisers and its limit is O(nk1)O(n_k^{-1}). This conjectured rate is motivated by Taylor expansions of the recovery sequence, which suggest an O(n1)O(n^{-1}) rate for the minima when the relevant H2H^2 regularity is available, together with a local bound controlling parameter distance by the energy error. The result is not proved in the source.

Sources & referencesView supporting material

Primary source

Matthew Thorpe and Yves van Gennip, “Deep Limits of Residual Neural Networks”, arXiv:1810.11741 (2022).

Progress summary

Never refreshed

Nothing recorded yet. Refresh searches the literature and the public web for attempts on this problem, and writes the first summary here.

Solutions 0

No solutions have been posted yet.