The n−1n^{-1} convergence-rate conjecture for deep-limit minimisers

About 8 years old · traced to

Let θ(n)\theta^{(n)} be minimisers of the finite-depth energies cEncE_n, and suppose the assumptions of the convergence theorem for cEncE_n and cE∞cE_\infty are satisfied. Let ctheta∈Θctheta\in\Theta be a limit point of {θ(n)}n∈N\{\theta^{(n)}\}_{n\in\mathbb N}, and let {θ(nk)}k∈N\{\theta^{(n_k)}\}_{k\in\mathbb N} be a subsequence converging to cthetactheta. With dd denoting the distance on the parameter space, the conjecture asserts that there exists C>0C>0 such that

d(θ(nk),θ)≤Cnk−1.d(\theta^{(n_k)},\theta)\leq C n_k^{-1}.

The n−1n^{-1} convergence-rate conjecture. Under these assumptions, the distance between the convergent subsequence of finite-depth minimisers and its limit is O(nk−1)O(n_k^{-1}). This conjectured rate is motivated by Taylor expansions of the recovery sequence, which suggest an O(n−1)O(n^{-1}) rate for the minima when the relevant H2H^2 regularity is available, together with a local bound controlling parameter distance by the energy error. The result is not proved in the source.

References

Primary source

Matthew Thorpe and Yves van Gennip, “Deep Limits of Residual Neural Networks”, arXiv:1810.11741 (2022).

Progress summary

Never refreshed

Nothing recorded yet. Refresh searches the literature and the public web for attempts on this problem, and writes the first summary here.

Solutions 0

No solutions have been posted yet.