Marion et al.’s correlated-initialization conjecture for deep residual networks
For deep residual networks whose weights are initialized with correlations across layers, the infinite-depth limiting dynamics should interpolate continuously between the stochastic differential equation obtained under independent initialization and the ordinary differential equation obtained under perfectly correlated initialization.
References
Source: arXiv
Progress summary
A new paper advances the conjecture by proving a broader limiting result, but only for a specified class of correlated Gaussian initializations.
The conjecture concerns the limiting behavior of deep residual networks under correlated initialization. The latest paper develops an interpolation between Brownian and ordinary differential-equation limits and treats stationary Gaussian features with regularly varying correlations.
September 2026 correlated-Gaussian extension
A paper titled Correlated initialization of deep residual networks claims convergence to a Young differential equation driven by a Hermite process, with the limit determined by correlation decay and Hermite rank. This is substantial progress for the stated Gaussian regime, but it does not settle arbitrary correlated initializations; the claim is unverified here.
Current status (as of September 2026): The conjecture has a claimed extension covering stationary correlated Gaussian features, while the full correlated-initialization problem remains open and the reported result is unverified.
Sources
Solutions 0
No solutions have been posted yet.