Data-dependent equivalence conjecture for random features

Let the assumptions in \Crefrange{asm:model}{asm:feat} hold. Define $\phi_n=p/n$. Let $k\le n$ be the subsample size and set $\bar{\psi}_n=p/k$. Suppose that $\varphi$ satisfies certain regularity conditions. For any $M\in\mathbb{N}\cup\{\infty\}$, define $\bar{\lambda}_n$ by

$$\frac{1}{M}\sum_{\ell=1}^{M}\frac{1}{k}\operatorname{tr}\left[\left(\frac{1}{k}\varphi(\mathbf{L}_{I_{\ell}}\mathbf{X}\mathbf{F}^{\top})\varphi(\mathbf{L}_{I_{\ell}}\mathbf{X}\mathbf{F}^{\top})^{\top}\right)^+\right]=\frac{1}{n}\operatorname{tr}\left[\left(\frac{1}{n}\varphi(\mathbf{X}\mathbf{F}^{\top})\varphi(\mathbf{X}\mathbf{F}^{\top})^{\top}+\bar{\lambda}_n\mathbf{I}_n\right)^{-1}\right].$$

Define the data-dependent path $\mathcal{P}_n=\mathcal{P}(\bar{\lambda}_n;\phi_n,\bar{\psi}_n)$. Random-feature equivalence conjecture. The conclusions of the risk-equivalence theorem, the corresponding prediction-risk proposition, and the linear-estimator equivalence theorem continue to hold for $(\varphi(\mathbf{X}\mathbf{F}^{\top}),\mathbf{y})$, with $\mathcal{P}$ replaced by $\mathcal{P}_n$. This conjecture extends the subsampling--ridge correspondence from the assumed random-matrix features to random features with nonlinear activation functions. Its status is not established in the supplied source.