Gaussian equivalence conjecture for the empirical conjugate kernel matrix

Let $\tilde{\boldsymbol X}$ and $\tilde{\boldsymbol y}$ be new training data and labels independent of $\boldsymbol W_1$. Assume the activation $\sigma$ is odd, Assumption 1 holds, and $[?]$ the learning rate satisfies $[?]$ $[?]$; define the empirical feature matrices

$$\boldsymbol\Phi=\frac{1}{\sqrt N}\sigma(\tilde{\boldsymbol X}\boldsymbol W_1),\qquad \overline{\boldsymbol\Phi}=\frac{1}{\sqrt N}\mathopen{}\mathclose\bgroup\originalleft(\mu_1\tilde{\boldsymbol X}\boldsymbol W_1+\mu_2\boldsymbol Z\aftergroup\egroup\originalright),$$

where the entries of $\boldsymbol Z$ are independent standard Gaussian variables. Let $\boldsymbol u_1$ and $\overline{\boldsymbol u}_1$ be the respective left leading singular vectors. Gaussian equivalence conjecture. The empirical feature matrix and its Gaussian equivalent have asymptotically matching singular values and leading-vector alignment with the labels:

$$\mathopen{}\mathclose\bgroup\originalleft|s_i(\boldsymbol\Phi)-s_i(\overline{\boldsymbol\Phi})\aftergroup\egroup\originalright|=o_{d,\mathbb P}(1)\quad\text{for all }i\in[n],$$

and

$$\mathopen{}\mathclose\bgroup\originalleft|\mathopen{}\mathclose\bgroup\originalleft\langle\boldsymbol u_1,\frac{\tilde{\boldsymbol y}}{\|\tilde{\boldsymbol y}\|}\aftergroup\egroup\originalright\rangle\aftergroup\egroup\originalright|^2=\mathopen{}\mathclose\bgroup\originalleft|\mathopen{}\mathclose\bgroup\originalleft\langle\overline{\boldsymbol u}_1,\frac{\tilde{\boldsymbol y}}{\|\tilde{\boldsymbol y}\|}\aftergroup\egroup\originalright\rangle\aftergroup\egroup\originalright|^2+o_{d,\mathbb P}(1).$$

This conjecture proposes that Gaussian equivalence precisely captures the BBP-type spectral transition of the empirical conjugate kernel matrix when the population covariance contains a spike. It is motivated by the corresponding Gaussian-equivalent feature model and the preceding population-level transition; its general validity remains open.