Progressive Feedforward Collapse conjecture for ResNet training

Let $\boldsymbol{H}^{l}$ denote the feature matrix at layer $l$, with class means $\boldsymbol{h}_k^l$ and global mean $\boldsymbol{h}_G^l$. For each layer, measure feature collapse toward class means, collapse of centered class means toward the simplex equiangular tight frame (ETF), and nearest class center (NCC) accuracy using the three progressive feedforward collapse metrics $\mathcal{PFC}_1$, $\mathcal{PFC}_2$, and $\mathcal{PFC}_3$. Here, $\mathcal{PFC}_2$ measures the distance between the normalized Gram matrix of the centered class means and the simplex ETF, while $\mathcal{PFC}_3$ is the NCC accuracy.

Progressive Feedforward Collapse conjecture. At the terminal phase of ResNet training, neural collapse emerges in the last-layer features. Before the effective depth, there exists an order in the degree of collapse of each layer, measured by the three PFC metrics: the features of each layer progressively collapse to their class means; the centered class means of each layer's features progressively collapse to the simplex ETF; and the NCC accuracy of each layer progressively collapses to $1$. During training, the metrics at each layer start at random initialization and gradually converge to the final order.

This conjecture proposes a layerwise progression of neural-collapse geometry through ResNet forward propagation, extending last-layer neural collapse to intermediate layers. The supplied text does not provide evidence that the conjecture has been proved or refuted.