Within-class collapsing conjecture for neural features

From papers

Consider the unconstrained feature model with KK classes, balanced class size nn, classifier weights wk\boldsymbol{w}_k and features hk,i\boldsymbol{h}_{k,i}, subject to wk=hk,i=1\\|\boldsymbol{w}_k\|=\|\boldsymbol{h}_{k,i}\|=1. Let (W,H)(\boldsymbol{W}^*,\boldsymbol{H}^*) be a global solution and write hki=1nhk,i\boldsymbol{h}_k^*\doteq\sum_{i=1}^n\boldsymbol{h}_{k,i}^*.

Within-class collapsing. All features within each class are identical: for each k=1,,Kk=1,\ldots,K, hk,ihk\boldsymbol{h}_{k,i}^*\equiv\boldsymbol{h}_k^* for every ii.

This conjecture asserts the within-class feature collapse predicted by neural-collapse phenomena. Its resolution is not specified in the source.

Progress summary

Nothing recorded yet. Refresh searches the literature and the public web for attempts on this problem, and writes the first summary here.

Sources & referencesView supporting material

Primary source

Jiachen Jiang, Jinxin Zhou, Peng Wang, Qing Qu, Dustin Mixon, Chong You and Zhihui Zhu, “Generalized Neural Collapse for a Large Number of Classes”, arXiv:2310.05351 (2023).

Solutions 0

No solutions have been posted yet.