Within-class collapsing conjecture for neural features

About 3 years old · traced to

Consider the unconstrained feature model with KK classes, balanced class size nn, classifier weights wk\boldsymbol{w}_k and features hk,i\boldsymbol{h}_{k,i}, subject to ∣wk∥=∥hk,i∥=1\\|\boldsymbol{w}_k\|=\|\boldsymbol{h}_{k,i}\|=1. Let (W∗,H∗)(\boldsymbol{W}^*,\boldsymbol{H}^*) be a global solution and write hk∗≐∑i=1nhk,i∗\boldsymbol{h}_k^*\doteq\sum_{i=1}^n\boldsymbol{h}_{k,i}^*.

Within-class collapsing. All features within each class are identical: for each k=1,…,Kk=1,\ldots,K, hk,i∗≡hk∗\boldsymbol{h}_{k,i}^*\equiv\boldsymbol{h}_k^* for every ii.

This conjecture asserts the within-class feature collapse predicted by neural-collapse phenomena. Its resolution is not specified in the source.

References

Primary source

Jiachen Jiang, Jinxin Zhou, Peng Wang, Qing Qu, Dustin Mixon, Chong You and Zhihui Zhu, “Generalized Neural Collapse for a Large Number of Classes”, arXiv:2310.05351 (2023).

Progress summary

Never refreshed

Nothing recorded yet. Refresh searches the literature and the public web for attempts on this problem, and writes the first summary here.

Solutions 0

No solutions have been posted yet.