Unimodality conjecture for minimal filling neural-network architectures

Fix the depth LL, the endpoint widths d0d_0 and dLd_L, and the activation degree rr. Let d53d=(d0,d1,d55fd53,dL)d53d=(d_0,d_1,d55fd53\dots,d_L) be a width vector, and let d53d\preccurlyeqd53dd53d\preccurlyeqd53d' mean coordinatewise comparison. An architecture is filling when its neurovariety d4b1d53d,rd4b1_{d53d,r} has the full expected ambient dimension; d53dd53d is minimal when no smaller width vector with the same fixed endpoints has this property.

Unimodality conjecture. Every minimal width vector whose architecture is filling is unimodal: there exists i{0,1,d55fd53,L}i\in\{0,1,d55fd53\dots,L\} such that (d0,d55fd53,di)(d_0,d55fd53\dots,d_i) is weakly increasing and (di,d55fd53,dL)(d_i,d55fd53\dots,d_L) is weakly decreasing.

The conjecture formalizes the architectural heuristic that a network should first expand its feature capacity and then contract it. The supplied text gives motivation but no proof or resolution.

Sources & referencesView supporting material

Primary source

Kaie Kubjas, Jiayi Li and Maximilian Wiesmann, “Geometry of Polynomial Neural Networks”, arXiv:2402.00949 (2024).

Progress summary

Never refreshed

Nothing recorded yet. Refresh searches the literature and the public web for attempts on this problem, and writes the first summary here.

Solutions 0

No solutions have been posted yet.