Max-function formulation of the ReLU depth-separation conjecture

Let n=2kn=2^k, and let ReLUn(k)\operatorname{ReLU}_n(k) be the set of continuous piecewise linear functions on Rn\mathbb{R}^n computable with kk hidden layers. Max-function formulation of the depth-separation conjecture. For n=2kn=2^k, the function

(x1,,xn)max{0,x1,,xn}(x_1,\ldots,x_n)\longmapsto\max\{0,x_1,\ldots,x_n\}

is not contained in ReLUn(k)\operatorname{ReLU}_n(k). This statement is presented as equivalent to the preceding depth-separation conjecture, based on a result of Wang. It is open; for example, it is unknown whether two hidden layers suffice for max{0,x1,x2,x3,x4}\max\{0,x_1,x_2,x_3,x_4\}.

Sources & referencesView supporting material

Primary source

Christian Haase, Christoph Hertrich and Georg Loho, “Lower Bounds on the Depth of Integral ReLU Neural Networks via Lattice Polytopes”, arXiv:2302.12553 (2023).

Progress summary

Never refreshed

Nothing recorded yet. Refresh searches the literature and the public web for attempts on this problem, and writes the first summary here.

Solutions 0

No solutions have been posted yet.