14 problems

Width-one identifiability conjecture. 1. There exist architectures $\mathcal{A}=(n_0,\ldots,n_{L+1})$ with $n_\ell=1$ for some $\ell\leq L$ for which no identifiable parameters exi…

Let $\operatorname{\mathsf{CPWL}}_n$ denote the class of continuous piecewise-linear functions in $n$ variables, and let $\operatorname{\mathsf{ReLU}}_{n,k}$ denote the functions c…

Consider architectures $(n_0,n_1,\ldots,n_L)$ with fixed $n_0$, and suppose that the weights and biases in each layer are selected independently and identically distributed from a…

Layerwise matrix-closedness conjecture. The set $\mathcal{F}_{\boldsymbol{I}}(\Omega)$ is closed in $(C^0(\Omega),\|\cdot\|_\infty)$ if and only if, for every tuple of binary diago…

Shallow-network closedness conjecture. The set $\mathcal{F}_{\boldsymbol{I}}(\Omega)$ is closed in $(C^0(\Omega),\|\cdot\|_\infty)$ if and only if $\boldsymbol{I}$ satisfies the se…

Closedness-equivalence conjecture. An architecture $\boldsymbol{I}$ has a closed function space on the cube if and only if its realization set is closed on arbitrary finite domains…

Let $n=2^k$, and let $\operatorname{ReLU}_n(k)$ be the set of continuous piecewise linear functions on $\mathbb{R}^n$ computable with $k$ hidden layers. Max-function formulation of…

Let $\operatorname{ReLU}_n(k)$ denote the set of continuous piecewise linear functions defined on $\mathbb{R}^n$ and computable with $k$ hidden layers. Hertrich–Lindner–Raginsky–Ba…

Let $F$ be the piecewise-linear function represented by a ReLU network, and let $4\mathcal{C}(F)\4$ be its canonical polyhedral complex. Consider a general training path in the net…

Consider fully connected deep ReLU networks and their network functions in the setting of random initialization. Gaussian-regime width conjecture. A plausible explanation for the o…

Generic total H-complexity conjecture. With probability $1$, the total $H$-complexity of $F$ is less than or equal to the number of $0$-cells of $\mathcal{C}(F)$.

Let $f\colon\mathbb{R}^4\to\mathbb{R}$ be … Let $H$ be the hyperplane arrangement consisting of the hyperplanes where two of $x_0=0,x_1,x_2,x_3,x_4$ are equal. A positively homogen…

For $n\boxin\mathbb{N}$, let … Here, $\operatorname{ReLU}_n(k)$ denotes the class of functions representable by ReLU neural networks with $n$ input variables and $k$ hidden layers,…

Uniform weight-scaling conjecture. Scaling the weights uniformly should approximately preserve the global number of activation regions.