Conjecture on identifiable parameters in ReLU architectures with width-one layers

Let $\mathcal{A}=(n_0,\ldots,n_{L+1})$ be a ReLU network architecture, with parameter space $\Theta_\mathcal{A}$, quotient parameter space $\overline{\Theta_\mathcal{A}}$, and quotient map identifying parameters up to the relevant within-layer permutations. A parameter is finitely identifiable if its fiber in $\overline{\Theta_\mathcal{A}}$ is a finite set.

Width-one identifiability conjecture.

1. There exist architectures $\mathcal{A}=(n_0,\ldots,n_{L+1})$ with $n_\ell=1$ for some $\ell\leq L$ for which no identifiable parameters exist.
2. For every architecture, there exists a finitely identifiable parameter.

The first assertion predicts that width-one non-output layers can prevent even individual identifiable parameters, because the resulting activation image is at most one-dimensional and transverse breakpoint assignments may remain ambiguous. The second asserts that finite identifiability should nevertheless occur in every architecture. The source provides no resolution of either assertion.