Block Khovanskii bound for multi-layer Pfaffian neural networks

Let $f\colon \mathbb{R}^n \to \mathbb{R}$ be the output of a fully connected neural network with $\ell$ hidden layers of widths $n_1,\ldots,n_\ell$ and a Pfaffian activation function $\sigma$ satisfying $\sigma'>0$. Let $\alpha$ and $\beta$ denote the Pfaffian degree parameters, and consider the Pfaffian system involving $f$ and its derivatives referred to in the source. Block Khovanskii bound. The number of regular solutions of this system is bounded by

$$2^{\frac{\ell(\ell-1)}{2}+\sum_{i=1}^{\ell}\frac{n_i(n_i-1)}{2}}\cdot p(\alpha,\beta,n,\ell,n_1,\ldots,n_\ell),$$

where $p$ is a polynomial in the indicated quantities. This conjecture seeks an exponential improvement over the generic Khovanskii bound by exploiting the layered structure of the neural-network Pfaffian chain; the multi-layer case is described as still open.