Generic identifiability of normalized deep self-attention parametrizations

Let normalized deep self-attention networks be parametrized via $(\mathbf{M},L)$, where these parameters encode the network as defined in the paper. The normalization removes layer-wise scaling, while the remaining symmetries inside the attention matrices and the transformations $C_i$ persist. Generic identifiability conjecture. For normalized deep self-attention networks, the generic fibers of the parametrization via $(\mathbf{M},L)$ are singletons; equivalently, the parametrization is generically one-to-one. This asserts that normalization only breaks the layer-wise scaling symmetry of the parametrization, extending the single-layer identifiability result to the deep case.