The distributional embedding conjecture for intelligent language models

Let $\mathcal{S}$ be the set of sentences, let $\mathscr{L}$ be a causal language model, and let

$$\rho_{\mathscr{L}}:\mathcal{S}\rightarrow\mathcal{B}(\mathcal{S},\mathbb{R})$$

map each sentence to its output-probability distribution, where $\mathcal{B}(\mathcal{S},\mathbb{R})$ is the metric space of absolutely summable real-valued functions on $\mathcal{S}$. The distributional embedding conjecture. If $\mathscr{L}$ is an intelligent language model, then $\rho_{\mathscr{L}}$ is a discrete embedding: every point $\rho_{\mathscr{L}}(\mathbf{s})$ has a neighborhood containing no other $\rho_{\mathscr{L}}(\mathbf{s}')$ with $\mathbf{s}'\neq\mathbf{s}$. The claim formalizes a distributional version of the linguistic idea that meaning is determined by context; the paper gives no resolution of whether intelligent language models satisfy this property.