4 problems

Let $\ell$ denote the context length, and let $\beta$ be the inverse temperature in a self-attention model considered in the proportional regime, where the context length grows wit…

Let $m(\varepsilon)$ denote the probability vector in the preceding first-order expansion, and let $\mu^{\text{guess}}_\varepsilon$ be a measure on the angular variable $\theta$. L…

Let normalized deep self-attention networks be parametrized via $(\mathbf{M},L)$, where these parameters encode the network as defined in the paper. The normalization removes layer…

Let $V$ be the matrix governing the self-attention dynamics on tokens $z_i(t)\in\mathbb{R}^d$, and let $k\geq 1$ be the number of eigenvalues of $V$ with positive real part. Codime…