6 problems

Uniform gradient-limit conjecture. Under suitable assumptions on $rho_X$, as $r\to+\infty$ the mapping $A\mapsto r\nabla g_f(rA)$ converges uniformly on $\mathbb{S}^{d^2-1}$ to

Softmax differential convergence conjecture. The differential of the softmax-attention mapping converges uniformly to the differential of the hardmax-attention mapping as the scale…

The attention heads in a transformer network carry information about relationships among input tokens. Attention-head information conjecture. Exploiting information carried by the…

Generic statistical-ensemble conjecture. All Attention schemes are variants of a generic, abstract statistical ensemble, with a more complete set of pairwise and/or higher-order in…

Consider attention mechanisms in which data is exchanged between tokens. The authors entertain the attention-mechanism invariance conjecture. The choice of attention mechanism matt…

Attention-based robustness conjecture. Attention allows the agents to focus on aspects of objects that they are familiar with, making them less susceptible to noise introduced by e…