11 problems

Universality conjecture. Analogous phenomena persist for asymmetric spiked tensor models.

Let $X \in \mathbb{R}^{n \times T}$ satisfy $C_X \succ 0$, and let $W(t)$ follow the continuous-time gradient-flow feedforward synaptic dynamics. Let $\mathcal{S}$ denote the stati…

Gradient-based QAOA saddle-point conjecture. Gradient-based optimization of QAOA often gets trapped by saddle points of the QAOA loss-function landscape.

Let $F$ be the loss function of a neural network, and distinguish local minima from saddle points, where a saddle point is a critical point that is neither a local minimum nor a lo…

Let $F$ be the loss of a neural network, and let $M(v_1,v_2,v_3)$ denote the linear space spanned by three vectors $v_1,v_2,v_3$. Mode-connectivity conjecture. If $\theta_1^*$ and…

Let $F$ be the loss of a neural network. For linearly independent vectors $v_1,v_2,v_3$, let $M(v_1,v_2,v_3)$ denote the linear space spanned by them. Barrier conjecture for global…

Let $F$ be the loss of a neural network and let $\theta^*$ be a global minimum. Choose two vectors $v_1,v_2$ according to a specified rule, for example a Gaussian distribution, and…

Monotone loss-profile conjecture. Under certain conditions on the neural nets, $F_{[\theta^0,\theta^*]}(\alpha)$ is a strictly decreasing function on $[0,1]$. This conjecture forma…

Gradient-descent performance conjecture. For a proper constant $\eta$, and for a random initial point drawn from a certain distribution, such as Xavier initialization,

A neural-network loss landscape is a function $F$ of parameters $\theta$, and noisy gradient descent is an optimization algorithm acting on these parameters. A local minimum is asy…

A training objective is an objective function arising from a machine-learning training problem, and its optimization landscape is the geometry of its function values and critical p…