4 problems
Consider a neural network trained on a binary classification problem with cross-entropy (CE) loss, and let its Hessian be decomposed into diagonal and off-diagonal parameter blocks…
Sparsity conjecture. The number of non-zero components of the hypervolume Hessian matrix grows linearly in and exponentially in .
Ordered-heuristic conjecture. The {tt Ordered} heuristic is in fact using a very powerful piece of information about the problem: when creating test problems, researchers naturally…
Hessian nondegeneracy conjecture. The Hessian of is nondegenerate at the critical point under discussion, especially at critical points corresponding to nondegenerate geometrie…