Saddle-point avoidance conjecture for backtracking gradient descent

Let f:RmRf:\mathbb{R}^m\rightarrow\mathbb{R} be a C1C^1 function that is C2C^2 near its critical points, and let {zn}\{z_n\} be the sequence generated by the Backtracking GD method from an initial point z0Rmz_0\in\mathbb{R}^m. A saddle-point avoidance conjecture asserts that the set of initial points z0Rmz_0\in\mathbb{R}^m for which the cluster points of {zn}\{z_n\} contain a saddle point has Lebesgue measure 00. The preceding discussion presents this as a heuristic expectation based on the conjectured long-run stabilization of Backtracking GD to a finite union of Standard GD processes; whether saddle-point avoidance holds under the stated C1C^1 and local C2C^2 assumptions remains open.

Sources & referencesView supporting material

Primary source

Tuyen Trung Truong and Tuan Hang Nguyen, “Backtracking gradient descent method for general C^1 functions, with applications to Deep Learning”, arXiv:1808.05160 (2019).

Progress summary

Never refreshed

Nothing recorded yet. Refresh searches the literature and the public web for attempts on this problem, and writes the first summary here.

Solutions 0

No solutions have been posted yet.