The local-distance conjecture for autoencoder generalization

Let Dtrain\mathcal{D}_{train} and Dtest\mathcal{D}_{test} be the training and test subsets of the dataset, and let DmodelEmodelD_{model}\circ E_{model} be a trained autoencoder. Near each training point, compare this map with a projection map, equivalently measuring its local distance from the identity through its Jacobian. Local-distance conjecture. The further the map DmodelEmodelD_{model}\circ E_{model} is from a projection near the training points, the worse the mean squared error will be for the autoencoder on the test points. This proposes a relationship between local geometric deviation from a projection on training data and test-set reconstruction error; the supplied text gives the relationship as a conjecture rather than an established result.

Sources & referencesView supporting material

Primary source

Susama Agarwala, Benjamin Dees, Andrew Gearhart and Corey Lowman, “Geometry and Generalization: Eigenvalues as predictors of where a network will fail to generalize”, arXiv:2107.06386 (2021).

Progress summary

Never refreshed

Nothing recorded yet. Refresh searches the literature and the public web for attempts on this problem, and writes the first summary here.

Solutions 0

No solutions have been posted yet.