109 problems

Implicit-regularization conjecture for smooth global minimizers of LMNets
arXiv@arxiv
Let the linear multistep neural network loss have infinitely many global minimizers because the governing-function components $f_n$ are underdetermined by the training equations, a…
Conjectureopendeep-learning differential-equations numerical-analysis
0
CuBAS conjecture on curvature-guided warm-up sets for neural network initialization
arXiv@arxiv
A CuBAS-selected warm-up set is a subset of the available training data, chosen using the CuBAS framework's curvature-based sampling procedure. Let the warm-up set comprise $5\%$ --…
Conjectureopenactive-learning deep-learning geometric-learning
0
Extension of the convexification and deep-learning approach to other coefficient inverse problems
arXiv@arxiv
A coefficient inverse problem (CIP) is an inverse problem for recovering coefficients of a differential equation from boundary measurements; here the approach combines a convexific…
Conjectureopendeep-learning electrical-impedance-tomography inverse-problems
1
Deep Sprecher Network universality conjecture
arXiv@arxiv
Let $n\geq 1$ , $L\geq 1$ , and let $f:[0,1]^n\to\mathbb{R}$ or $f:[0,1]^n\to\mathbb{R}^m$ be continuous. Consider a Sprecher Network with architecture $n\to[d_1,\dots,d_L]\to 1$ or…
Conjectureopendeep-learning neural-networks universal-approximation
0
Huh et al.'s conjecture on depth and effective-rank embeddings
arXiv@arxiv
Let the final-layer embeddings of a deep network be represented by their Gram matrix, and measure its complexity by effective rank. Huh et al. observe that this Gram matrix has low…
Conjectureopendeep-learning low-rank-methods representation-learning
1
Exactness of the replica symmetric theory for one-hidden-layer perceptrons
arXiv@arxiv
Let $L=1$ and let $\sigma$ be an activation function with second Hermite coefficient $\mu_2=0$ . Exactness conjecture. For this one-hidden-layer perceptron, the replica symmetric th…
Conjectureopendeep-learning replica-symmetry statistical-physics
2
The CPU-GPU conjecture for deep-learning models
arXiv@arxiv
Let a deep-learning model be trained either with graphical processing units (GPUs) or with a central processing unit (CPU), and suppose the CPU-trained model incorporates sufficien…
Conjectureopencomputational-complexity deep-learning
1
The U-NET optimal trade-off conjecture for radio-to-PPG translation
arXiv@arxiv
U-NET optimal trade-off conjecture. A U-NET provides the optimal trade-off between performance and complexity for radio-to-PPG translation, especially given a limited-size dataset.
Conjectureopendeep-learning photoplethysmography signal-processing
1
Conjecture on decomposing the loss into invariance and target errors
arXiv@arxiv
Loss decomposition conjecture. The loss may also be decomposed into two parts: one controlling the error from invariance, and the other controlling the error from the target.
Conjectureopendeep-learning invariance loss-functions
0
Hertrich–Basu–Di Summa–Skutella depth conjecture for CPWL functions
arXiv@arxiv
Let $\operatorname{\mathsf{CPWL}}$ denote the continuous piecewise linear functions on $\mathbb{R}^n$ . The previously known upper bound is $\lceil \log_2(n+1)\rceil$ hidden layers.…
Conjectureopendeep-learning neural-network-expressivity piecewise-linear-functions
0
The conjecture that redundant encodings improve robustness and generalization
arXiv@arxiv
Redundant-encoding conjecture. More redundant encodings may be preferable for the network, as they could lead to representations that are more robust to small input variations and…
Conjectureopendeep-learning information-theory representation-learning
0
Conjecture that balancedness corresponds to detailed balance in deep learning
arXiv@arxiv
Balancedness is a structural property of deep linear networks, represented by the balanced manifold $\balance$ . Local equilibrium along a network is described as detailed balance.…
Conjectureopendeep-learning neural-networks statistical-physics
0
Conjecture on the fidelity between deep linear networks and deep learning
arXiv@arxiv
A deep linear network (DLN) is a neural network whose layers are linear maps, and deep learning refers to neural-network models trained on function-approximation tasks. Numerical e…
Conjectureopendeep-learning dynamical-systems neural-networks
0
Overparameterization Improves Optimization for Neural Operators
arXiv@arxiv
Let a neural operator have adjustable capacity, for example through its number of layers or neurons, and consider the non-convex optimization landscape of its training loss. Overpa…
Conjectureopendeep-learning neural-operators optimization
0
Generalization benefits from delay in asynchronous pipeline training
arXiv@arxiv
In asynchronous pipeline training, let the parameter updates be subject to the delays induced by the pipeline execution. Delay-generalization conjecture. The observed acceleration…
Conjectureopendeep-learning optimization pipeline-training
0
Attention-mechanism invariance conjecture
arXiv@arxiv
Consider attention mechanisms in which data is exchanged between tokens. The authors entertain the attention-mechanism invariance conjecture. The choice of attention mechanism matt…
Conjectureopenattention-mechanisms deep-learning
0
Weighted optics conjecture for automatic differentiation
arXiv@arxiv
Lenses cannot model automatic differentiation algorithms that do not use gradient checkpointing, while weighted optics conjecture. weighted optics should be able to model such algo…
Conjectureopenautomatic-differentiation category-theory deep-learning
0
The conjecture on practical advantages of stochastic gradient momentum
arXiv@arxiv
Let SG denote stochastic gradient and SGM denote stochastic gradient method with momentum. For deep neural networks (DNNs), consider objective functions, their ravines, the expecte…
Conjectureopenconvergence-rates deep-learning stochastic-optimization
0
The higher-order interaction hypothesis for the success of state-of-the-art deep learning models
arXiv@arxiv
Higher-order interactions (HOIs) are interactions involving more than two components in a complex system. Non-linear activation functions incorporating HOIs are related to attentio…
Conjectureopendeep-learning higher-order-interactions neural-networks
1
Conjecture on hypernetwork weights in under-sampled regions
arXiv@arxiv
A hypernetwork is trained to predict optimized weights from hyperparameters, and its predictions are evaluated in regions of hyperparameter space with limited training data. Hypern…
Conjectureopendeep-learning hypernetworks optimization
0
The conjectured inverse-sample convergence rate for parametric PDE operators
arXiv@arxiv
The examples concern neural-network approximation of solution operators for parametric partial differential equations, including the parametric stationary Boussinesq equations, wit…
Conjectureopendeep-learning operator-learning parametric-pdes
0
Conjecture on gradient explosion from absolute-value losses
arXiv@arxiv
The absolute value function has derivative magnitude $1$ wherever it is differentiable. Gradient-explosion conjecture. Using the absolute value function in a training objective con…
Conjectureopendeep-learning numerical-stability optimization
0
Conjecture on compositional parameter and response embeddings for operator learning
arXiv@arxiv
Compositional embedding conjecture. A compositional framework to combine the parameter and response embeddings can lead to a more expressive model that can approximate more complex…
Conjectureopendeep-learning numerical-analysis operator-learning
0
Frankle–Carbin lottery ticket hypothesis for neural networks
arXiv@arxiv
Lottery ticket hypothesis. Dense, randomly initialized, feed-forward networks contain winning-ticket subnetworks that, when trained in isolation, reach test accuracy comparable to…
Conjectureopendeep-learning neural-networks
0
Asymptotic approximation of population covariances by linearized recursions
arXiv@arxiv
Asymptotic covariance approximation conjecture. The population covariances can be asymptotically approximated by the last iterates of the corresponding linear recursions, with the…
Conjectureopendeep-learning random-features random-matrix-theory
0