27 problems

The paper considers discrete diffusion models on a state space of dimension $d$, with computational complexity measured up to polylogarithmic factors using the notation…

Let LGS-DDPM denote the Lanczos-based Gaussian denoising diffusion probabilistic model, and consider the batching procedure with stale covariance estimates described in the experim…

Let $\Theta$ be the neural tangent kernel of a diffusion model and let $k_H$ denote the Hellinger kernel of its data distribution. A Hellinger kernel is understood here as the uniq…

The normalized score functions are said to satisfy the non-uniform Lipschitz condition with constant $L$ if, for every $\tau\in(0,1)$, … Here $d$ is the dimension, $T$ is the relev…

In a diffusion model, let a velocity-based parameterisation use a network that predicts the velocity variable $\mathbf{v}$ instead of the usual noise or score quantity. Velocity-pa…

Training-trajectory conjecture. The entire training trajectory might impact the final model quality.

Learning-rate-annealing conjecture. The mismatch between similar loss values and generative quality under ScheduleFree is partially due to missing learning-rate annealing: adding a…

Training-trajectory conjecture. The entire training trajectory, rather than only the final loss, is important for the quality of the trained diffusion model.

Generative models are considered in the setting where high-dimensional data concentrates on a lower-dimensional manifold, as described by the manifold hypothesis. Diffusion models…

MDNS interpolation conjecture. The interpolation between the masked and the target distribution possesses superior properties compared to the geometric annealing family used in LEA…

Let $d$ be the dimension, let $H$ be the step size, let $p$ denote the order of the Runge–Kutta scheme, and let $\varepsilon_{\rm score}$ be the score error for a diffusion probabi…

Let $Z\sim\mathbb{N}(0,I_d)$ be Gaussian noise, let $t$ denote a diffusion time step, and let $\epsilon_\theta^{(t)}$ be the score network at time $t$. A score network is approxima…

Let the score be approximated by a neural network, for example in the neural tangent kernel regime, and let the neural tangent kernel (NTK) have an equivalent kernel. Kernel-convol…

Balanced-schedule convergence conjecture. When the schedule is balanced, matching upper and lower bounds of order $1/T$ should be attainable.

The context describes a two-sided beam-search method for solving states in a Cayley graph and the empirical role of $T$-calibration, but it does not formally define a separate math…

The rejection sampling steps are considered in the importance fine-tuning procedure for diffusion models, with the objective referred to as the one from the DEFT paper. Rejection-s…

Let the target distribution be the mixture $\tfrac{1}{2}\delta_0+\tfrac{1}{2}\mathcal{N}(0,1)$, and let $m^\ast(\mathsf{r})$ denote the quantity defined in the diffuse-then-denoise…

Adaptability conjecture. The techniques of the concurrent work can be adapted into the authors' unified framework, potentially achieving improved results.

Equality conjecture. The expected adoption levels are identical:

Consider diffusion neural networks for image-generation tasks, in particular U-nets with convolutional layers, and graphical models having locality and invariance structures. U-net…

Let $(\boldsymbol{x},\boldsymbol{\theta})\sim\mu(\boldsymbol{x},\boldsymbol{\theta})\propto\exp\{\langle\boldsymbol{\sigma},\boldsymbol{A}\boldsymbol{\sigma}\rangle/2\}$, with the…

Let…

Let critically damped Langevin diffusion (CLD) denote the diffusion used as the forward process in a score-based generative model (SGM), and let DDPM denote the original denoising…

Let $\mathcal M$ be the manifold in the convergence analysis, and let $\varepsilon$ denote the truncation parameter. The convergence bound above has a dependence on $\varepsilon$ o…

Conjecture on performance differences. The performance difference may come from the (in)sensitivity to numerical discretization between the divergence objectives and the mean-match…