Central limit theorem for infinitesimal gradient boosting

At least 4 years old · documented by

Let F^[t/λ]λ\hat{F}^{\lambda}_{[t/\lambda]} be the stochastic gradient boosting process, let (F^t)t≥0(\hat{F}_t)_{t\geq 0} be its infinitesimal gradient boosting limit, let T:Lx2→Lx2\mathcal{T}:L^2_{\mathbf{x}}\to L^2_{\mathbf{x}} be the associated operator, and let T~\widetilde{T} denote the corresponding stochastic operator. Write D([0,∞),Lx2)\mathbb{D}([0,\infty),L^2_{\mathbf{x}}) for the Skorokhod path space and dF^tT\mathrm{d}_{\hat{F}_t}\mathcal{T} for the derivative of T\mathcal{T} at F^t\hat{F}_t. Central limit theorem. Possibly under stronger assumptions than Assumption A, T\mathcal{T} is continuously differentiable as an operator Lx2→Lx2L^2_{\mathbf{x}}\to L^2_{\mathbf{x}}, and

1λ(F^[t/λ]λ−F^t)t≥0⟶dF,\frac{1}{\sqrt{\lambda}}\bigl(\hat{F}^{\lambda}_{[t/\lambda]}-\hat{F}_t\bigr)_{t\geq 0}\stackrel{d}{\longrightarrow}\mathscr{F},

where convergence holds in distribution on D([0,∞),Lx2)\mathbb{D}([0,\infty),L^2_{\mathbf{x}}). The limit F=(Ft)t≥0\mathscr{F}=(\mathscr{F}_t)_{t\geq 0} is a zero-mean continuous Gaussian process in Lx2L^2_{\mathbf{x}} characterized by

dFt=dGt+(dF^tT)(Ft) dt,\mathrm{d}\mathscr{F}_t=\mathrm{d}\mathscr{G}_t+(\mathrm{d}_{\hat{F}_t}\mathcal{T})(\mathscr{F}_t)\,\mathrm{d}t,

where G\mathscr{G} is a cylindrical Wiener process on Lx2L^2_{\mathbf{x}} with covariance

E[⟨Gt,f⟩⟨Gs,g⟩]=∫0t∧sE[⟨f,T~(F^u)−T(F^u)⟩⟨g,T~(F^u)−T(F^u)⟩].\mathbb{E}\left[\langle\mathscr{G}_t,f\rangle\langle\mathscr{G}_s,g\rangle\right]=\int_0^{t\wedge s}\mathbb{E}\left[\langle f,\widetilde{T}(\hat{F}_u)-\mathcal{T}(\hat{F}_u)\rangle\langle g,\widetilde{T}(\hat{F}_u)-\mathcal{T}(\hat{F}_u)\rangle\right].

This is a conjectural functional central limit theorem describing the Gaussian fluctuations around the infinitesimal gradient boosting limit; the supplied text gives no resolution, so its status remains open.

References

Primary source

Clément Dombry and Jean-Jil Duchamps, “Infinitesimal gradient boosting”, arXiv:2104.13208 (2023).

Progress summary

Never refreshed

Nothing recorded yet. Refresh searches the literature and the public web for attempts on this problem, and writes the first summary here.

Solutions 0

No solutions have been posted yet.