Zero-error realizable CMI conjecture

At least 5 years old · documented by

Let X\mathcal{X} be a domain, let W\mathcal{W} be a class of functions f:X→{0,1}f:\mathcal{X}\to\{0,1\} with VC dimension d≥1d\geq 1, and let n≥1n\geq 1. For a dataset (x,y)∈(X×{0,1})n(x,y)\in(\mathcal{X}\times\{0,1\})^n, write ℓ(h,(x,y))\ell(h,(x,y)) for the empirical 00-11 loss, and let CMI(A)\mathsf{CMI}(A) denote the conditional mutual information of an algorithm's output. Zero-error realizable CMI conjecture. The realizable conjecture above holds with c′=0c'=0; equivalently, there exists an algorithm with CMI(A)≤cd\mathsf{CMI}(A)\leq c d whose expected empirical loss is zero on every realizable dataset.

This strengthens the realizable conjecture by requiring exact consistency rather than an O(d/n)O(d/n) error bound. It is stated as a further conjectural refinement of the proposed CMI guarantee.

References

Primary source

Thomas Steinke and Lydia Zakynthinou, “Reasoning About Generalization via Conditional Mutual Information”, arXiv:2001.09122 (2020).

Progress summary

Never refreshed

Nothing recorded yet. Refresh searches the literature and the public web for attempts on this problem, and writes the first summary here.

Solutions 0

No solutions have been posted yet.