Krzakala et al.'s TAP prediction for the free energy of Bayes linear regression

Let $\beta=(\beta_1,\ldots,\beta_p)$ have product prior $\pi_0=\pi^{\otimes p}$, where $\pi$ is a probability distribution on $[-1,1]$. Let $\mathcal{Z}_p$ denote the linear-regression partition function, with data matrix $X$, observations $y$, noise variance $\Delta$, and aspect ratio $\alpha$; write $\bar c=p^{-1}\sum_{i=1}^p c_i$. For $a\in\mathbb{R}^p$, define probability measures $Q_i$ by

$$\frac{\mathrm{d}Q_i}{\mathrm{d}\pi}(\beta)\propto\exp\left(-\frac{(\beta-R_i)^2}{2\Sigma_*^2}\right),$$

where $R_i$ is chosen so that $\mathbb{E}_{Q_i}[\beta]=a_i$, and set $c_i=\operatorname{Var}_{Q_i}(\beta)$. The parameter $\Sigma_*$ depends on $\Delta$ and $\alpha$ and is the smallest solution of a fixed-point equation. Krzakala et al.'s TAP conjecture. The log-partition function satisfies

$$\log \mathcal{Z}_p=\max_{\|a\|_\infty\leq 1}\left[-\frac{\lVert y-Xa\rVert^2}{2\Delta}-\frac{n}{2}\log\left(1+\frac{\bar c}{\Delta}\right)+\sum_{i=1}^p\log\mathbb{E}_{\pi}\left[\exp\left(-\frac{(\beta_i-R_i)^2}{2\Sigma_*^2}\right)\right]\right]+o(p).$$

The additional logarithmic term is the Onsager correction absent from the naive mean-field approximation. The conjecture predicts that NMF fails when the numbers of samples and features grow proportionally; the stated bounded-support formulation extends directly to priors supported on any bounded set.