Virtual Vafa–Witten formula for rank 2 sheaves on arbitrary surfaces

Let $S$ be a smooth projective surface with $b_1(S)=0$ and $p_g(S)>0$. Let $H,c_1,c_2$ be chosen such that there are no rank 2 strictly Gieseker $H$-semistable sheaves with Chern classes $c_1,c_2$, and let $M:=M_S^H(2,c_1,c_2)$. Write $\overline{\chi}_{-y}^{\mathrm{vir}}(M)$ for its virtual $\chi_y$-genus, and let $\operatorname{vd}(M)$ denote the virtual dimension. Define $\phi(x,y)$ by

$$\phi(x,y):=\prod_{n=1}^{\infty}\frac{1}{(1-x^{2n})^{10}(1-x^{2n}y)(1-x^{2n}y^{-1})}.$$

Virtual Vafa–Witten formula. $\overline{\chi}_{-y}^{\mathrm{vir}}(M)$ equals the coefficient of $x^{\operatorname{vd}(M)}$ in

$$\begin{aligned} \psi_{S,c_1}(x,y):={}&4\left(\frac{\phi(x,y)}{2}\right)^{\chi(\mathcal O_S)}\left(\frac{2\overline{\eta}(x^4)^2}{\theta_3(x,y^{\frac12})}\right)^{K_S^2}\sum_{a\in H^2(S,\mathbb Z)}\mathrm{SW}(a)(-1)^{c_1a}\left(\frac{\theta_3(x,y^{\frac12})}{\theta_3(-x,y^{\frac12})}\right)^{aK_S}. \end{aligned}$$

This is a conjectural virtual refinement of the Vafa–Witten formula for moduli spaces of rank 2 sheaves on arbitrary smooth projective surfaces with holomorphic 2-forms. The source notes that it is strictly stronger than a simpler conjecture stated in the introduction and presents evidence for it in the sections on K3 surfaces, elliptic surfaces, and blow-ups; its resolution is not specified here.