Karloff's smallest-eigenvalue conjecture for distance graphs of Johnson graphs

Let $J(n,d,j)$ be the distance-$j$ graph of the Johnson graph on the $d$-subsets of an $n$-set, and let its eigenvalues be $E_j(i)$, where

$$E_j(i)=\sum_{h=0}^i(-1)^{i-h}\binom{i}{h}\binom{d-h}{j}\binom{n-d-i+h}{n-d-j}.$$

Karloff's conjecture. If $n=2d$ and $j>d/2$, then the smallest eigenvalue of $J(n,d,j)$ is $E_j(1)$. The surrounding text says that this conjecture is settled in the paper, so the claim is solved.