Sun's finiteness conjecture for exceptional determinant primes

Let $m$ be a positive odd integer, and define

$$E_2(m)=\{p:p\text{ is a prime with }4\mid p-1\text{ and }p\mid S_{m+\frac{p-1}{2},2}(-1,p)\}.$$

Here $S_{n,2}(d,p)$ is the determinant considered in the paper. Sun's finiteness conjecture. For every positive odd integer $m$, the set $E_2(m)$ is finite. In particular,

$$E_2(5)=\{29\},\quad E_2(7)=\{13,53\},\quad E_2(9)=\{13,17,29\},$$ $$E_2(11)=\{17,29\},\quad E_2(13)=\{17,109,401\}.$$

The displayed finite sets were supplied by Sun from calculations for primes below $1000$. The general finiteness assertion remains unresolved according to the supplied text.