The cyclotomic difference-set 2, 4, or 8 conjecture

Let $q=p^{\ell}$ be an odd prime power, let $N\geq 2$ be an even divisor of $q-1$, and let $C_0^{(N,q)}$ be the subgroup of $\mathbb{F}_q^*$ of index $N$. A subset $D\subseteq (\mathbb{F}_q,+)$ is a difference set if every nonzero element of $\mathbb{F}_q$ has the same number of representations as a difference of two elements of $D$. The cyclotomic difference-set 2, 4, or 8 conjecture. If $C_0^{(N,q)}$ is a difference set in $(\mathbb{F}_q,+)$, then $N=2$, $4$, or $8$. This stronger folklore conjecture refines the power-of-two conjecture and is motivated by the complete results known for the small indices $N=2,4,6,8$.