Sharpness conjecture for the larger sieve in the small-density regime

Let $d\geq3$, let $A\subset[N]$, and suppose that for every prime $p\leq N^{1/d}$ the set $A\pmod p$ occupies at most $(1/d+o(1))p$ residue classes.

Larger sieve sharpness conjecture. Under these hypotheses,

$$|A|\ll N^{1/d-c}$$

for some small constant $c=c(d)>0$.

The conjecture asks for a power-saving improvement over the usual larger-sieve bound in the regime $\alpha=1/d$ with $d\geq3$. The source presents it as an expectation and does not report a resolution.