Semistability reduction conjecture for endomorphism bundles

Let $X$ be a smooth complex projective variety, let $\mathcal O_X(1)$ be ample, and let $E$ be a slope-semistable locally free sheaf. Choose a finitely generated model $\pi:\mathcal X\to S=\operatorname{Spec}R$ with fibrewise slope-semistable sheaves $\mathcal E$. Let $\Sigma^{\mathrm{nss}}$ be the set of closed points $s\in S$ such that $\mathcal E_s$ is not strongly slope semistable. A vector bundle is numerically flat when it and its dual are nef. Semistability reduction conjecture. If $\Sigma^{\mathrm{nss}}$ is infinite, then $\operatorname{End}E$ is numerically flat; moreover, $\operatorname{End}E$ is not étale trivializable. The nontrivial assertion remains in the surface case; the statement is known for curves.