The converse to E1-Morita invariance of module categories

Let $\CC$ be a cocomplete $E_m$-monoidal category, and let $A_1,A_2$ be $E_1$-algebras in $\CC$. Write $\Mod_A^{E_1}(\CC)$ for the $E_1$-module category of $A$.

The converse Morita conjecture. If

$$\Mod_{A_1}^{E_1}(\CC)\simeq\Mod_{A_2}^{E_1}(\CC)$$

as $E_1$-monoidal categories, then $A_1$ and $A_2$ are $E_1$-Morita equivalent.

This would make the $E_1$-monoidal module category a complete invariant for $E_1$-Morita equivalence, but the source offers no proof or resolution.