Affine-automorphism conjecture for abelian permutation groups

Let $G$ be an abelian permutation group, let $P(G)$ be its permutation polytope, and write $|G|$ for the order of $G$.

Affine-automorphism conjecture. If

$$|G|>2,$$

then the group of affine automorphisms of $P(G)$ contains more elements than $G$.

For nonabelian groups, left and right multiplication already provide the motivating source of additional affine automorphisms. The assertion for abelian groups of order greater than two is presented as open.