Proof

By hypothesis, $(G,\ast)$ is a finite group with a group order $|G|=p,$ where $p$ is a prime number.
Since finite order of an element equals group order, $\langle a\rangle$ is a subgroup of $G$ with a finite order $|\langle a\rangle|$ for any $a\in G.$
According to the Lagrange's theorem, the $|\langle a\rangle|$ is a divisor of $p.$
By definition of prime numbers, either $|\langle a\rangle|=1$ or $|\langle a\rangle|=p.$
Because $\langle a\rangle$ contains at least two elements, the neutral element $e$ and $a,$ we have $|\langle a\rangle|\neq 1.$
Therefore, $|\langle a\rangle|=p=|G|.$
Since all subgroups of cyclic groups are cyclic, $\langle a\rangle$ is a cyclic group.
Therefore, $G$ is a cyclic group.
By the classification of cyclic groups, $(G,\ast)$ is isomorphic to the additive subgroup of integers $(\mathbb Z_p, + ).$
∎

Thank you to the contributors under CC BY-SA 4.0!