$\beta$-Thorium phosphate diphosphate ($\beta$-TPD), considered as a very promising radwaste storage material, was obtained from thorium phosphate hydrogenphosphate hydrate (TPHPH) precursor through dehydration and hydrogen phosphate condensation. The structures of TPHPH, intermediate $\alpha$-thorium phosphate diphosphate ($\alpha$-TPD) and its hydrate ($\alpha$-TPDH) have been resolved ab initio by Rietveld analysis of their synchrotron diffraction patterns. All were found orthorhombic (space group Cmcm) and similarly composed of [ThPO$_4$]$_4^{4+}$ slabs alternating with disordered layers hosting either [HPO$_4$·H$_2$O]$_2^{4-}$ (TPHPH), [P$_2$O$_7$·2H$_2$O]$^{4-} ($\alpha$-TPDH), or [P$_2O_7$]$^{4-}$ ($\alpha$-TPD), unlike the 3D structure of $\beta$-TPD. The diphosphate groups of $\alpha$-TPD and $\alpha$-TPDH are strongly bent. The irreversible transition to the final $\beta$-TPD consists in a shearing of the slabs and a reduction of the interslabs cavities that explains the stability of this high-temperature form.