Abstract : We consider the most general, classically-conformal, three-dimensional $\mathcal{N}=1$ Chern-Simons-matter theory with global symmetry $Sp(2)$ and gauge group $U(N)\times U(N)$. We show that the Lagrangian in the on-shell formulation of the theory admits one more free parameter as compared to the theory formulated in off-shell $\mathcal{N}=1$ superspace. The theory on $T^3$ can be formally localized to a matrix model. We carry out the localization procedure for the theory on $T^3$ with periodic boundary conditions. In particular we show that restricting to the saddle points with vanishing gauge connection gives a trivial contribution to the partition function, i.e. the bosonic and fermionic contributions exactly cancel each other.