Abstract : Millisecond pulsars (MSPs) have been firmly established as a class of gamma-ray emitters via the detection of pulsations above 0.1 GeV from eight MSPs by the Fermi Large Area Telescope (LAT). Using thirteen months of LAT data significant gamma-ray pulsations at the radio period have been detected from the MSP PSR J0034-0534, making it the ninth clear MSP detection by the LAT. The gamma-ray light curve shows two peaks separated by 0.274$\pm$0.015 in phase which are very nearly aligned with the radio peaks, a phenomenon seen only in the Crab pulsar until now. The $\geq$0.1 GeV spectrum of this pulsar is well fit by an exponentially cutoff power law with a cutoff energy of 1.8$\pm 0.6\pm$0.1 GeV and a photon index of 1.5$\pm 0.2\pm$0.1, first errors are statistical and second are systematic. The near-alignment of the radio and gamma-ray peaks strongly suggests that the radio and gamma-ray emission regions are co-located and both are the result of caustic formation.