In this paper, a novel Mach-Zehnder interferometer (MZI) structure combined with multi-mode interference has been proposed and applied to vibration detection. Different from the traditional MZI which is based on the cascade of two 3-dB single-mode optical coupler (OC), the MZI is composed of multi-mode OC and single-mode OC. A 3 dB multimode OC with the coupling ratio of 1:1 was fabricated by the fusing and tapering using the step index multimode fiber with a core diameter of 105μm and the graded index multimode fiber with a core diameter of 62.5μm, and the MZI was constructed with the multimode OC and another 3 dB single-mode OC respectively. For comparison, two single-mode OC were also used to construct the MZI. Vibration is applied to an interference arm of the MZIs constructed with different types of OC, and data acquisition and analysis are carried out by an oscilloscope. The results show that the MZI based on a step index multimode coupler with a core diameter of 105μm has the largest detectable vibration frequency range of 1-20 KHZ, and has the best performance in vibration frequency deviation and vibration frequency discrimination ability. This method has a wide range of vibration frequency, simple production and low cost, which provides a new idea for the development of vibration sensor.