In order to realize nonlinear model identification of optical antenna servo system, an elastic deformation friction model is proposed to describe the nonlinear effect of the system The elastic deformation friction model considers that the friction force is originated by the elastic deformation of the asperities of the contacting surfaces. The contact behaves as a spring, and the contact deformation changes continuously during relative motion eliminating the discontinuity at zero velocity of Stribeck model. Compared with the LuGre model, this model has fewer parameters and does not need to solve differential equations, which reduces the difficulty of parameter identification. Then, the parameter identification of the three models is realized by using the genetic algorithm with the square sum of the error between the model and the measured data of servo system as the evaluation function. Experimental results indicate that:under the same genetic evolution process, the accuracy of the elastic deformation friction model is 2.3 times higher than that of the Stribeck model, and 4.6 times higher than that of the LuGre model. The results verify the feasibility of the elastic deformation friction model for nonlinear modeling of servo system.