Abstract: During the operation of the planetary gear box, the vibration signal comes mainly from the meshing vibration between the gears. The position between each meshing point and the sensor changes periodically, and each meshing vibration signal was collected by the sensor through different transmission paths at the same time, resulting in serious coupling of the signal collected by the sensor and very complicated, which is not conducive to effective failure diagnosis. To this end, considering the actual characteristics of signal acquisition, a rigid-flexible coupled dynamic simulation model was established by using PROE, ANSYS and dynamic simulation software ADAMS according to the laboratory equipment parameters. Combining with the time-varying dynamics model proposed by Inalpolat, and considering the local fault signal modulation, rotational modulation, the structure and constraint modulation of each gear of the gearbox, an improved time-varying dynamics model of local fault was established. By comparing the results of the improved local fault time-varying dynamic model and the simulation model, the characteristics of amplitude and side-frequency spacing in the planetary gearbox sideband signal were obtained, which verifies the consistency of the model and theory. Finally, compared with the experimental signal, the correctness of the conclusion was verified. The result of this study can provide an effective theory and engineering basis for planetary gearbox fault diagnosis.