To solve the problem of multi-target tracking model with the time-varying number of targets
a novel Gaussian mixture PHD filter is proposed for the nonlinear Gaussian system.A third-degree Spherical-Radial rule is applied to calculate the prediction and update distributions of target states for nonlinear multi-target models.The pruning method is optimized by using a fuzzy threshold to avoid the exponential increasing of the Gaussian components.The measurements are used to generate the density of new targets that appear randomly anywhere in the observation space.The performance of the four nonlinear Gaussian Mixture PHD filters is compared.The simulation results demonstrated the efficiency of the proposed algorithm.