Facial Action Units (FAU) tracking is a hard problem for the rigid and non-rigid transformations of human face.The constantly trembling in the tracking result and the tracking failures caused by the absence of constraint remain open problems.This paper presents a novel method to attack these problems by combining Gaussian Process and Particle Filtering.Gradient-based method and particle filtering based method are compared with our method and the experiment results are encouraging.