Fast speaker adaptation is very important in application of speaker independent continuous speech recognition.Recently speaker adaptation methods almost just update mean vector.A new adaptation algorithm proposed in this paper can adapt covariance matrixes rapidly.Gaussian similarity is used for measuring the distance of different covariance.A binary decision tree is constructed with this measure.And each middle node includes a covariance matrix of cluster center.Lots of covariance matrixes corresponding to speaker dependent model are trained based on this tree.During adaptation

covariance matrixes are updated through linear interpolating those covariances of cluster center.It can be shown from the experiments that error rate is descended from 29.49% to 27.55% in case of just one adapted sentence.