研究回馈神经网络(RNN)参数估计的新方法.利用隐含观测量,将复杂RNN的训练分解为线性输出层和多个单隐元的参数估计.基于每个隐元激励函数的多点线性近似,RNN可利用统计混合专家网络模型(ME)描述,从而将RNN的参数估计转化为包含隐含观测量的线性系统的最大似然估计问题,最后利用期望最大化(EM)算法获得RNN的隐含观测量及其参数估计.

A new method for training recurrent neural network (RNN) has been proposed.By introducing the hidden representation or hidden variables into RNN,training the complicated RNN is decomposed into training a set of single neurons and a linear output layer.Based on linear approximation of RNN hidden units,RNN is remodeled with a "mixture of experts"(ME) model.Morever,training RNN is also changed into a maximum likelihood estimation of the linear systems with hidden variables.Finally,training RNN is fulfilled with the expectation-maximization (EM) algorithm.