一种多尺度前向注意力模型的语音识别方法

doi:10.3969/j.issn.0372-2112.2020.07.002

电子学报 ›› 2020, Vol. 48 ›› Issue (7): 1255-1260.DOI: 10.3969/j.issn.0372-2112.2020.07.002

一种多尺度前向注意力模型的语音识别方法

唐海桃, 薛嘉宾, 韩纪庆

哈尔滨工业大学计算机科学与技术学院, 黑龙江哈尔滨 150001

收稿日期:2019-06-06 修回日期:2020-02-29 出版日期:2020-07-25 发布日期:2020-07-25
通讯作者: 韩纪庆
作者简介:唐海桃 男,1994年9月出生于四川省广安市.现为哈尔滨工业大学计算机科学与技术专业硕士研究生,主要研究方向为语音识别.E-mail:tanghaitao_ape@163.com;薛嘉宾 男,1993年7月出生于内蒙古自治区包头市.现为哈尔滨工业大学计算机科学与技术专业博士研究生,主要研究方向为语音识别.E-mail:xuejiabin@hit.edu.cn
基金资助:
国家重点研发项目（No.2017YFB1002102）

A Method of Multi-Scale Forward Attention Model for Speech Recognition

TANG Hai-tao, XUE Jia-bin, HAN Ji-qing

School of Computer Science and Technology, Harbin Institute of Technology, Harbin, Heilongjiang 150001, China

Received:2019-06-06 Revised:2020-02-29 Online:2020-07-25 Published:2020-07-25

摘要/Abstract

摘要： 注意力模型是当前语音识别中的主流模型，然而其存在一个缺点，即当前时刻的注意力模型可能产生异常得分.为此，本文首先提出前向注意力模型，其采用上一时刻正常注意力得分平滑当前时刻异常得分.接着通过对上一时刻的注意力得分添加约束因子来对前向注意力模型进行优化，达到自适应平滑的目的.最后，在优化模型基础上提出多尺度前向注意力模型，其通过引入多尺度模型来对不同等级的语音基元进行建模，进而将所得到的不同等级目标向量进行融合，以达到解决注意力得分异常值的目的.采用SwitchBoard作为训练集，Hub5'00作为测试集进行实验，相比于基线系统，多尺度前向注意力模型的词错误率（Word Error Rate，WER）相对降低14.28%.

关键词: 前向注意力机制, 自适应平滑, 多尺度, 语音识别

Abstract: Attention-based model is a popular model in speech recognition,however it has a disadvantage that the attention-based model may produce abnormal scores.To solve this problem,this paper first proposes a forward attention model,which adopts normal attention score at the previous moment to smooth the abnormal score at the current moment.Then,the model is optimized to add constraint factors to the attention score at the previous moment to achieve the purpose of adaptive smoothing of the above abnormal scores.Then,a multi-scale forward attention model is proposed on the above model.This model introduces a multi-scale method to model the speech primitives of different levels,and then fuses the target vectors of different levels to solve the outliers of attention score.In the experiment,SwitchBoard is adopted as the training set and Hub5'00 as the test set.Compared with the baseline system,the Word Error Rate (WER) of the proposed system decreased by 14.28% relatively.

Key words: forward attention mechanism, adaptive smoothing, multi-scale, speech recognition

中图分类号:

TN912.34

唐海桃, 薛嘉宾, 韩纪庆. 一种多尺度前向注意力模型的语音识别方法[J]. 电子学报, 2020, 48(7): 1255-1260.

TANG Hai-tao, XUE Jia-bin, HAN Ji-qing. A Method of Multi-Scale Forward Attention Model for Speech Recognition[J]. Acta Electronica Sinica, 2020, 48(7): 1255-1260.

参考文献

[1] KRIZHEVSKY A,SUTSKEVER I,HINTON G E.Imagenet classification with deep convolutional neural networks[A].Advances in Neural Information Processing Systems[C].[S.l.]:NIPS,2012.1097-1105.
[2] CIRESAN D,GIUSTI A,et al.Deep neural networks segment neuronal membranes in electron microscopy images[A].Advances in neural information processing systems[C].[S.l.]:NIPS,2012.2843-2851.
[3] DAHL G E,YU D,DENG L,et al.Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition[J].IEEE Transactions on Audio,Speech,and Language Processing,2011,20(1):30-42.
[4] DENG L,HINTON G,KINGSBURY B.New types of deep neural network learning for speech recognition and related applications:An overview[A].2013 IEEE International Conference on Acoustics,Speech and Signal Processing[C].USA:IEEE,2013.8599-8603.
[5] COLLOBERT R,WESTON J.A unified architecture for natural language processing:Deep neural networks with multitask learning[A].Proceedings of the 25th International Conference on Machine Learning[C].USA:ACM,2008.160-167.
[6] HIRSCHBERG J,et al.Advances in natural language processing[J].Science,2015,349(6245):261-266.
[7] BAHDANAU D,CHOROWSKI J,et al.End-to-end attention-based large vocabulary speech recognition[A].2016 IEEE International Conference on Acoustics,Speech and Signal Processing[C].USA:IEEE,2016.4945-4949.
[8] GAO F C,XIN L I,YONG H Y.Using highway connections to enable deep small-footprint LSTM-RNNs for speech recognition[J].Chinese Journal of Electronics,2019,28(1):107-112.
[9] ZEYER A,IRIE K,et al.Improved training of end-to-end attention models for speech recognition[A].Interspeech[C].Hyderabad:[s.n.],2018.1845-1859.
[10] MERBOLDT A,ZEYER A,et al.An analysis of local monotonic attention variants[A].Interspeech[C].Graz:[s.n.],2019.1398-1402.
[11] BAHAR P,ZEYER A,et al.On using 2D sequence-to-sequence models for speech recognition[A].IEEE International Conference on Acoustics,Speech and Signal Processing[C].USA:IEEE,2019.5671-5675.
[12] ZWEIG G,et al.Advances in all-neural speech recognition[A].IEEE International Conference on Acoustics,Speech and Signal Processing[C].USA:IEEE,2017.4805-4809.
[13] CHOROWSKI J,BAHDANAU D,CHO K,et al.End-to-end continuous speech recognition using attention-based recurrent nn:First results[J].Eprint Arxiv,2014.
[14] CHAN W,et al.Listen,attend and spell:A neural network for large vocabulary conversational speech recognition[A].IEEE International Conference on Acoustics,Speech and Signal Processing[C].USA:IEEE,2016.4960-4964.
[15] BAHDANAU D,CHOROWSKI J,et al.End-to-end attention-based large vocabulary speech recognition[A].IEEE International Conference on Acoustics,Speech and Signal Processing[C].USA:IEEE,2016.4945-4949.
[16] MARTINS A,ASTUDILLO R.From softmax to sparsemax:A sparse model of attention and multi-label classification[A].International Conference on Machine Learning[C].[S.l.]:[s.n.],2016.1614-1623.
[17] KIM Y.Convolutional neural networks for sentence classification[A].Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)[C].USA:Association for Computational Linguistics,2014.1746-1751.
[18] SUTSKEVER I,VINYALS O,LE Q V.Sequence to sequence learning with neural networks[A].Advances in Neural Information Processing Systems[C].[S.l.]:NIPS,2014.3104-3112.
[19] CHOROWSKI J K,et al.Attention-based models for speech recognition[A].Advances in Neural Information Processing Systems[C].[S.l.]:NIPS,2015.577-585.
[20] GODFREY J J,HOLLIMAN E C,MCDANIEL J.SWITCHBOARD:Telephone speech corpus for research and development[A].IEEE International Conference on Acoustics,Speech,and Signal Processing[C].USA:IEEE,1992.517-520.
[21] WENBIN J,PEILIN L,FEI W.Speech magnitude spectrum reconstruction from MFCCs using deep neural network[J].Chinese Journal of Electronics,2018,27(2):393-398.
[22] WEN M C,TIAN C H.The multi-weight neuron with geometry algorithm and its application[J].Chinese Journal of Electronics,2008,17(2):261-264.
[23] JI X U,JIE L P,YONG H Y.Agglutinative language speech recognition using automatic allophone deriving[J].Chinese Journal of Electronics,2016,25(2):134-139.
[24] KIM S,HORI T.Joint CTC-attention based end-to-end speech recognition using multi-task learning[A].IEEE International Conference on Acoustics,Speech and Signal Processing[C].USA:IEEE,2017.4835-4839.
[25] WATANABE S,HORI T,KIM S,et al.Hybrid CTC/attention architecture for end-to-end speech recognition[J].IEEE Journal of Selected Topics in Signal Processing,2017,11(8):1240-1253.

一种多尺度前向注意力模型的语音识别方法

A Method of Multi-Scale Forward Attention Model for Speech Recognition

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	王相海, 宋若曦, 曲思洁, 穆振华, 宋传鸣. 图像多尺度几何分析域隐马尔可夫树模型研究进展[J]. 电子学报, 2022, 50(1): 238-249.
[2]	李宝奇, 黄海宁, 刘纪元, 李宇. 基于改进CycleGAN的光学图像迁移生成水下小目标合成孔径声纳图像算法研究[J]. 电子学报, 2021, 49(9): 1746-1753.
[3]	杨红红, 王刘丽, 张玉梅, 吴晓军, 党允彤. 基于序列多尺度特征融合表示的层级舞蹈动作姿态估计方法[J]. 电子学报, 2021, 49(12): 2428-2436.
[4]	吕坤儒, 吴春国, 梁艳春, 袁宇平, 任智敏, 周柚, 时小虎. 融合语言模型的端到端中文语音识别算法[J]. 电子学报, 2021, 49(11): 2177-2185.
[5]	张荣国, 贾玉闪, 胡静, 刘小君, 李晓明. 超像素内容感知先验的多尺度贝叶斯显著性检测方法[J]. 电子学报, 2020, 48(8): 1509-1515.
[6]	周伯阳, 郭志民, 王延松, 阮伟, 吴春明, 周宁, 张伟, 程国振. 基于多尺度低秩模型的电力无线接入网异常流量检测方法[J]. 电子学报, 2020, 48(8): 1552-1557.
[7]	吴文, 万毅. 基于低尺度细节恢复的单幅图像阴影去除方法[J]. 电子学报, 2020, 48(7): 1293-1302.
[8]	秦福强, 王丽芳. 基于变分的多尺度遥感图像融合算法[J]. 电子学报, 2020, 48(6): 1084-1090.
[9]	石鹏, 张启忠, 张华平, 席旭刚. 基于多元经验模态分解的多元多尺度熵静态平衡能力评估[J]. 电子学报, 2020, 48(4): 670-674.
[10]	李雅倩, 盖成远, 肖存军, 吴超, 刘佳甲. 基于细化多尺度深度特征的目标检测网络[J]. 电子学报, 2020, 48(12): 2360-2366.
[11]	欧先锋, 晏鹏程, 王汉谱, 涂兵, 何伟, 张国云, 徐智. 基于深度帧差卷积神经网络的运动目标检测方法研究[J]. 电子学报, 2020, 48(12): 2384-2393.
[12]	张学军, 景鹏, 何涛, 孙知信. 基于变分模态分解的癫痫脑电信号分类方法[J]. 电子学报, 2020, 48(12): 2469-2475.
[13]	周德龙, 张捷, 朱思聪. 多方向多尺度Gabor特征表示及其匹配算法[J]. 电子学报, 2019, 47(9): 1998-2002.
[14]	周岩, 王鹏, 辛罡, 李波, 王德志. MQHOA优化算法能级稳定过程及判据研究[J]. 电子学报, 2019, 47(6): 1337-1343.
[15]	魏亭, 邱实, 李晨, 王锐. 计算机多尺度辅助定位车牌算法[J]. 电子学报, 2018, 46(9): 2188-2193.