基于深度主题模型的飞行员脑疲劳检测

doi:10.12263/DZXB.20201267

电子学报 ›› 2022, Vol. 50 ›› Issue (8): 1801-1810.DOI: 10.12263/DZXB.20201267

基于深度主题模型的飞行员脑疲劳检测

吴奇¹^,²^,³, 陈琪琦¹, 彭献永⁴, 仇峰¹

^1.上海交通大学自动化系，上海 200240
^2.系统控制与信息处理教育部重点实验，上海 200240
^3.上海工业智能管控工程技术研究中心，上海 200240
^4.中国矿业大学低碳能源与动力工程学院，江苏徐州 221116

收稿日期:2020-11-09 修回日期:2021-10-18 出版日期:2022-08-25
- 通讯作者:
- 彭献永
- 作者简介:
- 吴奇男，2009年于东南大学控制理论与工程专业获工学博士学位.上海交通大学系统控制与信息处理教育部重点实验室教授.研究方向为空战神经生理认知学.E-mail: Edmondqwu@gmail.com.
  陈琪琦女，2022年获上海交通大学工学学士学位.主要研究方向为深度学习、脑影像分析.E-mail: cqq.jx@sjtu.edu.cn.
  彭献永男，2011年获东南大学博士学位.中国矿业大学低碳能源与动力工程学院副教授.主要研究方向为智能电厂、状态监测与先进控制.
  仇峰男，2020年获上海通大学硕士学位.主要研究方向为数字电厂、认知计算.E-mail: edmondqwu@sjtu.edu.cn.
- 基金资助:
- 国家自然科学基金 (U1933125)

Pilot’s Brain Fatigue Detection Based on Deep Topic Model

WU Qi¹^,²^,³, CHEN Qi-qi¹, PENG Xian-yong⁴, QIU Feng¹

^1.Department of Automation，Shanghai Jiao Tong University，Shanghai 200240，China
^2.Key Laboratory of System Control and Information Processing，Ministry of Education，Shanghai 200240，China
^3.Shanghai Engineering Research Center of Intelligent Control and Management，Shanghai 200240，China
^4.School of Low-Carbon Energy and Power Engineering，China University of Mining and Technology，Xuzhou，Jiangsu 221116，China

Received:2020-11-09 Revised:2021-10-18 Online:2022-08-25 Published:2022-09-08
- Corresponding author:
- PENG Xian-yong

摘要/Abstract

摘要：

飞行员脑疲劳状态检测需要解决脑认知图谱生成和脑疲劳检测模型构建问题.针对第一个问题，本文通过等距方位投影法将全脑电极位置的脑疲劳指标映射为二维脑功率图谱，形成一种新型脑认知图谱.针对第二个问题，本文建立一种深度主题学习模型，即深度潜狄利克雷模型（Deep Latent Dirichlet Model，DLDM），解决了飞行员疲劳状态主题学习问题.DLDM深度模型通过多项式分布逐层扩展脑功率图谱中蕴含的概率分布信息，推理脑功率图谱的层次概率分布特征，实现更有效的飞行员疲劳状态主题学习.同时为了避免启发式假设，本文提出一种有效的不同层与主题间自适应学习率的随机梯度下降推断方法，更加高效地推理DLDM网络结构参数.实验结果显示，DLDM网络可以逐层扩展脑功率图谱中蕴含的概率分布信息，推理出更丰富的抽象特征信息，实现脑疲劳认知主题学习.对比其他脑疲劳检测方法，本文方法分类精度可提升2%.

关键词: 脑功率图谱, 疲劳认知状态, 主题学习, 深度模型, 概率推断, 狄利克雷模型

Abstract:

The detection of a pilot's brain fatigue state faces two important problems, which are how to generate a brain cognitive map and how to build a brain fatigue detection model. To solve the first problem, this paper uses the isometric azimuth projection to map brain fatigue indicators into a new type of brain power map. To solve the second problem, this work develops a deep latent Dirichlet model(DLDM), which solves the topic detection problem of pilot fatigue state. DLDM expands the probability distribution information contained in the developed brain power map layer by layer through multiple distributions, infers their hierarchical probability distribution characteristics, and gets more effective topic detection accuracy of pilot fatigue state. In order to avoid heuristic assumptions, this paper also proposes an effective stochastic gradient descent inference method with an adaptive learning rate between different layers and topics to more efficiently infer structure parameters of DLDM. The experimental results show that the DLDM can expand the probability distribution information of the brain power map layer by layer, infer richer abstract feature information, and detect brain fatigue cognitive topic. Compared with other brain fatigue detection methods, the classification accuracy of the proposed method can be improved by 2%.

Key words: brain power map, fatigue cognitive state, topic learning, deep model, probability inference, Dirichlet model

中图分类号:

TP181

吴奇, 陈琪琦, 彭献永, 等. 基于深度主题模型的飞行员脑疲劳检测[J]. 电子学报, 2022, 50(8): 1801-1810.

Qi WU, Qi-qi CHEN, Xian-yong PENG, et al. Pilot’s Brain Fatigue Detection Based on Deep Topic Model[J]. Acta Electronica Sinica, 2022, 50(8): 1801-1810.

图/表 10

参考文献 30

1	MA J, ZHANG J, GONG Z Y, et al. Study on fatigue driving detection model based on steering operation features and eye movement features[C]//2018 IEEE 4th International Conference on Control Science and Systems Engineering. Wuhan, China: IEEE, 2018: 472-475.
2	SETIAWAN A, WIBAWA A D, PANE E S, et al. EEG-based mental fatigue detection using cognitive tests and RVM classification[C]//2019 International Conference of Artificial Intelligence and Information Technology(ICAIIT). Yogyakarta, Indonesia: IEEE, 2019: 180-185.
3	KARUPPUSAMY N S, KANG B Y. Multimodal system to detect driver fatigue using EEG, gyroscope, and image processing[J]. IEEE Access, 2020, 8: 129645-129667.
4	JAP B T, LAL S, FISCHER P, et al. Using EEG spectral components to assess algorithms for detecting fatigue[J]. Expert Systems With Applications, 2009, 36(2): 2352-2359.
5	SIEMIONOW V, FANG Y, CALABRESE L, et al. Altered central nervous system signal during motor performance in chronic fatigue syndrome[J]. Clinical Neurophysiology, 2004, 115(10): 2372-2381.
6	PAPADELIS C, KOURTIDOU-pAPADELI C, BAMIDIS P D, et al. Indicators of sleepiness in an ambulatory EEG study of night driving[C]//2006 International Conference of the IEEE Engineering in Medicine and Biology Society. New York, NY, USA: IEEE, 2006: 6201-6204.
7	WU Q, DENG P Y, QIU X Y, et al. Detecting fatigue status of pilots based on deep learning network using EEG signals[J]. IEEE Transactions on Cognitive and Developmental Systems, 2021, 13(3): 575-585.
8	MING Y R, WU D R, WANG Y K, et al. EEG-based drowsiness estimation for driving safety using deep Q-learning[J]. IEEE Transactions on Emerging Topics in Computational Intelligence, 2021, 5(4): 583-594.
9	LIN C T, CHUANG C H, HUNG Y C, et al. A driving performance forecasting system based on brain dynamic state analysis using 4-D convolutional neural networks[J]. IEEE Transactions on Cybernetics, 2021, 51(10): 4959-4967.
10	DU G L, WANG Z Y, LI C Q, et al. A TSK-type convolutional recurrent fuzzy network for predicting driving fatigue[J]. IEEE Transactions on Fuzzy Systems, 2021, 29(8): 2100-2111.
11	ZHENG W L, LU B L. Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks[J]. IEEE Transactions on Autonomous Mental Development, 2015, 7(3): 162-175.
12	LI J H, STRUZIK Z, ZHANG L Q, et al. Feature learning from incomplete EEG with denoising autoencoder[J]. Neurocomputing, 2015, 165: 23-31.
13	CECOTTI H, GRAESER A. Convolutional neural network with embedded fourier transform for EEG classification[C]//2008 19th International Conference on Pattern Recognition. Tampa, FL, USA: IEEE, 2008: 1-4.
14	ALHUSSEIN M, MUHAMMAD G, Hossain M S. EEG pathology detection based on deep learning[J]. IEEE Access, 2019, 7: 27781-27788.
15	SONG Y G, WANG D L, YUE K, et al. EEG-based motor imagery classification with deep multi-task learning[C]//2019 International Joint Conference on Neural Networks(IJCNN). Budapest, Hungary: IEEE, 2019: 1-8.
16	BHARDWAJ R, PARAMESWARAN S, BALASUBRAMANIAN V. Performance comparison of machine learning and deep learning while classifying driver's cognitive state[C]//2018 IEEE 13th International Conference on Industrial and Information Systems. Rupnagar, India: IEEE, 2018: 89-93.
17	孙曜, 文成林, 韦巍. 基于脑电和眼电的运动想象多尺度识别方法研究[J]. 电子学报, 2018, 46(3): 714-720.
	SUN Y, WEN C L, WEI W. Research on EEG and EOG based multiscale recognization method of motor imagery[J]. Acta Electronica Sinica, 2018, 46(3): 714-720. (in Chinese)
18	张学军, 景鹏, 何涛, 等. 基于变分模态分解的癫痫脑电信号分类方法[J]. 电子学报, 2020, 48(12): 2469-2475.
	ZHANG X J, JING P, HE T, et al. An epileptic electroencephalogram signal classification method based on variational mode decomposition[J]. Acta Electronica Sinica, 2020, 48(12): 2469-2475. (in Chinese)
19	AHMED S, MAURICIO MERINO L, MAO Z, et al. A deep learning method for classification of images RSVP events with EEG data[C]//2013 IEEE Global Conference on Signal and Information Processing. Austin, TX, USA: IEEE, 2013: 33-36.
20	GAO Y B, LEE H J, MEHMOOD R M. Deep learninig of EEG signals for emotion recognition[C]//2015 IEEE International Conference on Multimedia & Expo Workshops. Turin, Italy: IEEE, 2015: 1-5.
21	YANG Y X, GAO Z K, LI Y L, et al. A complex network-based broad learning system for detecting driver fatigue from EEG signals[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2021, 51(9): 5800-5808.
22	ZHANG C, SUN L N, CONG F Y, et al. Spatiotemporal dynamical analysis of brain activity during mental fatigue process[J]. IEEE Transactions on Cognitive and Developmental Systems, 2021, 13(3): 593-606.
23	WU Q, ZHU L M, LI G J, et al. Nonparametric hierarchical hidden semi-Markov model for brain fatigue behavior detection of pilots during flight[J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 23(6): 5245-5256.
24	CONG Y L, CHEN B, LIU H W, et al. Deep latent dirichlet allocation with topic-layer-adaptive stochastic gradient Riemannian MCMC[C]//ICML'17: Proceedings of the 34th International Conference on Machine Learning-Volume 70. Sydney, NSW, Australia: ACM. 2017: 864-873.
25	GOLUB G H, C F VAN LOAN. Matrix Computations[M]. Washington, D.C, USA: Johns Hopkins University Press, 2012.
26	WU X, SPALL J C. Improved Monte Carlo estimation of the fisher information matrix with independent perturbations[C]//2021 55th Annual Conference on Information Sciences and Systems(CISS). Baltimore, MD, USA: IEEE, 2021: 1-5.
27	SNYDER J P. Map Projections--A working Manual[M]. Washington, DC, USA: US Government Printing Office, 1987: 55-90.
28	BENGIO Y, LAMBLIN P, POPOVICI D, et al. Greedy layer-wise training of deep networks[C]//Advances in Neural Information Processing Systems 19: Proceedings of the 2006 Conference. Vancouver, B.C., Canada: MIT Press, 2007: 153-160.
29	AMARI S I. Natural gradient works efficiently in learning[J]. Neural Computation, 1998, 10(2): 251-276.
30	LAWHERN V J, SOLON A J, WAYTOWICH N R, et al. EEGNet: A compact convolutional neural network for EEG-based brain-computer interfaces[J]. Journal of Neural Engineering, 2018, 15(5): 056013.

脑电信号图像化形式	训练集正确率/%	验证集正确率/%
原始信号	80.12	67.42
Alpha	83.21	73.46
Theta	78.52	64.23
脑功率图谱	92.56	83.95

脑电信号图像化形式	训练集正确率/%	验证集正确率/%
原始信号	80.12	67.42
Alpha	83.21	73.46
Theta	78.52	64.23
脑功率图谱	92.56	83.95

参数推断方法	网络架构	分类精度/%	运行用时/s
SGRLD	128	88.209	9 306
SGRLD	128-64	89.403	65 222
SGRLD	128-64-32	90.298	74 281
TLFSGR	128	87.015	9 179
TLFSGR	128-64	88.952	23 041
TLFSGR	128-64-32	89.104	48 901
TLASGR	128	87.911	17 101
TLASGR	128-64	89.105	24 181
TLASGR	128-64-32	89.152	52 260

参数推断方法	网络架构	分类精度/%	运行用时/s
SGRLD	128	88.209	9 306
SGRLD	128-64	89.403	65 222
SGRLD	128-64-32	90.298	74 281
TLFSGR	128	87.015	9 179
TLFSGR	128-64	88.952	23 041
TLFSGR	128-64-32	89.104	48 901
TLASGR	128	87.911	17 101
TLASGR	128-64	89.105	24 181
TLASGR	128-64-32	89.152	52 260

参数方法	网络结构	第三层分类精度/%	第二层分类精度/%	第一层分类精度/%
SGRLD	128-64	—	89.10	84.63
SGRLD	128-64-32	90.30	87.16	81.05
TLFSGR	128-64	—	88.95	83.73
TLFSGR	128-64-32	89.10	78.83	36.57
TLASGR	128-64	—	89.11	78.06
TLASGR	128-64-32	89.15	82.69	79.40

基于深度主题模型的飞行员脑疲劳检测

Pilot’s Brain Fatigue Detection Based on Deep Topic Model

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 10

参考文献 30

相关文章 15

编辑推荐

Metrics

[1]	许新征, 李杉. 基于特征膨胀卷积模块的轻量化技术研究[J]. 电子学报, 2023, 51(2): 355-364.
[2]	张重生, 王斌. 基于序列相似性计算的甲骨残片缀合算法[J]. 电子学报, 2023, (): 1-10.
[3]	徐佳伟, 罗倩. 基于遗传非参数MDL-BW方法的HMM结构优化[J]. 电子学报, 2022, 50(11): 2765-2772.
[4]	刘浩阳, 林耀进, 刘景华, 吴镒潾, 毛煜, 李绍滋. 由粗到细的分层特征选择[J]. 电子学报, 2022, 50(11): 2778-2789.
[5]	汪成亮, 赵凯, 刘嘉敏. 智能环境下基于边缘设备规则推理的数据预部署研究[J]. 电子学报, 2022, 50(10): 2347-2360.
[6]	高雷阜, 张梦瑶, 赵世杰. 融合簇边界移动与自适应合成的混合采样算法[J]. 电子学报, 2022, 50(10): 2517-2529.
[7]	李青青, 马慧芳, 李举, 李志欣, 姜彦斌. 属性网络中结合用户偏好的社区搜索和离群点检测[J]. 电子学报, 2022, 50(9): 2172-2180.
[8]	王雪松, 张翰林, 程玉虎. 基于自编码器和超图的半监督宽度学习系统[J]. 电子学报, 2022, 50(3): 533-539.
[9]	高俊涛, 王梅, 徐光会, 刘聪. 正则语言推断综述[J]. 电子学报, 2021, 49(12): 2479-2489.
[10]	孟海宁, 冯锴, 朱磊, 张贝贝, 童新宇, 黑新宏. 基于Laplacian图谱的短文本聚类算法[J]. 电子学报, 2021, 49(9): 1716-1723.
[11]	孙新, 盖晨, 申长虹, 张颖捷. 基于短语向量和主题加权的关键词抽取方法[J]. 电子学报, 2021, 49(9): 1682-1690.
[12]	周东明, 张灿龙, 李志欣, 王智文. 基于多层级视觉融合的图像描述模型[J]. 电子学报, 2021, 49(7): 1286-1290.
[13]	唐焕玲, 郑涵, 刘艳红, 马思源, 窦全胜, 鲁明羽. Tr-SLDA:一种面向交叉领域的迁移主题模型[J]. 电子学报, 2021, 49(3): 605-613.
[14]	金苍宏, 董腾然, 陈天翼, 吴明晖, 李国军, 周胜利. 融合序列分解与时空卷积的时序预测算法[J]. 电子学报, 2021, 49(2): 233-238.
[15]	汪成亮, 郑诚, 曾卓. 基于软件定义智能的睡眠动作识别[J]. 电子学报, 2021, 49(1): 85-89.

模型	分类准确率/%
DLDM+TLASGR	89.11
DBN	86.06
CNN^[8]	86.23
EEGNet^[23]	87.06

模型	分类准确率/%
DLDM+TLASGR	89.11
DBN	86.06
CNN^[8]	86.23
EEGNet^[23]	87.06