TFS-GENM：一种基于EEG多模态特征融合的驾驶行为分类方法

黄亚宁; 闫光辉; 常文文; 程文鑫; 吴佰靖

doi:10.12263/DZXB.20250482

您当前的位置：

首页 >

文章列表页 >

TFS-GENM：一种基于EEG多模态特征融合的驾驶行为分类方法

学术论文 | 更新时间：2026-02-10

- TFS-GENM：一种基于EEG多模态特征融合的驾驶行为分类方法
- TFS-GENM: An EEG Multi-Modal Feature Fusion-Based Driving Behavior Classification Method
- 电子学报 2025年53卷第11期页码：4051-4064
- 作者机构：
  
  1.兰州交通大学电子与信息工程学院，甘肃兰州 730070
  2.甘肃警察学院，甘肃兰州 730046
- 作者简介：
  
  [ "黄亚宁女，1994年7月出生于甘肃省陇南市.现为兰州交通大学电子与信息工程学院博士研究生，甘肃警察学院助教.主要研究方向为人工智能、脑机接口、驾驶行为分析、深度学习.E-mail: bellanwnu@126.com" ]
  [ "闫光辉男，1970年10月出生于河南省商丘市.现为兰州交通大学电子与信息工程学院教授、博士生导师.主要研究方向为人工智能、智慧交通、复杂网络分析、脑功能网络及脑电特征分析.中国电子学会会员编号：E190158600M.E-mail: ghyan@mail.lzjtu.cn" ]
- 基金信息：
  
  国家自然科学基金(62366028;62466032);甘肃省自然科学基金(24JRRA256);甘肃省教育厅青年博士支持项目(2023QB-038)
- DOI：10.12263/DZXB.20250482
  中图分类号： TP391;
- 收稿：2025-06-05，
  
  录用：2025-11-19，
  
  纸质出版：2025-11-25
- 稿件说明：
移动端阅览
黄亚宁, 闫光辉, 常文文, 等. TFS-GENM：一种基于EEG多模态特征融合的驾驶行为分类方法[J]. 电子学报, 2025, 53(11): 4051-4064.

HUANG Ya-ning, YAN Guang-hui, CHANG Wen-wen, et al. TFS-GENM: An EEG Multi-Modal Feature Fusion-Based Driving Behavior Classification Method[J]. Acta Electronica Sinica, 2025, 53(11): 4051-4064.
黄亚宁, 闫光辉, 常文文, 等. TFS-GENM：一种基于EEG多模态特征融合的驾驶行为分类方法[J]. 电子学报, 2025, 53(11): 4051-4064. DOI：10.12263/DZXB.20250482

HUANG Ya-ning, YAN Guang-hui, CHANG Wen-wen, et al. TFS-GENM: An EEG Multi-Modal Feature Fusion-Based Driving Behavior Classification Method[J]. Acta Electronica Sinica, 2025, 53(11): 4051-4064. DOI：10.12263/DZXB.20250482

摘要

在传统基于脑电信号（ElectroEncephaloGraphy，EEG）的驾驶行为检测技术中，不同维度特征的提取及融合方法对分类效果有很大的影响，现有方法多基于时域、频域等单一模态特征提取，没有充分利用非线性动力学分析和空间域分析方法，从而难以全面捕捉大脑不同区域和频段的有效特征，限制了识别效果.因此，本文针对性地提出一种结合时域、频域、空间域等多尺度特征，及图卷积神经网络（Graph Convolution neural Networks，GCN）和EEGNet双分支的多维特征融合模型，以提升分类效果.首先提取EEG原始信号的几何性质、频段分布情况，构建时域频域维度的特征；然后计算相位锁定值（Phase Locking Value，PLV）、相位滞后指数（Phase Lag Index，PLI）和互信息（Mutual Information，MI），度量不同状态下的脑网络连接，再使用GCN动态优化邻接矩阵、聚合节点信息，以构建空间域层面的特征；利用EEGNet提取局部的时空特征，增加了模型可解释性；得到多维特征数据后进行拼接融合和分类.本文模型基于公开数据集进行了各个维度的性能验证，达到95.87%以上的分类平均准确率，最高准确率达98.65%，相较当前最优分类结果提升了2.95%.该方法解决了因单一模态特征造成的分类效果不佳、鲁棒性不高等问题，为后续开发可穿戴设备智能驾驶系统提供了理论基础，特别是为驾驶过程中存在肢体操作困难的残障人士提供新型辅助技术路径.

Abstract

In traditional driving behavior detection technology based on electroencephalography (EEG)

the extraction and fusion methods of multi-dimensional features significantly affect classification performance. Existing approaches are predominantly based on single-modal feature extraction from time or frequency domains

failing to fully utilize nonlinear dynamics or spatial domain analysis. This limitation hinders the comprehensive capture of effective features across different brain regions and frequency bands

thus restricting recognition accuracy. To address this

we propose a multi-dimensional feature fusion model integrating multi-scale time-domain

frequency-domain

and spatial-domain features through dual branches utilizing graph convolutional neural networks (GCN) and EEGNet. First

we extract geometric properties and frequency band distributions from the raw EEG signals to construct time-frequency features. Next

brain network connectivity under different states is measured by calculating phase locking value (PLV)

phase lag index (PLI)

and mutual information (MI). Subsequently

GCN dynamically optimizes the adjacency matrix and aggregates node information to build spatial-domain features. EEGNet is then employed to extract local spatio-temporal features

enhancing model interpretability. The resulting multi-dimensional features are concatenated

fused

and classified. Our proposed model was evaluated across various dimensions on public datasets

achieving an average classification accuracy exceeding 95.87%

with a peak accuracy of 98.65%. This represents an improvement of 2.95% over the current state-of-the-art results. Our method effectively resolves the problems of suboptimal classification performance and low robustness stemming from reliance on single-modal features. This work provides a theoretical foundation for the development of wearable intelligent driving systems

particularly offering novel assistive technology pathways for individuals with disabilities who experience difficulties with physical vehicle operation during driving.

关键词

Keywords

references

吴宇轩 , 虞慧群 , 范贵生 . 基于误差补偿的多模态协同交通流预测模型 [J ] . 电子学报 , 2024 , 52 ( 8 ): 2878 - 2890 .

WU Y X , YU H Q , FAN G S . Multimodal cooperative traffic flow prediction model based on error compensation [J ] . Acta Electronica Sinica , 2024 , 52 ( 8 ): 2878 - 2890 . (in Chinese)

ZHANG C , ESKANDARIAN A . A survey and tutorial of EEG-based brain monitoring for driver state analysis [J ] . IEEE/CAA Journal of Automatica Sinica , 2021 , 8 ( 7 ): 1222 - 1242 .

YAO Z B , DING L , LIU Y D , et al . Enhancing emergency braking intention recognition accuracy based on EEG spatiotemporal activation mapping [C ] // 2024 International Symposium on Intelligent Robotics and Systems . Piscataway : IEEE , 2024 : 93 - 97 .

PARK H , OH T , KIM I . Effects of driver’s braking behavior by the real-time pedestrian scale warning system [J ] . Accident Analysis & Prevention , 2024 , 205 : 107685 .

TAN H Q , SUN D X , GUO H W , et al . LDIPRS: A novel longitudinal driving intention prior recognition technique empowered by TENG and deep learning [J ] . Nano Energy , 2024 , 129 : 110087 .

常文文 , 闫光辉 , 杨志飞 , 等 . 基于脑电熵值特征和功能连接的不同线型道路下驾驶状态检测 [J ] . 电子学报 , 2023 , 51 ( 10 ): 2874 - 2883 .

CHANG W W , YAN G H , YANG Z F , et al . Detection of driving state under different curve road based on entropy and functional connectivity of EEG [J ] . Acta Electronica Sinica , 2023 , 51 ( 10 ): 2874 - 2883 . (in Chinese)

LI X , ZHANG Y Z , TIWARI P , et al . EEG based emotion recognition: A tutorial and review [J ] . ACM Computing Surveys , 2022 , 55 ( 4 ): 1 - 57 .

HAUFE S , TREDER M S , GUGLER M F , et al . EEG potentials predict upcoming emergency brakings during simulated driving [J ] . Journal of Neural Engineering , 2011 , 8 ( 5 ): 056001 .

HAUFE S , KIM J W , KIM I H , et al . Electrophysiology-based detection of emergency braking intention in real-world driving [J ] . Journal of Neural Engineering , 2014 , 11 ( 5 ): 056011 .

HERNÁNDEZ L G , MOZOS O M , FERRÁNDEZ J M , et al . EEG-based detection of braking intention under different car driving conditions [J ] . Frontiers in Neuroinformatics , 2018 , 12 : 29 .

TENG T , BI L Z , LIU Y L . EEG-based detection of driver emergency braking intention for brain-controlled vehicles [J ] . IEEE Transactions on Intelligent Transportation Systems , 2018 , 19 ( 6 ): 1766 - 1773 .

XU C Y , XU L , HAN C L , et al . Detection of emergency braking intention based on pedal-switching process [J ] . IEEE Access , 2025 , 13 : 197150 - 197159 .

LIANG X B , YU Y , LIU Y D , et al . EEG-based emergency braking intention detection during simulated driving [J ] . BioMedical Engineering OnLine , 2023 , 22 ( 1 ): 65 .

JU J W , BI L Z , FELEKE A G . Noninvasive neural signal-based detection of soft and emergency braking intentions of drivers [J ] . Biomedical Signal Processing and Control , 2022 , 72 : 103330 .

袁月婷 , 闫光辉 , 常文文 , 等 . 基于脑电信号空域特征的紧急制动行为识别 [J ] . 电子科技大学学报 , 2024 , 53 ( 1 ): 84 - 91 .

YUAN Y T , YAN G H , CHANG W W , et al . Emergency braking behavior recognition based on spatial features of EEG [J ] . Journal of University of Electronic Science and Technology of China , 2024 , 53 ( 1 ): 84 - 91 . (in Chinese)

LIU T Q , QIN Y J , ZHANG S H , et al . A diffusion-based feature enhancement approach for driving behavior classification with EEG data [J ] . Advanced Engineering Informatics , 2025 , 65 : 103279 .

WEI M H , LIN F . A novel multi-dimensional features fusion algorithm for the EEG signal recognition of brain’s sensorimotor region activated tasks [J ] . International Journal of Intelligent Computing and Cybernetics , 2020 , 13 ( 2 ): 239 - 260 .

WANG Y M , ZHANG B , DI L M . Research progress of EEG-based emotion recognition: A survey [J ] . ACM Computing Surveys , 2024 , 56 ( 11 ): 1 - 49 .

STAM C J , NOLTE G , DAFFERTSHOFER A . Phase lag index: Assessment of functional connectivity from multi channel EEG and MEG with diminished bias from common sources [J ] . Human Brain Mapping , 2007 , 28 ( 11 ): 1178 - 1193 .

WANG Y D , WU Q F , WANG S C , et al . MI-EEG: Generalized model based on mutual information for EEG emotion recognition without adversarial training [J ] . Expert Systems with Applications , 2024 , 244 : 122777 .

KRASKOV A , STÖGBAUER H , GRASSBERGER P . Estimating mutual information [J ] . Physical Review E-Statistical, Nonlinear, and Soft Matter Physics , 2004 , 69 ( 6 ): 066138 .

XIONG H , YAN Y , CHEN Y M , et al . Graph convolution network-based eeg signal analysis: A review [J ] . Medical & Biological Engineering & Computing , 2025 , 63 ( 6 ): 1609 - 1625 .

CAO L W , ZHAO W F , SUN B . Emotion recognition using multi-scale EEG features through graph convolutional attention network [J ] . Neural Networks , 2025 , 184 : 107060 .

PAIN S , ROY S , SARMA M , et al . Detection of alcoholism by combining EEG local activations with brain connectivity features and graph neural network [J ] . Biomedical Signal Processing and Control , 2023 , 85 : 104851 .

KIPF T N , WELLING M . Semi-supervised classification with graph convolutional networks [EB/OL ] . ( 2017-02-22 )[ 2025-10-10 ] . https://arXiv.org/abs/1609.02907 https://arXiv.org/abs/1609.02907 .

HAMMOND D K , VANDERGHEYNST P , GRIBONVAL R . Wavelets on graphs via spectral graph theory [J ] . Applied and Computational Harmonic Analysis , 2011 , 30 ( 2 ): 129 - 150 .

LAWHERN V J , SOLON A J , WAYTOWICH N R , et al . EEGNet: A compact convolutional neural network for EEG-based brain-computer interfaces [J ] . Journal of Neural Engineering , 2018 , 15 ( 5 ): 056013 .

WANG H T , YUAN Z Z , ZHANG H Y , et al . Hybrid EEG-fNIRS decoding with dynamic graph convolutional-capsule networks for motor imagery/execution [J ] . Biomedical Signal Processing and Control , 2025 , 104 : 107570 .

HAGHANI M , BLIEMER M C J , FAROOQ B , et al . Applications of brain imaging methods in driving behaviour research [J ] . Accident Analysis & Prevention , 2021 , 154 : 106093 .

HUANG W Q , CHANG W W , YAN G H , et al . EEG-based motor imagery classification using convolutional neural networks with local reparameterization trick [J ] . Expert Systems with Applications , 2022 , 187 : 115968 .

HU M T , XU D , HE K J , et al . Cross-subject emotion recognition with contrastive learning based on EEG signal correlations [J ] . Biomedical Signal Processing and Control , 2025 , 104 : 107511 .

代艳莹 , 周鹏 , 姜劲 , 等 . 基于互信息的警觉度脑网络特性研究 [J ] . 载人航天 , 2024 , 30 ( 4 ): 426 - 433 .

DAI Y Y , ZHOU P , JIANG J , et al . Characteristics of alertness brain network based on mutual information [J ] . Manned Spaceflight , 2024 , 30 ( 4 ): 426 - 433 . (in Chinese)

WANG H K , BI L Z , TENG T . Neural correlates and detection of braking intention under critical situations based on the power spectra of electroencephalography signals [J ] . Science China Information Sciences , 2019 , 63 ( 1 ): 119202 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

多模态生理特征融合的驾驶行为识别研究

基于脑电熵值特征和功能连接的不同线型道路下驾驶状态检测

Ho³⁺和Er³⁺在YLiF₄晶体中的光谱强度和Ω_λ参数

完整大豆叶片中自由基对的相互作用

Na₅Eu(MO₄)₄(M=Mo,W)的B(λkq)强度参数和跃迁几率