Research on Dynamic Continuous Emotional Recognition of EEG Based on Improved TCNN Algorithm

JIE Li-lin; LIU Yong; WANG Ming-xun; ZOU Yang-meng; XU Yi-lu; LU Yu-ming

doi:10.12263/DZXB.20240997

您当前的位置：

首页 >

文章列表页 >

Research on Dynamic Continuous Emotional Recognition of EEG Based on Improved TCNN Algorithm

PAPERS | 更新时间：2025-07-24

- Research on Dynamic Continuous Emotional Recognition of EEG Based on Improved TCNN Algorithm
- ACTA ELECTRONICA SINICA Vol. 53, Issue 4, Pages: 1347-1360(2025)
- 作者机构：
  
  1.南昌航空大学仪器科学与光电工程学院，江西南昌 330063
  2.南昌航空大学航空服务与音乐学院，江西南昌 330063
  3.江西农业大学软件学院，江西南昌 330045
  4.南昌航空大学航空制造与机械工程学院，江西南昌 330063
- 作者简介：
- 基金信息：
  
  National Natural Science Foundation of China(62166020);Natural Science Foundation of Jiangxi(20242BAB25094;20224BAB206089)
- DOI：10.12263/DZXB.20240997
  CLC： TP391;
- Received：05 November 2024，
  
  Revised：2025-03-11，
  
  Published：25 April 2025
- 稿件说明：
移动端阅览
揭丽琳, 刘勇, 王铭勋, 等. 基于改进TCNN算法的脑电动态连续情绪识别研究[J]. 电子学报, 2025, 53(04): 1347-1360.

JIE Li-lin, LIU Yong, WANG Ming-xun, et al. Research on Dynamic Continuous Emotional Recognition of EEG Based on Improved TCNN Algorithm[J]. Acta Electronica Sinica, 2025, 53(04): 1347-1360.
揭丽琳, 刘勇, 王铭勋, 等. 基于改进TCNN算法的脑电动态连续情绪识别研究[J]. 电子学报, 2025, 53(04): 1347-1360. DOI：10.12263/DZXB.20240997

JIE Li-lin, LIU Yong, WANG Ming-xun, et al. Research on Dynamic Continuous Emotional Recognition of EEG Based on Improved TCNN Algorithm[J]. Acta Electronica Sinica, 2025, 53(04): 1347-1360. DOI：10.12263/DZXB.20240997

摘要

在现实生活中，人类情绪具有动态和多样化的特征，受外部环境、社交互动以及个体内在状态的共同影响.针对脑电情绪识别研究通常局限于实验室的静态场景，未能充分考虑情绪的动态连续性的问题，本文提出了一种基于改进TCNN算法的脑电动态连续情绪识别方法.首先，设计了适用于动态情境的脑电数据采集范式，使用64通道的脑电设备收集24名受试者在经历开心至平静、平静至开心、平静至悲伤、悲伤至平静、平静至紧张和紧张至平静六种动态连续情绪转变时的脑电信号，并进行了动态连续情绪标签的标注.其次，对现有的TCNN算法进行了改进，构建了一种双流网络模型进行动态连续情绪识别.该模型通过短期流利用时序卷积模块捕捉局部时间序列特征，而长期流则通过Transformer模块捕捉全局时间序列特征.最后，对提取的脑电特征进行特征层融合，以获得更加精准的动态连续情绪识别结果.结果表明：在采集的动态连续情绪数据集上，本文方法在六种情绪的valence和arousal上分别取得了最小误差均值0.083和0.084；在DEAP数据集上，valence和arousal的误差分别低至0.108和0.113.与四种传统机器学习算法以及GRU、CGRU、CNN、CNN-LSTM、CNN-Bi-LSTM、TCNN等六种深度学习模型相比，本文方法表现出了更高的识别精度和稳定性，能够有效满足应用场景的需求.

Abstract

In real life

human emotions possess dynamic and diverse characteristics

influenced by external environments

social interactions

and an individual’s internal state. Given that EEG emotion recognition research is often confined to static laboratory scenarios and fails to adequately consider the dynamic continuity of emotions

this paper proposes a novel method for dynamic continuous emotional recognition of EEG based on an improved TCNN algorithm. Firstly

an EEG acquisition paradigm suitable for dynamic scenarios was designed. A 64-channel EEG device was used to collect EEG signals from 24 subjects experiencing six types of dynamic emotional transitions: happy to calm

calm to happy

calm to sad

sad to calm

calm to tense

and tense to calm. Dynamic continuous emotional labels are also annotated for these signals. Secondly

the existing TCNN algorithm is improved to construct a dual-stream network model for dynamic continuous emotion recognition. This model captures local temporal features through a short-term stream utilizing a time-series convolutional module

while the long-term stream captures global temporal features via a Transformer module. Lastly

feature-level fusion of the extracted EEG features is performed to achieve more accurate dynamic continuous emotion recognition results. The results show that

on the collected dataset

the proposed method achieves the smallest mean errors of 0.083 and 0.084 for valence and arousal across six emotions

respectively. On the DEAP dataset

the errors for valence and arousal are reduced to 0.108 and 0.113

respectively. Moreover

compared to four traditional machine learning methods and six deep learning approaches including GRU

CGRU

CNN

CNN-LSTM

CNN-Bi-LSTM

and TCNN

the proposed method demonstrats higher recognition accuracy and stability

effectively meeting the requirements of application scenarios.

关键词

Keywords

references

CHENG P , LANGEVIN R . Difficulties with emotion regulation moderate the relationship between child maltreatment and emotion recognition [J ] . Child Abuse & Neglect , 2023 , 139 : 106094 .

YILDIRIM-CELIK H , EROGLU S , OGUZ K , et al . Emotional context effect on recognition of varying facial emotion expression intensities in depression [J ] . Journal of Affective Disorders , 2022 , 308 : 141 - 146 .

DU Y X , DING H , WU M , et al . MES-CTNet: A novel capsule transformer network base on a multi-domain feature map for electroencephalogram-based emotion recognition [J ] . Brain Sciences , 2024 , 14 ( 4 ): 344 .

SAMAL P , HASHMI M F . Role of machine learning and deep learning techniques in EEG-based BCI emotion recognition system: A review [J ] . Artificial Intelligence Review , 2024 , 57 ( 3 ): 50 .

LIU H , LOU T Y , ZHANG Y Z , et al . EEG-based multimodal emotion recognition: A machine learning perspective [J ] . IEEE Transactions on Instrumentation and Measurement , 2024 , 73 : 1 - 29 .

JACKSON M C , ARLEGUI-PRIETO M . Variation in normal mood state influences sensitivity to dynamic changes in emotional expression [J ] . Emotion , 2016 , 16 ( 2 ): 145 - 149 .

DU X L , MENG Y F , QIU S M , et al . EEG emotion recognition by fusion of multi-scale features [J ] . Brain Sciences , 2023 , 13 ( 9 ): 1293 .

YUVARAJ R , THAGAVEL P , THOMAS J , et al . Comprehensive analysis of feature extraction methods for emotion recognition from multichannel EEG recordings [J ] . Sensors , 2023 , 23 ( 2 ): 915 .

常文文 , 闫光辉 , 杨志飞 , 等 . 基于脑电熵值特征和功能连接的不同线型道路下驾驶状态检测 [J ] . 电子学报 , 2023 , 51 ( 10 ): 2874 - 2883 .

CHANG W W , YAN G H , YANG Z F , et al . Detection of driving state under different curve road based on entropy and functional connectivity of EEG [J ] . Acta Electronica Sinica , 2023 , 51 ( 10 ): 2874 - 2883 . (in Chinese)

张晶 , 王翌歆 , 任永功 . 统一全局空间表达的脑电信号跨被试情感识别 [J ] . 电子学报 , 2023 , 51 ( 5 ): 1396 - 1404 .

ZHANG J , WANG Y X , REN Y G . Unified global spatial representation for EEG subject-independent emotion recognition [J ] . Acta Electronica Sinica , 2023 , 51 ( 5 ): 1396 - 1404 . (in Chinese)

SINGH U , SHAW R , PATRA B K . A data augmentation and channel selection technique for grading human emotions on DEAP dataset [J ] . Biomedical Signal Processing and Control , 2023 , 79 : 104060 .

BHATTACHARYYA A , TRIPATHY R K , GARG L , et al . A novel multivariate-multiscale approach for computing EEG spectral and temporal complexity for human emotion recognition [J ] . IEEE Sensors Journal , 2021 , 21 ( 3 ): 3579 - 3591 .

CUI H , LIU A P , ZHANG X , et al . EEG-based emotion recognition using an end-to-end regional-asymmetric convolutional neural network [J ] . Knowledge-Based Systems , 2020 , 205 : 106243 .

ZHU X L , RONG W T , ZHAO L , et al . EEG emotion classification network based on attention fusion of multi-channel band features [J ] . Sensors , 2022 , 22 ( 14 ): 5252 .

GUO Y Q , ZHANG B W , FAN X M , et al . A comprehensive interaction in multiscale multichannel EEG signals for emotion recognition [J ] . Mathematics , 2024 , 12 ( 8 ): 1180 .

WU X , ZHANG Y M , LI J J , et al . FC-TFS-CGRU: A temporal-frequency-spatial electroencephalography emotion recognition model based on functional connectivity and a convolutional gated recurrent unit hybrid architecture [J ] . Sensors , 2024 , 24 ( 6 ): 1979 .

PATLAR A F . Hybrid deep convolutional model-based emotion recognition using multiple physiological signals [J ] . Computer Methods in Biomechanics and Biomedical Engineering , 2022 , 25 ( 15 ): 1678 - 1690 .

HUANG Z T , MA Y H , WANG R R , et al . A model for EEG-based emotion recognition: CNN-Bi-LSTM with attention mechanism [J ] . Electronics , 2023 , 12 ( 14 ): 3188 .

YAO X Z , LI T W , DING P , et al . Emotion classification based on transformer and CNN for EEG spatial-temporal feature learning [J ] . Brain Sciences , 2024 , 14 ( 3 ): 268 .

MA Y L , ZHAO W C , MENG M , et al . Cross-subject emotion recognition based on domain similarity of EEG signal transfer learning [J ] . IEEE Transactions on Neural Systems and Rehabilitation Engineering , 2023 , 31 : 936 - 943 .

LI G F , OUYANG D L , YUAN Y F , et al . An EEG data processing approach for emotion recognition [J ] . IEEE Sensors Journal , 2022 , 22 ( 11 ): 10751 - 10763 .

SAHA O , MAHMUD M S , FATTAH S A , et al . Automatic emotion recognition from multi-band EEG data based on a deep learning scheme with effective channel attention [J ] . IEEE Access , 2022 , 11 : 2342 - 2350 .

FOX N A . If it’s not left, it’s right: Electroencephalograph asymmetry and the development of emotion [J ] . American Psychologist , 1991 , 46 ( 8 ): 863 - 872 .

QIAO W X , SUN L , WU J H , et al . EEG emotion recognition model based on attention and GAN [J ] . IEEE Access , 2024 , 12 : 32308 - 32319 .

WANG J G , SHAO H M , YAO Y , et al . Electroencephalograph-based emotion recognition using convolutional neural network without manual feature extraction [J ] . Applied Soft Computing , 2022 , 128 : 109534 .

ZHANG J H , HAO Y R , WEN X , et al . Subject-independent emotion recognition based on EEG frequency band features and self-adaptive graph construction [J ] . Brain Sciences , 2024 , 14 ( 3 ): 271 .

QU G G , WANG F , BI J Y , et al . A hybrid critical channel selection framework for EEG emotion recognition [J ] . IEEE Sensors Journal , 2024 , 24 ( 9 ): 14881 - 14893 .

AL-QAZZAZ N K , SABIR M K , ALI S H B M , et al . Electroencephalogram profiles for emotion identification over the brain regions using spectral, entropy and temporal biomarkers [J ] . Sensors , 2019 , 20 ( 1 ): 59 .

ZHAO G Z , ZHANG Y L , ZHANG G H , et al . Multi-target positive emotion recognition from EEG signals [J ] . IEEE Transactions on Affective Computing , 2023 , 14 ( 1 ): 370 - 381 .

AGARWAL R , ANDUJAR M , CANAVAN S . Classification of emotions using EEG activity associated with different areas of the brain [J ] . Pattern Recognition Letters , 2022 , 162 : 71 - 80 .

CHEN T , YIN H F , YUAN X H , et al . Emotion recognition based on fusion of long short-term memory networks and SVMs [J ] . Digital Signal Processing , 2021 , 117 : 103153 .

BAI S J , KOLTER J Z , KOLTUN V . An empirical evaluation of generic convolutional and recurrent networks for sequence modeling [J ] . Computer Science , 2018 , 1 : 1803 .01271.

DU X B , MA C X , ZHANG G H , et al . An efficient LSTM network for emotion recognition from multichannel EEG signals [J ] . IEEE Transactions on Affective Computing , 2022 , 13 ( 3 ): 1528 - 1540 .

HOSSEINI S M , TALEPASSAND S , BIGDELI I . Brain activity and affect: Overall and asymmetric activity of the brain lobes in affective states [J ] . Journal of Research in Medical Sciences , 2009 , 14 ( 5 ): 309 - 311 .

VEMPATI R , SHARMA L D . EEG rhythm based emotion recognition using multivariate decomposition and ensemble machine learning classifier [J ] . Journal of Neuroscience Methods , 2023 , 393 : 109879 .

YU X K , LI Z J , ZANG Z B , et al . Real-time EEG-based emotion recognition [J ] . Sensors , 2023 , 23 ( 18 ): 7853 .

LIU S Q , ZHAO Y Y , AN Y L , et al . GLFANet: A global to local feature aggregation network for EEG emotion recognition [J ] . Biomedical Signal Processing and Control , 2023 , 85 : 104799 .

LI G F , YUAN B W , OUYANG D L , et al . Emotion recognition based on selected EEG signals by common spatial pattern [J ] . IEEE Sensors Journal , 2024 , 24 ( 6 ): 8414 - 8426 .

SONG Y B , FAN C L , MAO X Q . Optimization of epilepsy detection method based on dynamic EEG channel screening [J ] . Neural Networks , 2024 , 172 : 106119 .

LI G F , OUYANG D L , YANG L , et al . Cross-subject EEG linear domain adaption based on batch normalization and depthwise convolutional neural network [J ] . Knowledge-Based Systems , 2023 , 280 : 111011 .

LI Q , LIU Y Q , LIU Q Y , et al . Multidimensional feature in emotion recognition based on multi-channel EEG signals [J ] . Entropy , 2022 , 24 ( 12 ): 1830 .

LI X , SHEN F Y , PENG Y , et al . Efficient sample and feature importance mining in semi-supervised EEG emotion recognition [J ] . IEEE Transactions on Circuits and Systems II: Express Briefs , 2022 , 69 ( 7 ): 3349 - 3353 .

ZHANG Y K , PENG Y , LI J H , et al . SIFIAE: An adaptive emotion recognition model with EEG feature-label inconsistency consideration [J ] . Journal of Neuroscience Methods , 2023 , 395 : 109909 .

GUO J Y , CAI Q , AN J P , et al . A Transformer based neural network for emotion recognition and visualizations of crucial EEG channels [J ] . Physica A: Statistical Mechanics and Its Applications , 2022 , 603 : 127700 .

KANG D , KIM D , KANG D , et al . Beyond superficial emotion recognition: Modality-adaptive emotion recognition system [J ] . Expert Systems with Applications , 2024 , 235 : 121097 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Research on Holism-Based Feature Extraction and Fusion for Steganalysis

Variable Horizon Multi-Directional Scanning Method for Time Series Anomaly Detection

Low-Light Small Target Detection Method Combining Feature Fusion Enhancement and Detail Features

Object Detection Based on EIMYOLO for High-Resolution Remote Sensing Images

Related Author

GUO Yan-qing

HE Dequan

YOU Xin-gang

KONG Xiang-wei

WANG Bo

HUANG Yu-zhe

GUAN Yong-yuan

WEI Song-jie

Related Institution

Information Security Research Center of Dalian University of Technology

Beijing Institute of Electronic Technology and Application

Information Security Research Center of Dalian University of Technology Dalian Liaoning China

Beijing Institute of Electronic Technology and Application Beijing China

School of Computer Science and Engineering, School of Cyber Science and Engineering, Nanjing University of Science and Technology

⁰