A Hierarchical Automatic Sleep Staging Model Based on Transformer Architecture

JIN Zheng; JIA Ke-bin

doi:10.12263/DZXB.20240596

您当前的位置：

首页 >

文章列表页 >

A Hierarchical Automatic Sleep Staging Model Based on Transformer Architecture

PAPERS | 更新时间：2025-12-08

- A Hierarchical Automatic Sleep Staging Model Based on Transformer Architecture
- ACTA ELECTRONICA SINICA Vol. 53, Issue 2, Pages: 545-557(2025)
- 作者机构：
  
  1.北京工业大学信息科学技术学院，北京 100124
  2.先进信息网络北京实验室，北京 100124
  3.计算智能与智能系统北京市重点实验室，北京 100124
- 作者简介：
- 基金信息：
  
  Natural Science Foundation of Beijing Municipality(4212001)
- DOI：10.12263/DZXB.20240596
  CLC： TP391;
- Received：25 June 2024，
  
  Revised：2024-10-28，
  
  Published：25 February 2025
- 稿件说明：
移动端阅览
金峥, 贾克斌. 一种基于Transformer架构的多层级自动睡眠分期模型[J]. 电子学报, 2025, 53(02): 545-557.

JIN Zheng, JIA Ke-bin. A Hierarchical Automatic Sleep Staging Model Based on Transformer Architecture[J]. Acta Electronica Sinica, 2025, 53(02): 545-557.
金峥, 贾克斌. 一种基于Transformer架构的多层级自动睡眠分期模型[J]. 电子学报, 2025, 53(02): 545-557. DOI：10.12263/DZXB.20240596

JIN Zheng, JIA Ke-bin. A Hierarchical Automatic Sleep Staging Model Based on Transformer Architecture[J]. Acta Electronica Sinica, 2025, 53(02): 545-557. DOI：10.12263/DZXB.20240596

摘要

睡眠是人体保持健康的重要生理过程，基于多导睡眠图（PolySomnoGraphy，PSG）的睡眠分期是诊疗睡眠疾病和评估睡眠质量的重要依据.人工睡眠分期法在处理大规模PSG数据时存在耗时久、效率低的问题，采用深度学习模型有效表征PSG的自动睡眠分期法显现出广阔的研究前景.针对现有模型未充分考虑PSG片段内波形信息、通道间相关性信息、片段间睡眠转换信息的问题，本文提出一种基于Transformer架构的多层级睡眠分期网络模型（Hierarchical transFormer sleep staging model，HierFormer），采用Transformer编码器有效提取片段内波形特征、通道相关性特征、片段间转换特征，并结合注意力机制综合提升模型对于PSG片段内、通道间、片段间三种视角信号特性的可解释性.基于睡眠集-欧洲数据格式（sleep-European Data Format，sleep-EDF）扩展睡眠数据集开展的实验结果表明：本文模型利用更少的参数量取得优于多种现有基线模型的分期性能，分类准确率、宏平均精确率、宏平均召回率、宏平均F1分数、科恩卡帕系数分别可达到0.807、0.784、0.735、0.750和0.721.通过在三种视角下不同特征编码方式的性能对比和注意力分数的可视化，本文进一步证明了所提模型良好的编码能力和可解释性.本研究旨在为睡眠分期领域的深度学习应用提供新途径和新技术，从而辅助医生提升睡眠疾病诊疗效率.

Abstract

Sleep is the significant physiological process to keep healthy. Sleep stage classification based on polysomnography (PSG) is the fundamental evidence to diagnose sleep disorders and assess sleep quality. Manual sleep staging method has some typical problems when handling the large-scale PSG data

such as time-consuming and low-efficiency. The automatic sleep staging method that utilizing deep learning models to effectively learn PSG representations shows extensive researching prospects. Most existing models cannot fully consider the epoch-level waveform information

channel-wise correlations

sequence-level sleep transitions. This paper proposes a transformer-based hierarchical sleep staging model (HierFormer)

which employs transformer encoder to extract valid epoch-level waveform features

channel-wise correlation features

sequence-level transition features. Meanwhile

it adopts attention mechanism to improve the model interpretability of signal properties from three views

including epoch-level

channel-wise

and sequence-level views. Experimental results on the sleep-european data format (sleep-EDF) database expanded dataset show that the proposed model achieves better sleep staging performance with less parameters compared with various baseline models. The overall accuracy

macro-averaging precision

macro-averaging recall

macro-averaging F1-score

and Cohen’s-kappa coefficient achieve 0.807

0.784

0.735

0.750

and 0.721

respectively. According to the performance comparisons of different feature encoding methods from three views and the visualization of attention weights

this paper further demonstrates the satisfied encoding ability and interpretability of proposed model. This study aims to provide innovative deep learning approaches and technologies for the research of sleep staging applications

thus assisting sleep experts to improve the efficiency of sleep disorder diagnosis and treatment.

关键词

Keywords

references

TAI C H , LIAO T Y , CHEN S P , et al . Sleep stage classification using Light Gradient Boost Machine: Exploring feature impact in depressive and healthy participants [J ] . Biomedical Signal Processing and Control , 2024 , 88 : 105647 .

金峥 , 贾克斌 , 袁野 . 基于混合注意力时序网络的睡眠分期算法研究 [J ] . 生物医学工程学杂志 , 2021 , 38 ( 2 ): 241 - 248 .

JIN Z , JIA K B , YUAN Y . A hybrid attention temporal sequential network for sleep stage classification [J ] . Journal of Biomedical Engineering , 2021 , 38 ( 2 ): 241 - 248 . (in Chinese)

JIN Z , JIA K B . SAGSleepNet: A deep learning model for sleep staging based on self-attention graph of polysomnography [J ] . Biomedical Signal Processing and Control , 2023 , 86 : 105062 .

IBER C , ANCOLI-ISRAEL S , CHESSON A J , et al . The AASM manual for the scoring of sleep and associated events [M ] . Westchester : American Academy of Sleep Medicine , 2007 .

ZHANG L D , FABBRI D , UPENDER R , et al . Automated sleep stage scoring of the Sleep Heart Health Study using deep neural networks [J ] . Sleep , 2019 , 42 ( 11 ): zsz159 .

SHAHBAKHTI M , BEIRAMVAND M , EIGIRDAS T , et al . Discrimination of wakefulness from sleep stage I using nonlinear features of a single frontal EEG channel [J ] . IEEE Sensors Journal , 2022 , 22 ( 7 ): 6975 - 6984 .

VAN DER DONCKT J , VAN DER DONCKT J , DEPROST E , et al . Do not sleep on traditional machine learning Simple and interpretable techniques are competitive to deep learning for sleep scoring [J ] . Biomedical Signal Processing and Control , 2023 , 81 : 104429 .

MEMAR P , FARADJI F . A novel multi-class EEG-based sleep stage classification system [J ] . IEEE Transactions on Neural Systems and Rehabilitation Engineering , 2018 , 26 ( 1 ): 84 - 95 .

SATAPATHY S K , BHOI A K , LOGANATHAN D , et al . Machine learning with ensemble stacking model for automated sleep staging using dual-channel EEG signal [J ] . Biomedical Signal Processing and Control , 2021 , 69 : 102898 .

HUANG J , REN L F , FENG L F , et al . AI empowered virtual reality integrated systems for sleep stage classification and quality enhancement [J ] . IEEE Transactions on Neural Systems and Rehabilitation Engineering , 2022 , 30 : 1494 - 1503 .

KARIMZADEH F , BOOSTANI R , SERAJ E , et al . A distributed classification procedure for automatic sleep stage scoring based on instantaneous electroencephalogram phase and envelope features [J ] . IEEE Transactions on Neural Systems and Rehabilitation Engineering , 2018 , 26 ( 2 ): 362 - 370 .

SEKKAL R N , BEREKSI-REGUIG F , RUIZ-FERNANDEZ D , et al . Automatic sleep stage classification: From classical machine learning methods to deep learning [J ] . Biomedical Signal Processing and Control , 2022 , 77 : 103751 .

ZAIDI T F , FAROOQ O . EEG sub-bands based sleep stages classification using Fourier Synchrosqueezed transform features [J ] . Expert Systems with Applications , 2023 , 212 : 118752 .

SUPRATAK A , DONG H , WU C , et al . DeepSleepNet: A model for automatic sleep stage scoring based on raw single-channel EEG [J ] . IEEE Transactions on Neural Systems and Rehabilitation Engineering , 2017 , 25 ( 11 ): 1998 - 2008 .

ZHOU D D , XU Q , WANG J , et al . LightSleepNet: A lightweight deep model for rapid sleep stage classification with spectrograms [C ] // The 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) . Piscataway : IEEE , 2021 : 43 - 46 .

PERSLEV M , JENSEN M H , DARKNER S , et al . U-time: A fully convolutional network for time series segmentation applied to sleep staging [C ] // The 33rd Conference on Neural Information Processing Systems (NeurIPS) . California : NIPS , 2019 : 4415 - 4426 .

PHAN H , ANDREOTTI F , COORAY N , et al . SeqSleepNet: End-to-end hierarchical recurrent neural network for sequence-to-sequence automatic sleep staging [J ] . IEEE Transactions on Neural Systems and Rehabilitation Engineering , 2019 , 27 ( 3 ): 400 - 410 .

GUILLOT A , SAUVET F , DURING E H , et al . Dreem open datasets: Multi-scored sleep datasets to compare human and automated sleep staging [J ] . IEEE Transactions on Neural Systems and Rehabilitation Engineering , 2020 , 28 ( 9 ): 1955 - 1965 .

KHALILI E , MOHAMMADZADEH ASL B . Automatic sleep stage classification using temporal convolutional neural network and new data augmentation technique from raw single-channel EEG [J ] . Computer Methods and Programs in Biomedicine , 2021 , 204 : 106063 .

GUILLOT A , THOREY V . RobustSleepNet: Transfer learning for automated sleep staging at scale [J ] . IEEE Transactions on Neural Systems and Rehabilitation Engineering , 2021 , 29 : 1441 - 1451 .

CHAMBON S , GALTIER M N , ARNAL P J , et al . A deep learning architecture for temporal sleep stage classification using multivariate and multimodal time series [J ] . IEEE Transactions on Neural Systems and Rehabilitation Engineering , 2018 , 26 ( 4 ): 758 - 769 .

YUAN Y , JIA K B , MA F L , et al . A hybrid self-attention deep learning framework for multivariate sleep stage classification [J ] . BMC Bioinformatics , 2019 , 20 ( Suppl 16 ): 586 .

JIA Z Y , LIN Y F , WANG J , et al . GraphSleepNet: Adaptive spatial-temporal graph convolutional networks for sleep stage classification [C ] // The 29th International Joint Conference on Artificial Intelligence . California : IJCAI , 2020 : 1324 - 1330 .

JIN Z , JIA K B . A temporal multi-scale hybrid attention network for sleep stage classification [J ] . Medical & Biological Engineering & Computing , 2023 , 61 ( 9 ): 2291 - 2303 .

ELDELE E , CHEN Z H , LIU C Y , et al . An attention-based deep learning approach for sleep stage classification with single-channel EEG [J ] . IEEE Transactions on Neural Systems and Rehabilitation Engineering , 2021 , 29 : 809 - 818 .

WANG H F , LU C G , ZHANG Q , et al . A novel sleep staging network based on multi-scale dual attention [J ] . Biomedical Signal Processing and Control , 2022 , 74 : 103486 .

PATHAK S , LU C Q , NAGARAJ S B , et al . STQS: Interpretable multi-modal Spatial-Temporal-seQuential model for automatic Sleep scoring [J ] . Artificial Intelligence in Medicine , 2021 , 114 : 102038 .

VASWANI A , SHAZEER N , PARMAR N , et al . Attention is all you need [C ] // The 31st Annual Conference on Neural Information Processing Systems (NeurIPS) . California : NIPS , 2017 : 5999 - 6009 .

PHAN H , MIKKELSEN K , CHÉN O Y , et al . SleepTransformer: Automatic sleep staging with interpretability and uncertainty quantification [J ] . IEEE Transactions on Biomedical Engineering , 2022 , 69 ( 8 ): 2456 - 2467 .

ZHANG W J , LI C , PENG H , et al . CTCNet: A CNN Transformer capsule network for sleep stage classification [J ] . Measurement , 2024 , 226 : 114157 .

CHEN Z , YANG Z W , ZHU L W , et al . Automated sleep staging via parallel frequency-cut attention [J ] . IEEE Transactions on Neural Systems and Rehabilitation Engineering , 2023 , 31 : 1974 - 1985 .

PENG L , REN Y Z , LUAN Z H , et al . SleepViTransformer: Patch-based sleep spectrogram transformer for automatic sleep staging [J ] . Biomedical Signal Processing and Control , 2023 , 86 : 105203 .

GOLDBERGER A L , AMARAL L A , GLASS L , et al . PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals [J ] . Circulation , 2000 , 101 ( 23 ): E215 - E220 .

KEMP B , ZWINDERMAN A H , TUK B , et al . Analysis of a sleep-dependent neuronal feedback loop: The slow-wave microcontinuity of the EEG [J ] . IEEE Transactions on Biomedical Engineering , 2000 , 47 ( 9 ): 1185 - 1194 .

WOLPERT E A . A manual of standardized terminology, techniques and scoring system for sleep stages of human subjects [J ] . Archives of General Psychiatry , 1969 , 20 ( 2 ): 246 - 247 .

IMTIAZ S A , RODRIGUEZ-VILLEGAS E . An open-source toolbox for standardized use of PhysioNet Sleep EDF Expanded Database [C ] // The 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) . Piscataway : IEEE , 2015 : 6014 - 6017 .

PASZKE A , GROSS S , CHINTALA S , et al . Automatic differentiation in pytorch [C ] // The 31st Annual Conference on Neural Information Processing Systems (NeurIPS) . California : NIPS , 2017 : 1 - 4 .

KINGMA D P , BA J , HAMMAD M M . Adam: A method for stochastic optimization [EB/OL ] . ( 2017-01-30 )[ 2024-06-25 ] . https://arxiv.org/abs/1412.6980v9 https://arxiv.org/abs/1412.6980v9 .

DAVIES H J , NAKAMURA T , MANDIC D P . A transition probability based classification model for enhanced N1 sleep stage identification during automatic sleep stage scoring [C ] // The 41st Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) . Piscataway : IEEE , 2019 : 3641 - 3644 .

BAEVSKI A , ZHOU Y H , MOHAMED A , et al . Wav2vec 2.0: A framework for self-supervised learning of speech representations [C ] // The 34th Conference on Neural Information Processing Systems (NeurIPS) . California : NIPS , 2020 : 12449 - 12460 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Modulation Recognition Method Based on Improved Res2Net and Adaptive Multi-Scale Window Pooling

A TCAD-DNN-Based Total-Ionizing-Dose Effect Model for FinFET Devices

Multi-Sound Source Separation Method Based on Multi-Resolution Parallel Feature Extraction

Sequence-Based and Cross-Modal Alignment Model for Protein Function Prediction

Related Author

WANG Dan

LI Wanjie

JIANG Fengyang

ZHOU Ying

LIU Xiaonian

LIU Yansen

CAO Peng

GAO Shang

Related Institution

School of Communications and Information Engineering, Chongqing University of Posts and Telecommunications

School of Physics and Electronics, Hunan Normal University

Key Laboratory of Physics and Devices in Post-Moore Era, College of Hunan Province

School of Artificial Intelligence and Big Data, Hefei University

College of Information Science and Engineering, Hunan Normal University

⁰