Abnormal Detection Based on Graph Attention Mechanisms and Transformer

YAN Li; ZHANG Kai; XU Hao; HAN Sheng-ya; LIU Shen-qi; SHI Yu-liang

doi:10.12263/DZXB.20210722

您当前的位置：

首页 >

文章列表页 >

Abnormal Detection Based on Graph Attention Mechanisms and Transformer

CROSS-DISCIPLINARY INNOVATIONS OF MACHINE LEARNING | 更新时间：2025-12-08

- Abnormal Detection Based on Graph Attention Mechanisms and Transformer
- ACTA ELECTRONICA SINICA Vol. 50, Issue 4, Pages: 900-908(2022)
- 作者机构：
  
  1.国网山东省电力公司信息通信公司，山东济南 250013
  2.山东大学软件学院，山东济南 250101
- 作者简介：
- 基金信息：
- DOI：10.12263/DZXB.20210722
  CLC： TP391;
- Received：07 June 2021，
  
  Revised：2022-01-19，
  
  Published：25 April 2022
- 稿件说明：
移动端阅览
严莉,张凯,徐浩等.基于图注意力机制和Transformer的异常检测[J].电子学报,2022,50(04):900-908.

YAN Li,ZHANG Kai,XU Hao,et al.Abnormal Detection Based on Graph Attention Mechanisms and Transformer[J].ACTA ELECTRONICA SINICA,2022,50(04):900-908.
严莉,张凯,徐浩等.基于图注意力机制和Transformer的异常检测[J].电子学报,2022,50(04):900-908. DOI： 10.12263/DZXB.20210722.

YAN Li,ZHANG Kai,XU Hao,et al.Abnormal Detection Based on Graph Attention Mechanisms and Transformer[J].ACTA ELECTRONICA SINICA,2022,50(04):900-908. DOI： 10.12263/DZXB.20210722.

摘要

异常检测对电力行业的发展有着重要的影响，如何根据大规模电力数据进行异常检测是重要的研究热点.目前，大多数研究通过聚类或神经网络进行异常检测. 但是这些方法忽略了时序数据之间潜在的关联关系及某些特点的重要信息，没有充分挖掘出数据的潜在价值. 因此，提出了一种基于图注意力和Transformer的异常检测模型. 该模型首先根据数据中台中获取的电力数据（主要包括用户ID、电能表ID、用户类型、电流、电压、功率等数据）构建一个异构信息网络；然后，为了减少模型参数和避免出现过拟合的现象，在图卷积网络（Graph Convolutional Network，GCN）模型的基础上，引入非负矩阵分解（Non-Negative Matrix Factorization，NNMF）的方法来进行相似性学习；最后采用图注意力网络（Graph Attention Network，GAT）和Transformer共同捕获数据间的相互关联关系，从而提高检测精度. 以中国某地区的电力数据为基础进行验证，实验结果表明所提出的方法可以有效进行异常检测.

Abstract

Anomaly detection has an important impact on the development of the electric power industry

and how to detect anomalies based on large-scale power data is a research hotspot. At present

most researches use clustering or neural network to detect anomalies. But these methods ignore the potential relationship between the data and miss some specific important information

and do not fully exploit the potential value of the data. Therefore

an abnormal detection model based on graph attention and transformer is proposed. The model first constructs a heterogeneous information network based on the power data (mainly including user ID

meter ID

user type

electrical current

voltage

power

etc.) collected in the data center; then

in order to reduce the model parameters and avoid the phenomenon of overfitting

on the basis of the graph convolutional network (GCN) model

a non-negative matrix factorization (NNMF) method is introduced to perform similarity learning; finally

a graph attention network (GAT) and Transformer are jointly used to capture the correlation relationships between data

thus improving the detection accuracy. The validation analysis is carried out based on the power data of a region in China. The experimental results show that the proposed method can effectively perform anomaly detection.

关键词

Keywords

references

谢敬东 , 卢浩哲 , 陆池鑫 , 等 . 基于分阶段离群点检测的电力市场异常辨识 [J]. 科学技术与工程 , 2021 , 21 ( 9 ): 3633 - 3641 .

XIE J D , LU H Z , LU C X , et al . Identification of abnormal behavior in power market based on phased outlier detection [J]. Science Technology and Engineering , 2021 , 21 ( 9 ): 3633 - 3641 . (in Chinese)

Al-Dhamari A , Sudirman R , Mahmood N H , et al . Online video-based abnormal detection using highly motion techniques and statistical measures [J]. Telkomnika , 2019 , 17 ( 4 ): 2039 - 2047 .

丁小欧 , 于晟健 , 王沐贤 , 等 . 基于相关性分析的工业时序数据异常检测 [J]. 软件学报 , 2020 , 31 ( 3 ): 22 .

DING X O , YU S J , WANG M X , et al . Anomaly Detection on Industrial Time Series Based on Correlation Analysis [J]. Journal of Software , 2020 , 31 ( 3 ): 22 . (in Chinese)

LIM J , CHOI J . Web based online real-time outage cost assessment information system of power system [J]. Review of Scientific Instruments , 2012 , 37 ( 2 ): 171 - 172 .

BRAITMAN L E . Confidence intervals extract clinically useful information from data [J]. Annals of Internal Medicine , 1988 , 108 ( 2 ): 296 - 298 .

ZHANG S , VITTAL V . Design of wide-area power system damping controllers resilient to communication failures [J]. IEEE Transactions on Power Systems , 2013 , 28 ( 4 ): 4292 - 4300 .

SHAFAGHI A . Equipment failure rate updating-Bayesian estimation [J]. Journal of Hazardous Materials , 2008 , 159 ( 1 ): 87 - 91 .

SORENSEN P , CUTULULIS N A , VIGUERAS-RODRIGUEZ A , et al . Power fluctuations from large wind farms [J]. IEEE Transactions on Power Systems , 2007 , 22 ( 3 ): 958 - 965 .

XU L , CHOW M Y , TAYLOR L S . Power distribution fault cause identification with imbalanced data using the data mining-based fuzzy classification E-algorithm [J]. IEEE Transactions on Power Systems , 2007 , 22 ( 1 ): 164 - 171 .

BU S , YU F R , CAI Y , et al . When the smart grid meets energy Efficient communications: Green wireless cellular networks powered by the smart grid [J]. IEEE Transactions on Wireless Communications , 2012 , 11 ( 8 ): 3014 - 3024 .

孙毅 , 李世豪 , 崔灿等 . 基于高斯校函数的电力用户用电数据离群点检测方法 [J]. 电网技术， 2018 ， 42 ( 5 )： 1595 - 1606 .

SUN Y , LI S H , CUI C , et al . Improved outlier detection method of power consumer data based on Gaussian kernel function [J]. Power System. Technology. , 2018 , 42 ( 5 ): 1595 - 1606 . (in Chinese)

MONEDERO I , BISCARRI F , LEÓN C , et al . Detection of frauds and other non-technical losses in a power utility using Pearson coeffificient, Bayesian networks and decision trees [J]. International Journal of Electrical Power & Energy Systems , 2012 , 34 ( 1 ): 90 - 98 .

WANG Z , Li G , WANG X , et al . Analysis of 10kV non-technical loss detection with data-driven approaches [C]// 2019 IEEE Innovative Smart Grid Technologies - Asia (ISGT Asia) . Chengdu : IEEE , 2019 : 4154 - 4158 .

BUZAU M M , Tejedor-Aguilera J , Cruz-Romero P , et al . Hybrid deep neural networks for detection of non-technical losses in electricity smart meters [J]. IEEE Transactions on Power Systems , 2020 , 35 ( 2 ): 1254 - 1263 .

CHAHLA C , SNOUSSI H , MERGHEM L , et al . A deep learning approach for anomaly detection and prediction in power consumption data [J]. Energy Efficiency , 2020 , 13 ( 8 ): 1633 - 1651 .

BARUA A , MUTHIRAYAN D , KHARGONEKAR P P , et al . Hierarchical temporal memory based machine learning for real-time, unsupervised anomaly detection in smart grid: WiP abstract [C]// 2020 ACM/IEEE 11th International Conference on Cyber-Physical Systems(ICCPS) . Sydney, NSW, Australia : ACM , 2020 : 188 - 189 .

ROUZBAHANI H M , HADIS KARIMIPOUR H , LEI L . An ensemble deep convolutional neural network model for electricity theft detection in smart grids [C]// 2020 IEEE International Conference on Systems, Man, and Cybernetics(SMC) . Singapore . Singapore : IEEE , 2020 : 3637 - 3642

LO Y L , HUANG S C , LU C N . Non-technical loss detection using smart distribution network measurement data [C]// IEEE PES Innovative Smart Grid Technologies . Tianjin : IEEE , 2012 : 1 - 5 .

张承智 , 肖先勇 , 郑子萱 . 基于实值深度置信网络的用户侧窃电行为检测 [J]. 电网技术 , 2019 , 43 ( 3 ): 1083 - 1091 .

ZHANG C Z , XIAO X Y , ZHENG Z X . Electricity theft detection for customers in power utility based on real-valued deep belief network [J]. Power System . Technology , 2019 , 43 ( 3 ): 1083 - 1091 . (in Chinese)

WANG B Q , JIANG T H , ZHOU X , et al . Variance error of multi-classification based anomaly detection for time series data [J]. J. Comput. Methods Sci. Eng . 2021 , 21 ( 4 ): 875 - 890 .

ADIL M , NADEEM , ZIA U , et al . Electricity theft detection using machine learning techniques to secure smart grid [C]// Proceedings of the 14th International Conference on Complex, Intelligent and Software Intensive Systems . Lodz : Springer . 2020 : 233 - 243 .

KHALEDIAN E , PANDEY S , KUNDU P , et al . Real-time synchrophasor data anomaly detection and classification using isolation forest, KMeans, and LoOP [J]. IEEE Transactions on Smart Grid , 2020 , 12 ( 3 ): 2378 - 2388 .

LONG Z . A study of intelligent analysis of abnormal power consumption behavior based on daily load curve [C]// AIAM2020: 2nd International Conference on Artificial Intelligence and Advanced Manufacture . Shanghai : ACM , 2020 : 209 - 216

MISHRA S , KSHIRSAGAR V , DWIVEDULA R , et al . Attention-Based Bi-LSTM for Anomaly Detection on Time-Series Data [C]// Proceedings of the 30th International Conference on Artificial Neural Networks . Bratislava : ENNS , 2021 : 129 - 140 .

HOMAYOUNI H , GHOSH S , RAY I , et al . An Autocorrelation-based LSTM-Autoencoder for Anomaly Detection on Time-Series Data [C]// 2020 IEEE International Conference on Big Data . Atlanta, Georgia : IEEE , 2020 : 5068 - 5077 .

PASSOS JÚNIOR L A , OBA RAMOS C C , RODRIGUES D , et al . Unsupervised non-technical losses identifification through optimum-path forest [J]. Electric Power Systems Research , 2016 , 140 ( 1 ): 413 - 423 .

CUPER M , LÓDERER M , ROZINAJOVÁ V . Detection of abnormal load consumption in the power grid using clustering and statistical analysis [C]// Intelligent Data Engineering and Automated Learning . Manchester : Springer , 2019 : 464 - 475 .

曾惟如 , 吴佳 , 闫飞 . 基于层级实时记忆算法的时间序列异常检测算法 [J]. 电子学报 , 2018 , 46 ( 2 ): 325 - 332 .

ZENG W R , WU J , YAN F . time series anomaly detection model based on hierarchical temporal memory [J]. Acta Electronica Sinica , 2018 , 46 ( 2 ): 325 - 332 . (in Chinese)

周伯阳 , 郭志民 , 王延松 , 等 . 基于多尺度低秩模型的电力无线接入网异常流量检测方法 [J]. 电子学报 , 2020 , 48 ( 8 ): 1552 - 1557 .

ZHOU B Y , GUO Z M , WANG Y S , et al . An anomaly traffic detection method using multi-resolution low rank model for wireless access network of electric power grids [J]. Acta Electronica Sinica , 2020 , 48 ( 8 ): 1552 - 1557 . (in Chinese)

PENG Y L , YANG Y N , XU Y J , et al . Electricity theft detection in ami based on clustering and local outlier factor [J]. IEEE Access , 2021 , 9 ( 1 ): 107250 - 107259 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

A Medical Image Segmentation Network Based on Cross-Visual State Space and Multi-Branch Interactive Attention

A Motion Planning Method for Autonomous Driving Based on Spatiotemporal Attention Transformer

Cross-Modal Light-3Dformer Model for Lung Tumor Classification

Discriminative Category Prompt Learning Based on Image Content Understanding

A Progressive Abnormal Detection Method for Water Treatment System Based on Autoencoder and Isolation Forest

Related Author

XUE Wei

CHEN Chuang-hui

DU Ming-yang

ZHONG Ping

ZHENG Xiao

YUAN Ding

LI Yuan

MENG Yu-qian

Related Institution

College of Electronic Science and Technology, National University of Defense Technology

College of Electronic Engineering, National University of Defense Technology

School of Computer Science and Technology, Anhui University of Technology, Maanshan

School of Astronautics, Beihang University

School of medical information & Engineering, Ningxia Medical University

⁰