电子学报 ›› 2022, Vol. 50 ›› Issue (9): 2205-2214.DOI: 10.12263/DZXB.20210152
毛国君, 王者浩, 黄山, 王翔
收稿日期:
2021-01-24
修回日期:
2021-12-31
出版日期:
2022-09-25
作者简介:
基金资助:
MAO Guo-jun, WANG Zhe-hao, HUANG Shan, WANG Xiang
Received:
2021-01-24
Revised:
2021-12-31
Online:
2022-09-25
Published:
2022-10-26
摘要:
图神经网络自2005年以来已经逐步成为图学习中的一个重要的研究分支,其中最为活跃的是图卷积神经网络.由于图数据在现实世界中广泛存在,因此有效地完成图结构数据的学习具有很大的应用前景.目前出现的大多数图卷积神经网络模型基本都是浅层结构,过平滑问题成为制约该领域发展的瓶颈问题.本文提出了一种称为dri-GCN(Graph Convolutional Network via dropedge, residual and identity mapping)的图残差卷积深层网络模型,该模型集成了图剪边、初始残差和恒等映射技术.主要思想包括:利用图剪边技术增加学习数据的多样性,以防止学习过程中的过拟合现象;构建恒等映射下的初始残差网络,来扩展残差单元的学习路径,以削弱学习过程中的过平滑问题.实验结果表明,本文提出的dri-GCN模型可以帮助构建深层图卷积神经网络,通过网络层次的加深可以获得优于浅层网络的学习准确率.
中图分类号:
毛国君, 王者浩, 黄山, 等. 基于剪边策略的图残差卷积深层网络模型[J]. 电子学报, 2022, 50(9): 2205-2214.
Guo-jun MAO, Zhe-hao WANG, Shan HUANG, et al. A Deep Residual Graph Convolution Network Based on Dropedge Method[J]. Acta Electronica Sinica, 2022, 50(9): 2205-2214.
数据集 | 节点数 | 边数 | 类别数 | 特征数 |
---|---|---|---|---|
Cora | 2708 | 5429 | 7 | 1433 |
Citeseer | 3327 | 4732 | 6 | 3703 |
Pubmed | 19717 | 44338 | 3 | 500 |
表1 三个数据集的基本信息
数据集 | 节点数 | 边数 | 类别数 | 特征数 |
---|---|---|---|---|
Cora | 2708 | 5429 | 7 | 1433 |
Citeseer | 3327 | 4732 | 6 | 3703 |
Pubmed | 19717 | 44338 | 3 | 500 |
数据集 | 学习率lr | 剪输出系数dropout | 剪边系数p | 残差系数α | 权重控制系数 |
---|---|---|---|---|---|
Cora | 0.01 | 0.6 | 0.9 | 0.1 | 0.5 |
Citeseer | 0.01 | 0.6 | 0.9 | 0.1 | 0.5 |
Pubmed | 0.01 | 0.6 | 0.98 | 0.1 | 0.5 |
表2 实验模型所用的参数
数据集 | 学习率lr | 剪输出系数dropout | 剪边系数p | 残差系数α | 权重控制系数 |
---|---|---|---|---|---|
Cora | 0.01 | 0.6 | 0.9 | 0.1 | 0.5 |
Citeseer | 0.01 | 0.6 | 0.9 | 0.1 | 0.5 |
Pubmed | 0.01 | 0.6 | 0.98 | 0.1 | 0.5 |
层数 | 模型 | |||
---|---|---|---|---|
GCN (G) | d_GCN (d+G) | GCNII (r+G) | dri_GCN (d+r+G) | |
2 | 80.4 | 80.6 | ||
4 | 80.3 | 82.0 | 81.3 | 81.4 |
8 | 69.5 | 75.8 | 82.6 | 82.8 |
16 | 64.9 | 75.7 | 84.0 | 84.2 |
32 | 60.3 | 62.5 | 84.5 | |
64 | 28.7 | 49.5 | 84.5 | 84.5 |
128 | — | — | 84.3 |
表3 不同方法在Cora数据集上的分类准确率(%)
层数 | 模型 | |||
---|---|---|---|---|
GCN (G) | d_GCN (d+G) | GCNII (r+G) | dri_GCN (d+r+G) | |
2 | 80.4 | 80.6 | ||
4 | 80.3 | 82.0 | 81.3 | 81.4 |
8 | 69.5 | 75.8 | 82.6 | 82.8 |
16 | 64.9 | 75.7 | 84.0 | 84.2 |
32 | 60.3 | 62.5 | 84.5 | |
64 | 28.7 | 49.5 | 84.5 | 84.5 |
128 | — | — | 84.3 |
层数 | 模型 | |||
---|---|---|---|---|
GCN (G) | d_GCN (d+G) | GCNII (r+G) | dri_GCN (d+r+G) | |
2 | 65.9 | 66.2 | ||
4 | 67.6 | 70.6 | 66.1 | 66.5 |
8 | 30.2 | 61.4 | 69.3 | 70.0 |
16 | 18.3 | 57.2 | 71.7 | 72.0 |
32 | 25.0 | 41.6 | ||
64 | 20.0 | 34.4 | 72.1 | 72.2 |
128 | — | — | 72.0 | 72.3 |
表4 不同方法在Citeseer数据集上的分类准确率(%)
层数 | 模型 | |||
---|---|---|---|---|
GCN (G) | d_GCN (d+G) | GCNII (r+G) | dri_GCN (d+r+G) | |
2 | 65.9 | 66.2 | ||
4 | 67.6 | 70.6 | 66.1 | 66.5 |
8 | 30.2 | 61.4 | 69.3 | 70.0 |
16 | 18.3 | 57.2 | 71.7 | 72.0 |
32 | 25.0 | 41.6 | ||
64 | 20.0 | 34.4 | 72.1 | 72.2 |
128 | — | — | 72.0 | 72.3 |
层数 | 模型 | |||
---|---|---|---|---|
GCN (G) | d_GCN (d+G) | GCNII (r+G) | dri_GCN (d+r+G) | |
2 | 78.1 | 77.8 | ||
4 | 76.5 | 79.4 | 78.3 | 78.4 |
8 | 61.2 | 78.1 | 79.1 | 79.1 |
16 | 40.9 | 78.5 | 79.2 | 79.3 |
32 | 22.4 | 77.0 | ||
64 | 35.3 | 61.5 | 79.2 | 79.3 |
128 | — | — | 79.1 | 79.3 |
表5 不同方法在Pubmed数据集上的分类准确率(%)
层数 | 模型 | |||
---|---|---|---|---|
GCN (G) | d_GCN (d+G) | GCNII (r+G) | dri_GCN (d+r+G) | |
2 | 78.1 | 77.8 | ||
4 | 76.5 | 79.4 | 78.3 | 78.4 |
8 | 61.2 | 78.1 | 79.1 | 79.1 |
16 | 40.9 | 78.5 | 79.2 | 79.3 |
32 | 22.4 | 77.0 | ||
64 | 35.3 | 61.5 | 79.2 | 79.3 |
128 | — | — | 79.1 | 79.3 |
p值 | 层数 | ||||||
---|---|---|---|---|---|---|---|
2 | 4 | 8 | 16 | 32 | 64 | 128 | |
p=0.9 | 80.7 | 81.4 | 83.0 | 84.4 | 84.9 | 84.5 | 84.4 |
p=0.8 | 81.0 | 81.1 | 82.6 | 84.2 | 84.2 | 84.0 | 83.5 |
p=0.7 | 80.4 | 81.6 | 82.2 | 83.7 | 83.9 | 83.1 | 83.1 |
p=0.6 | 81.5 | 79.5 | 80.9 | 82.2 | 82.7 | 81.9 | 82.4 |
p=0.5 | 81.3 | 79.4 | 80.8 | 81.9 | 82.1 | 81.9 | 82.1 |
p=0.4 | 80.7 | 76.5 | 79.7 | 81.5 | 81.4 | 81.9 | 81.7 |
p=0.3 | 80.1 | 75.4 | 79.3 | 79.9 | 80.9 | 81.3 | 80.9 |
p=0.2 | 80.0 | 72.4 | 79.1 | 79.9 | 80.0 | 80.6 | 80.1 |
表6 Cora数据集p参数实验(%)
p值 | 层数 | ||||||
---|---|---|---|---|---|---|---|
2 | 4 | 8 | 16 | 32 | 64 | 128 | |
p=0.9 | 80.7 | 81.4 | 83.0 | 84.4 | 84.9 | 84.5 | 84.4 |
p=0.8 | 81.0 | 81.1 | 82.6 | 84.2 | 84.2 | 84.0 | 83.5 |
p=0.7 | 80.4 | 81.6 | 82.2 | 83.7 | 83.9 | 83.1 | 83.1 |
p=0.6 | 81.5 | 79.5 | 80.9 | 82.2 | 82.7 | 81.9 | 82.4 |
p=0.5 | 81.3 | 79.4 | 80.8 | 81.9 | 82.1 | 81.9 | 82.1 |
p=0.4 | 80.7 | 76.5 | 79.7 | 81.5 | 81.4 | 81.9 | 81.7 |
p=0.3 | 80.1 | 75.4 | 79.3 | 79.9 | 80.9 | 81.3 | 80.9 |
p=0.2 | 80.0 | 72.4 | 79.1 | 79.9 | 80.0 | 80.6 | 80.1 |
1 | 白铂, 刘玉婷, 马驰骋, 等. 图神经网络[J]. 中国科学: 数学, 2020, 50(3): 367-384. |
BAI B, LIU Y T, MA C C, et al. Graph neural network[J]. Scientia Sinica (Mathematica), 2020, 50(3): 367-384. (in Chinese) | |
2 | 钱榕, 张茹, 张克君, 等. 融合全局和局部特征的胶囊图神经网络[J]. 吉林大学学报(工学版), 2021, 51(3): 1048-1054. |
QIAN R, ZHANG R, ZHANG K J, et al. Capsule graph neural network based on global and local features fusion[J]. Journal of Jilin University (Engineering and Technology Edition), 2021, 51(3): 1048-1054. (in Chinese) | |
3 | 高留杰, 赵文, 张君福, 等. G2S: 基于语义块的知识图谱问答语义解析[J]. 电子学报, 2021, 49(6): 1132-1141. |
GAO L J, ZHAO W, ZHANG J F, et al. G2S: semantic segment based semantic parsing for question answering over knowledge graph[J]. Acta Electronica Sinica, 2021, 49(6): 1132-1141. (in Chinese) | |
4 | HAMILTON W L, YING R, LESKOVEC J. Inductive representation learning on large graphs[C]//NIPS'17: Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook, NY: Curran Associates Inc, 2017: 1025-1035. |
5 | WANG X L, YE Y F, GUPTA A. Zero-shot recognition via semantic embeddings and knowledge graphs[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 6857-6866. |
6 | LIN W Y, JI S X, LI B C. Adversarial attacks on link prediction algorithms based on graph neural networks[C]//ASIA CCS'20: Proceedings of the 15th ACM Asia Conference on Computer and Communications Security. New York: ACM, 2020: 370-380. |
7 | 车向北, 康文倩, 邓彬, 等. 一种基于图神经网络的SDN路由性能预测模型[J]. 电子学报, 2021, 49(3): 484-491. |
CHE X B, KANG W Q, DENG B, et al. A prediction model of SDN routing performance based on graph neural network[J]. Acta Electronica Sinica, 2021, 49(3): 484-491. (in Chinese) | |
8 | ZITNIK M, AGRAWAL M, LESKOVEC J. Modeling polypharmacy side effects with graph convolutional networks[J]. Bioinformatics(Oxford, England), 2018, 34(13): i457-i466. |
9 | PEROZZI B, AL-RFOU R, SKIENA S. DeepWalk: online learning of social representations[C]//KDD'14: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2014: 701-710. |
10 | KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks[EB/OL]. (2016-09-09). . |
11 | KARPATHY A, TODERICI G, SHETTY S, et al. Large-scale video classification with convolutional neural networks[C]//CVPR'14: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus:IEEE, 2014: 1725-1732. |
12 | SHELHAMER E, LONG J, DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence. Piscataway: IEEE, 2016, 39(4): 640-651. |
13 | CHEN M, WEI Z, HUANG Z, et al. Simple and deep graph convolutional networks[C]//International Conference on Machine Learning. Vienna: ICML, 2020: 1725-1735. |
14 | XU K, LI C, TIAN Y, et al. Representation learning on graphs with jumping knowledge networks[C]//International Conference on Machine Learning. Stockholm: ICML, 2018: 5453-5462. |
15 | LI G H, MÜLLER M, THABET A, et al. DeepGCNs: can GCNs go as deep as CNNs?[C]//2019 IEEE/CVF International Conference on Computer Vision(ICCV). Piscataway: IEEE, 2019: 9266-9275. |
16 | WU H B, GU X D. Towards dropout training for convolutional neural networks[J]. Neural Networks, 2015, 71: 1-10. |
17 | DO T H, NGUYEN D M, BEKOULIS G, et al. Graph convolutional neural networks with node transition probability-based message passing and DropNode regularization[J]. Expert Systems With Applications, 2021, 174: 114711. |
18 | RONG Y, HUANG W B, XU T Y, et al. DropEdge: towards deep graph convolutional networks on node classification[EB/OL]. (2019-07-25). . |
19 | SRIVASTAVA N, HINTON G E, KRIZHEVSKY A, et al. Dropout: a simple way to prevent neural networks from overfitting[J]. Journal of Machine Learning Research, 2014, 15(1): 1929-1958. |
20 | SCARSELLI F, GORI M, TSOI A C, et al. The graph neural network model[J]. IEEE Transactions on Neural Networks, 2009, 20(1): 61-80. |
21 | BRUNA J, ZAREMBA W, SZLAM A, et al. Spectral networks and locally connected networks on graphs[EB/OL].(2013-12-21). . |
22 | 郭嘉琰, 李荣华, 张岩, 等. 基于图神经网络的动态网络异常检测算法[J]. 软件学报, 2020, 31(3): 748-762. |
GUO J Y, LI R H, ZHANG Y, et al. Graph neural network based anomaly detection in dynamic networks[J]. Journal of Software, 2020, 31(3): 748-762. (in Chinese) | |
23 | WANG R, HUANG C Y, WANG X Y. Global relation reasoning graph convolutional networks for human pose estimation[J]. IEEE Access, 2020, 8: 38472-38480. |
24 | YU K, JIANG H, LI T, et al. Data fusion oriented graph convolution network model for rumor detection[J]. IEEE Transactions on Network and Service Management, 2020, 17(4): 2171-2181. |
25 | HE K M, ZHANG X Y, REN S Q, et al. Identity Mappings in Deep Residual Networks[C]//European Conference on Computer Vision. Cham: Springer, 2016: 630-645. |
26 | OONO K, SUZUKI T. Graph neural networks exponentially lose expressive power for node classification[EB/OL]. (2019-05-27). . |
27 | SEN P, NAMATA G, BILGIC M, et al. Collective classification in network data[J]. AI Magazine, 2008, 29(3): 93. |
[1] | 范兵兵, 何庭建, 张聪炫, 陈震, 黎明. 联合遮挡约束与残差补偿的特征金字塔光流计算方法[J]. 电子学报, 2023, 51(3): 648-657. |
[2] | 李滔, 董秀成, 林宏伟. 基于深监督跨尺度注意力网络的深度图像超分辨率重建[J]. 电子学报, 2023, 51(1): 128-138. |
[3] | 王相海, 赵晓阳, 王鑫莹, 赵克云, 宋传鸣. 非抽取小波边缘学习深度残差网络的单幅图像超分辨率重建[J]. 电子学报, 2022, 50(7): 1753-1765. |
[4] | 胡向东, 李之涵. 基于胶囊网络的工业互联网入侵检测方法[J]. 电子学报, 2022, 50(6): 1457-1465. |
[5] | 廖勇, 李玉杰. 一种轻量化低复杂度的FDD大规模MIMO系统CSI反馈方法[J]. 电子学报, 2022, 50(5): 1211-1217. |
[6] | 袁冠, 邴睿, 刘肖, 代伟, 张艳梅, 蔡卓. 基于时空图神经网络的手势识别[J]. 电子学报, 2022, 50(4): 921-931. |
[7] | 刘小洋, 刘加苗, 刘超, 张宜浩. 融合字符级滑动窗口和深度残差网络的僵尸网络DGA域名检测方法[J]. 电子学报, 2022, 50(1): 250-256. |
[8] | 张聿远, 张立民, 闫文君. 基于深度多级残差网络的低信噪比下空频分组码识别方法[J]. 电子学报, 2022, 50(1): 79-88. |
[9] | 苏志刚, 武瑞, 郝敬堂. 面向无源定位的直达波传感器识别算法[J]. 电子学报, 2021, 49(6): 1178-1186. |
[10] | 乔思波, 庞善臣, 王敏, 翟雪, 于世行, 丁桐. 基于残差混合注意力机制的脑部CT图像分类卷积神经网络模型[J]. 电子学报, 2021, 49(5): 984-991. |
[11] | 车向北, 康文倩, 邓彬, 杨柯涵, 李剑. 一种基于图神经网络的SDN路由性能预测模型[J]. 电子学报, 2021, 49(3): 484-491. |
[12] | 江泽涛, 钱艺, 伍旭, 张少钦. 一种基于ARD‑GAN的低照度图像增强方法[J]. 电子学报, 2021, 49(11): 2160-2165. |
[13] | 周涛, 霍兵强, 陆惠玲, 任海玲. 残差神经网络及其在医学图像处理中的应用研究[J]. 电子学报, 2020, 48(7): 1436-1447. |
[14] | 秦鑫, 黄洁, 查雄, 骆丽萍, 胡德秀. 基于扩张残差网络的雷达辐射源信号识别[J]. 电子学报, 2020, 48(3): 456-462. |
[15] | 罗会兰, 敖阳, 袁璞. 一种生成对抗网络用于图像修复的方法[J]. 电子学报, 2020, 48(10): 1891-1898. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||