电子学报 ›› 2022, Vol. 50 ›› Issue (10): 2398-2408.DOI: 10.12263/DZXB.20201372
收稿日期:
2020-12-01
修回日期:
2021-01-29
出版日期:
2022-10-25
通讯作者:
作者简介:
基金资助:
Received:
2020-12-01
Revised:
2021-01-29
Online:
2022-10-25
Published:
2022-10-11
Corresponding author:
摘要:
网络剪枝是一种有效的卷积神经网络压缩方法.多数现有压缩方法因迭代剪枝了“不重要”的网络结构,一方面破坏了网络结构的信息整体性,另一方面其迭代操作耗费了大量的计算资源与时间.为了解决上述问题,论文从网络结构全局考虑,提出基于自适应层信息熵的卷积神经网络压缩方法.首先,在获取压缩网络结构的过程中,本文设计了一种端到端的结构化网络剪枝方案,将卷积层看作一个整体,利用层信息熵之间的关联性直接确定各卷积层过滤器的保留率,避免迭代剪枝训练造成的信息损失.其次,对剪裁后的网络进行重训练时,综合考虑压缩过程中使用的层信息熵指标,通过对卷积层与批归一化(Batch Normalization,BN)层进行自适应联合嫁接,让网络学习到更多的信息,提升网络性能.针对3种主流网络在不同的数据集上进行了实验,验证了所提方法的有效性与优越性.例如在CIFAR-10上,针对ResNet-56,相比于基线网络,在计算量压缩36.2%的情况下,本文方法准确率提升了1%;针对ResNet-110,在计算量压缩52.4%的情况下,本文方法准确率提升了1.42%;针对轻量型网络MobileNetV2,在计算量压缩55.2%的情况下,本文方法准确率提升了1.29%.
中图分类号:
魏钰轩, 陈莹. 基于自适应层信息熵的卷积神经网络压缩[J]. 电子学报, 2022, 50(10): 2398-2408.
Yu-xuan WEI, Ying CHEN . Convolutional Neural Network Compression Based on Adaptive Layer Entropy[J]. Acta Electronica Sinica, 2022, 50(10): 2398-2408.
符号 | 描述 | 符号 | 描述 |
---|---|---|---|
N | 卷积神经网络模型 | 模型原始层信息熵区间 | |
L | 卷积层个数 | 卷积层过滤器最大保留率 | |
第 | 模型层信息熵区间 | ||
第 | 第i层过滤器保留率 | ||
信息熵 | 网络 | ||
自适应嫁接系数 | 网络 | ||
SE | 最小信息熵 | ||
BE | 最大信息熵 | 剪枝后第 |
表1 ALE中符号定义及描述
符号 | 描述 | 符号 | 描述 |
---|---|---|---|
N | 卷积神经网络模型 | 模型原始层信息熵区间 | |
L | 卷积层个数 | 卷积层过滤器最大保留率 | |
第 | 模型层信息熵区间 | ||
第 | 第i层过滤器保留率 | ||
信息熵 | 网络 | ||
自适应嫁接系数 | 网络 | ||
SE | 最小信息熵 | ||
BE | 最大信息熵 | 剪枝后第 |
模型 | 准确率 | 计算量(压缩率) | 参数量(压缩率) |
---|---|---|---|
ResNet-56 | 93.26% | 125.49M(0.0%) | 0.85M(0.0%) |
PF-A[ | 93.10% | 112.00M(10.7%) | 0.77M(9.4%) |
PF-B[ | 93.06% | 90.90M(27.6%) | 0.73M(14.1%) |
SSS[ | 93.39% | 89.35M(28.8%) | 0.59M(30.6%) |
HRank1[ | 93.52% | 88.72M(29.3%) | 0.71M(16.8%) |
LEGR1[ | 94.10% | 87.80M(30.0%) | |
NISP[ | 93.01% | 81.00M(35.5%) | 0.49M(42.4%) |
ALE-100% | 94.26% | 80.10M(36.2%) | 0.54M(32.9%) |
GAL-0.6[ | 93.38% | 78.30M(37.6%) | 0.75M(11.8%) |
HRank2[ | 93.17% | 62.72M(50.0%) | 0.49M(42.4%) |
Hinge[ | 93.69% | 62.72M(50.0%) | 0.44M(51.27%) |
KSE[ | 93.23% | 60.00M(50.0%) | 0.43M(49.4%) |
FPGM[ | 93.26% | 59.40M(52.6%) | |
LEGR2[ | 93.70% | 58.90M(53.1%) | |
ABC-70%[ | 93.23% | 58.54M(54.1%) | 0.39M(54.2%) |
ALE-60% | 93.64% | 49.53M(60.5%) | 0.36M(57.6%) |
C-SGD[ | 93.31% | 49.13M(60.85%) |
表2 ResNet-56在CIFAR-10上结果
模型 | 准确率 | 计算量(压缩率) | 参数量(压缩率) |
---|---|---|---|
ResNet-56 | 93.26% | 125.49M(0.0%) | 0.85M(0.0%) |
PF-A[ | 93.10% | 112.00M(10.7%) | 0.77M(9.4%) |
PF-B[ | 93.06% | 90.90M(27.6%) | 0.73M(14.1%) |
SSS[ | 93.39% | 89.35M(28.8%) | 0.59M(30.6%) |
HRank1[ | 93.52% | 88.72M(29.3%) | 0.71M(16.8%) |
LEGR1[ | 94.10% | 87.80M(30.0%) | |
NISP[ | 93.01% | 81.00M(35.5%) | 0.49M(42.4%) |
ALE-100% | 94.26% | 80.10M(36.2%) | 0.54M(32.9%) |
GAL-0.6[ | 93.38% | 78.30M(37.6%) | 0.75M(11.8%) |
HRank2[ | 93.17% | 62.72M(50.0%) | 0.49M(42.4%) |
Hinge[ | 93.69% | 62.72M(50.0%) | 0.44M(51.27%) |
KSE[ | 93.23% | 60.00M(50.0%) | 0.43M(49.4%) |
FPGM[ | 93.26% | 59.40M(52.6%) | |
LEGR2[ | 93.70% | 58.90M(53.1%) | |
ABC-70%[ | 93.23% | 58.54M(54.1%) | 0.39M(54.2%) |
ALE-60% | 93.64% | 49.53M(60.5%) | 0.36M(57.6%) |
C-SGD[ | 93.31% | 49.13M(60.85%) |
模型 | 准确率 | 计算量(压缩率) | 参数量(压缩率) |
---|---|---|---|
ResNet-110 | 93.50% | 252.89M(0.0%) | 1.72M(0.0%) |
GAL-0.1[ | 93.59% | 205.7M(18.7%) | 1.65M(4.07%) |
NISP[ | 93.32% | 143.35M(43.3%) | |
GAL-0.5[ | 92.74% | 130.20M(48.5%) | 0.95M(44.8%) |
FPGM[ | 93.74% | 121.00M(52.2%) | |
ALE-80% | 94.92% | 120.29M(52.4%) | 0.89M(48.3%) |
HRank[ | 93.36% | 105.70M(58.2%) | 0.70M(59.3%) |
ALE-60% | 94.36% | 92.68M(63.4%) | 0.69M(59.9%) |
ABC-60%[ | 93.58% | 89.87M(64.5%) | 0.56M(67.4%) |
表3 ResNet-110在CIFAR-10上结果
模型 | 准确率 | 计算量(压缩率) | 参数量(压缩率) |
---|---|---|---|
ResNet-110 | 93.50% | 252.89M(0.0%) | 1.72M(0.0%) |
GAL-0.1[ | 93.59% | 205.7M(18.7%) | 1.65M(4.07%) |
NISP[ | 93.32% | 143.35M(43.3%) | |
GAL-0.5[ | 92.74% | 130.20M(48.5%) | 0.95M(44.8%) |
FPGM[ | 93.74% | 121.00M(52.2%) | |
ALE-80% | 94.92% | 120.29M(52.4%) | 0.89M(48.3%) |
HRank[ | 93.36% | 105.70M(58.2%) | 0.70M(59.3%) |
ALE-60% | 94.36% | 92.68M(63.4%) | 0.69M(59.9%) |
ABC-60%[ | 93.58% | 89.87M(64.5%) | 0.56M(67.4%) |
模型 | 准确率 | 计算量(压缩率) | 参数量(压缩率) |
---|---|---|---|
VGGNet | 93.96% | 313.74M(0.0%) | 14.95M(0.0%) |
SSS[ | 93.02% | 183.13M(42.3%) | 3.93M(73.7%) |
GAL-0.1[ | 93.42% | 171.89M(45.2%) | 2.67M(82.2%) |
HRank1[ | 92.34% | 108.61M(65.4%) | 2.64M(82.3%) |
HRank2[ | 91.23% | 73.70M(76.5%) | 1.78M(88.2%) |
ALE-90% | 92.85% | 71.29M(77.28%) | 2.87M(80.8%) |
LEGR[ | 92.40% | 70.30M(77.6%) | --- |
GAL-0.2[ | 91.89% | 65.85M(79.0%) | 3.35M(77.6%) |
ALE-80% | 92.81% | 60.23M(80.88%) | 2.46M(83.6%) |
表4 VGGNet在CIFAR-10上结果
模型 | 准确率 | 计算量(压缩率) | 参数量(压缩率) |
---|---|---|---|
VGGNet | 93.96% | 313.74M(0.0%) | 14.95M(0.0%) |
SSS[ | 93.02% | 183.13M(42.3%) | 3.93M(73.7%) |
GAL-0.1[ | 93.42% | 171.89M(45.2%) | 2.67M(82.2%) |
HRank1[ | 92.34% | 108.61M(65.4%) | 2.64M(82.3%) |
HRank2[ | 91.23% | 73.70M(76.5%) | 1.78M(88.2%) |
ALE-90% | 92.85% | 71.29M(77.28%) | 2.87M(80.8%) |
LEGR[ | 92.40% | 70.30M(77.6%) | --- |
GAL-0.2[ | 91.89% | 65.85M(79.0%) | 3.35M(77.6%) |
ALE-80% | 92.81% | 60.23M(80.88%) | 2.46M(83.6%) |
模型 | 准确率 | 计算量(压缩率) | 参数量(压缩率) |
---|---|---|---|
MobileNetV2 | 92.25% | 91.15M(0.0%) | 2.26M(0.0%) |
ALE-90% | 93.54% | 40.86M(55.2%) | 0.94M(58.4%) |
ALE-70% | 92.79% | 32.66M(64.2%) | 0.83M(63.3%) |
表5 MobileNetV2在CIFAR-10上结果
模型 | 准确率 | 计算量(压缩率) | 参数量(压缩率) |
---|---|---|---|
MobileNetV2 | 92.25% | 91.15M(0.0%) | 2.26M(0.0%) |
ALE-90% | 93.54% | 40.86M(55.2%) | 0.94M(58.4%) |
ALE-70% | 92.79% | 32.66M(64.2%) | 0.83M(63.3%) |
模型 | 准确率 | 计算量(压缩率) | 参数量(压缩率) |
---|---|---|---|
ResNet-56 | 71.92% | 125.49M(0.0%) | 0.85M(0.0%) |
PF-A[ | 70.42% | 112.44M(10.4%) | 0.77M(9.4%) |
PF-B[ | 69.95% | 90.85M(27.6%) | 0.73M(13.7%) |
ALE-90% | 70.91% | 64.77M(48.4%) | 0.53M(37.7%) |
ALE-80% | 70.72% | 57.97M(53.8%) | 0.47M(44.7%) |
表6 ResNet-56在CIFAR-100上结果
模型 | 准确率 | 计算量(压缩率) | 参数量(压缩率) |
---|---|---|---|
ResNet-56 | 71.92% | 125.49M(0.0%) | 0.85M(0.0%) |
PF-A[ | 70.42% | 112.44M(10.4%) | 0.77M(9.4%) |
PF-B[ | 69.95% | 90.85M(27.6%) | 0.73M(13.7%) |
ALE-90% | 70.91% | 64.77M(48.4%) | 0.53M(37.7%) |
ALE-80% | 70.72% | 57.97M(53.8%) | 0.47M(44.7%) |
模型 | 准确率 | 计算量(压缩率) | 参数量(压缩率) |
---|---|---|---|
ResNet-56 | 96.38% | 125.49M(0.0%) | 0.85M(0.0%) |
GAL-0.6[ | 95.82% | 73.58M(41.4%) | 0.56M(34.1%) |
ALE-60% | 96.52% | 50.17M(60.0%) | 0.35M(58.8%) |
ResNet-110 | 96.36% | 252.89M(0.0%) | 1.72M(0.0%) |
GAL-0.06[ | 96.63% | 120.77M(52.2%) | 0.83M(51.7%) |
ALE-60% | 96.65% | 98.50M(61.1%) | 0.71M(58.7%) |
表7 ResNet-56和ResNet-110在SVHN上结果
模型 | 准确率 | 计算量(压缩率) | 参数量(压缩率) |
---|---|---|---|
ResNet-56 | 96.38% | 125.49M(0.0%) | 0.85M(0.0%) |
GAL-0.6[ | 95.82% | 73.58M(41.4%) | 0.56M(34.1%) |
ALE-60% | 96.52% | 50.17M(60.0%) | 0.35M(58.8%) |
ResNet-110 | 96.36% | 252.89M(0.0%) | 1.72M(0.0%) |
GAL-0.06[ | 96.63% | 120.77M(52.2%) | 0.83M(51.7%) |
ALE-60% | 96.65% | 98.50M(61.1%) | 0.71M(58.7%) |
模型 | mAP | Rank1 | 计算量 (压缩率) | 参数量 (压缩率) |
---|---|---|---|---|
ResNet-50 | 65.1% | 84.1% | 4087.14M (0.0%) | 24.99M (0.0%) |
ALE-90% | 62.2% | 82.1% | 1577.95M (61.4%) | 8.28M (66.9%) |
ALE-80% | 61.5% | 81.7% | 1419.49M (65.3%) | 7.48M (70.1%) |
表8 ResNet-50在Market1501上结果
模型 | mAP | Rank1 | 计算量 (压缩率) | 参数量 (压缩率) |
---|---|---|---|---|
ResNet-50 | 65.1% | 84.1% | 4087.14M (0.0%) | 24.99M (0.0%) |
ALE-90% | 62.2% | 82.1% | 1577.95M (61.4%) | 8.28M (66.9%) |
ALE-80% | 61.5% | 81.7% | 1419.49M (65.3%) | 7.48M (70.1%) |
模型 | mAP | Rank1 | 计算量 (压缩率) | 参数量 (压缩率) |
---|---|---|---|---|
ResNet-50 | 52.3% | 72.1% | 4087.14M (0.0%) | 24.89M (0.0%) |
ALE-90% | 51.7% | 71.1% | 1639.11M (59.9%) | 7.41M (70.2%) |
ALE-80% | 51.0% | 70.5% | 1508.50M (63.1%) | 7.22M (71.0%) |
表9 ResNet-50在Duke上结果
模型 | mAP | Rank1 | 计算量 (压缩率) | 参数量 (压缩率) |
---|---|---|---|---|
ResNet-50 | 52.3% | 72.1% | 4087.14M (0.0%) | 24.89M (0.0%) |
ALE-90% | 51.7% | 71.1% | 1639.11M (59.9%) | 7.41M (70.2%) |
ALE-80% | 51.0% | 70.5% | 1508.50M (63.1%) | 7.22M (71.0%) |
模型 | 准确率 | 计算量(压缩率) | 参数量(压缩率) |
---|---|---|---|
ResNet-56 | 71.92% | 125.49M(0.0%) | 0.85M(0.0%) |
ALE-80%-10 | 69.31% | 69.93M(44.3%) | 0.52M(38.8%) |
ALE-80%-15 | 70.06% | 68.31M(45.6%) | 0.53M(37.7%) |
ALE-90%-122 | 70.91% | 64.77M(48.4%) | 0.53M(37.7%) |
ALE-80%-60 | 70.48% | 63.74M(49.2%) | 0.48M(43.5%) |
ALE-80%-20 | 70.26% | 61.97M(50.6%) | 0.50M(41.2%) |
ALE-80%-90 | 70.93% | 61.67M(50.9%) | 0.49M(42.4%) |
ALE-80%-30 | 70.19% | 61.50M(51.0%) | 0.47M(44.7%) |
ALE-80%-122 | 70.72% | 57.97M(53.8%) | 0.47M(44.7%) |
ALE-80%-150 | 70.65% | 55.35M(55.9%) | 0.44M(48.2%) |
ALE-80%-5 | 69.17% | 50.32M(59.9%) | 0.36M(57.6%) |
表10 ResNet-56在CIFAR-100上结果
模型 | 准确率 | 计算量(压缩率) | 参数量(压缩率) |
---|---|---|---|
ResNet-56 | 71.92% | 125.49M(0.0%) | 0.85M(0.0%) |
ALE-80%-10 | 69.31% | 69.93M(44.3%) | 0.52M(38.8%) |
ALE-80%-15 | 70.06% | 68.31M(45.6%) | 0.53M(37.7%) |
ALE-90%-122 | 70.91% | 64.77M(48.4%) | 0.53M(37.7%) |
ALE-80%-60 | 70.48% | 63.74M(49.2%) | 0.48M(43.5%) |
ALE-80%-20 | 70.26% | 61.97M(50.6%) | 0.50M(41.2%) |
ALE-80%-90 | 70.93% | 61.67M(50.9%) | 0.49M(42.4%) |
ALE-80%-30 | 70.19% | 61.50M(51.0%) | 0.47M(44.7%) |
ALE-80%-122 | 70.72% | 57.97M(53.8%) | 0.47M(44.7%) |
ALE-80%-150 | 70.65% | 55.35M(55.9%) | 0.44M(48.2%) |
ALE-80%-5 | 69.17% | 50.32M(59.9%) | 0.36M(57.6%) |
模型 | 准确率 | 计算量(压缩率) | 参数量(压缩率) |
---|---|---|---|
ResNet-56 | 93.26% | 125.49M(0.0%) | 0.85M(0.0%) |
ALE-60% | 93.64% | 49.53M(60.5%) | 0.36M(57.6%) |
20%-ALE-60% | 93.50% | 52.80M(57.9%) | 0.37M(56.5%) |
ResNet-110 | 93.50% | 252.89M(0.0%) | 1.72M(0.0%) |
ALE-80% | 94.92% | 120.29M(52.4%) | 0.89M(48.3%) |
20%-ALE-80% | 94.55% | 131.50M(48.0%) | 0.96M(44.2%) |
MobileNetV2 | 92.25% | 91.15M(0.0%) | 2.26M(0.0%) |
ALE-90% | 93.54% | 40.86M(55.2%) | 0.94M(58.4%) |
20%- ALE-90% | 94.11% | 46.20M(49.3%) | 1.08M(52.2%) |
表11 层最小保留率的影响
模型 | 准确率 | 计算量(压缩率) | 参数量(压缩率) |
---|---|---|---|
ResNet-56 | 93.26% | 125.49M(0.0%) | 0.85M(0.0%) |
ALE-60% | 93.64% | 49.53M(60.5%) | 0.36M(57.6%) |
20%-ALE-60% | 93.50% | 52.80M(57.9%) | 0.37M(56.5%) |
ResNet-110 | 93.50% | 252.89M(0.0%) | 1.72M(0.0%) |
ALE-80% | 94.92% | 120.29M(52.4%) | 0.89M(48.3%) |
20%-ALE-80% | 94.55% | 131.50M(48.0%) | 0.96M(44.2%) |
MobileNetV2 | 92.25% | 91.15M(0.0%) | 2.26M(0.0%) |
ALE-90% | 93.54% | 40.86M(55.2%) | 0.94M(58.4%) |
20%- ALE-90% | 94.11% | 46.20M(49.3%) | 1.08M(52.2%) |
模型 | 嫁接个数 | 准确率 |
---|---|---|
ResNet-110 | 93.50% | |
ALE-80% | 1 | 93.39% |
2 | 93.86% | |
3 | 94.55% | |
4 | 94.21% | |
5 | 94.71% | |
6 | 94.92% |
表12 嫁接网络个数的影响
模型 | 嫁接个数 | 准确率 |
---|---|---|
ResNet-110 | 93.50% | |
ALE-80% | 1 | 93.39% |
2 | 93.86% | |
3 | 94.55% | |
4 | 94.21% | |
5 | 94.71% | |
6 | 94.92% |
模型 | 准确率 | 计算量(压缩率) | 参数量 (压缩率) |
---|---|---|---|
ResNet-56 | 93.26% | 125.49M(0.0%) | 0.85M(0.0%) |
ALE-100%(扩) | 94.26% | 80.10M(36.2%) | 0.54M(32.9%) |
ALE-100% | 94.11% | 81.47M(35.1%) | 0.59M(30.6%) |
ALE-60%(扩) | 93.64% | 49.53M(60.5%) | 0.36M(57.6%) |
ALE-60% | 93.37% | 49.38M(60.7%) | 0.36M(57.6%) |
ResNet-110 | 93.50% | 252.89M(0.0%) | 1.72M(0.0%) |
ALE-80%(扩) | 94.92% | 120.29M(52.4%) | 0.89M(48.3%) |
ALE-80% | 94.81% | 120.88M(52.2%) | 0.91M(47.1%) |
MobileNetV2 | 92.25% | 91.15M(0.0%) | 2.26M(0.0%) |
ALE-90%(扩) | 93.54% | 40.86M(55.2%) | 0.94M(58.4%) |
ALE-90% | 93.42% | 38.90M(57.3%) | 0.89M(60.6%) |
表13 是否扩充信息熵实验对比
模型 | 准确率 | 计算量(压缩率) | 参数量 (压缩率) |
---|---|---|---|
ResNet-56 | 93.26% | 125.49M(0.0%) | 0.85M(0.0%) |
ALE-100%(扩) | 94.26% | 80.10M(36.2%) | 0.54M(32.9%) |
ALE-100% | 94.11% | 81.47M(35.1%) | 0.59M(30.6%) |
ALE-60%(扩) | 93.64% | 49.53M(60.5%) | 0.36M(57.6%) |
ALE-60% | 93.37% | 49.38M(60.7%) | 0.36M(57.6%) |
ResNet-110 | 93.50% | 252.89M(0.0%) | 1.72M(0.0%) |
ALE-80%(扩) | 94.92% | 120.29M(52.4%) | 0.89M(48.3%) |
ALE-80% | 94.81% | 120.88M(52.2%) | 0.91M(47.1%) |
MobileNetV2 | 92.25% | 91.15M(0.0%) | 2.26M(0.0%) |
ALE-90%(扩) | 93.54% | 40.86M(55.2%) | 0.94M(58.4%) |
ALE-90% | 93.42% | 38.90M(57.3%) | 0.89M(60.6%) |
1 | ZHU F, ZHU L, YANG Y. Sim-real joint reinforcement transfer for 3d indoor navigation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. California: IEEE, 2019: 11388-11397. |
2 | 权宇, 李志欣, 张灿龙, 等. 融合深度扩张网络和轻量化网络的目标检测模型[J]. 电子学报, 2020, 48(2): 390-397. |
QUAN Y, LI Z X, ZHANG C L, et al. Fusing deep dilated convolutions network and light-weight network for object detection[J]. Acta Electronica Sinica, 2020, 48(2): 390-397. (in Chinese) | |
3 | 周涛, 霍兵强, 陆惠玲, 等. 残差神经网络及其在医学图像处理中的应用研究[J]. 电子学报, 2020, 48(7): 1436-1447. |
ZHOU T, HUO B Q, LU H L, et al. Research on residual neural network and its application on medical image processing[J]. Acta Electronica Sinica, 2020, 48(7): 1436-1447. (in Chinese) | |
4 | DING X, DING G, GUO Y, et al. Centripetal sgd for pruning very deep convolutional networks with complicated structure[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. California: IEEE, 2019: 4943-4953. |
5 | 饶川, 陈靓影, 徐如意, 等. 一种基于动态量化编码的深度神经网络压缩方法[J]. 自动化学报, 2019, 45(10):1960-1968. |
RAO C, CHEN J Y, XU R Y, et al. A dynamic quantization coding based deep neural network compression method[J]. Acta Automatica Sinica, 2019, 45(10): 1960-1968. (in Chinese) | |
6 | ADRIANA R, NICOLAS B, SAMIRA E K, et al. Fitnets: Hints for thin deep nets[C]//International Conference on Learning Representations. California: OpenReview.net, 2015: 1-13. |
7 | ZHANG X, ZOU J, HE K, et al. Accelerating very deep convolutional networks for classification and detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 38(10): 1943-1955. |
8 | WANG Y, XU C, XU C, et al. Beyond filters: Compact feature map for portable deep model[C]//International Conference on Machine Learning. Sydney: ACM, 2017: 3703-3711. |
9 | HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778. |
10 | SZEGEDY C, LIU W, JIA Y, et al. Going deeper with convolutions[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 1-9. |
11 | CARREIRA-PERPINAN M A, IDELBAYEV Y. Learning-compression algorithms for neural net pruning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 8532-8541. |
12 | HAN S, POOL J, TRAN J, et al. Learning both weights and connections for efficient neural network[C]//Advances in Neural Information Processing Systems. Quebec: MIT Press, 2015: 1135-1143. |
13 | WEN W, WU C, WANG Y, et al. Learning structured sparsity in deep neural networks[C]//Advances in Neural Information Processing Systems. Barcelona: MIT Press, 2016: 2074-2082. |
14 | LUO J H, WU J, LIN W. Thinet: A filter level pruning method for deep neural network compression[C]//Proceedings of the IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 5058-5066. |
15 | WANG D, ZHOU L, ZHANG X, et al. Exploring Linear Relationship in Feature Map Subspace for Convnets Compression[EB/OL]. (2018-03-15)[2020-12-01]. . |
16 | LI H, KADAV A, DURDANOVIC I, et al. Pruning filters for efficient convnets[C]//International Conference on Learning Representations. Toulon: OpenReview.net, 2017: 1-13. |
17 | YE J, LU X, LIN Z, et al. Rethinking the smaller-norm-less-informative assumption in channel pruning of convolution layers[C]//International Conference on Learning Representations. Vancouver: OpenReview.net, 2018: 1-11. |
18 | ZHUO H, QIAN X, FU Y, et al. Scsp: Spectral Clustering Filter Pruning with Soft Self-Adaption Manners[EB/OL]. (2018-07-14)[2020-12-01]. . |
19 | HE Y, LIU P, WANG Z, et al. Filter pruning via geometric median for deep convolutional neural networks acceleration[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. California: IEEE, 2019: 4340-4349. |
20 | LIU Z, SUN M, ZHOU T, et al. Rethinking the value of network pruning[C]//International Conference on Learning Representations. New Orleans: OpenReview.net, 2019: 1-21. |
21 | LIN S, JI R, YAN C, et al. Towards optimal structured cnn pruning via generative adversarial learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. California: IEEE, 2019: 2790-2799. |
22 | LIN M, JI R, ZHANG Y, et al. Channel pruning via automatic structure search[C]//International Joint Conference on Artificial Intelligence. Yokohama: Morgan Kaufmann, 2020: 1-7. |
23 | LIU Z, ZHANG X, SHEN Z, et al. Joint Multi-Dimension Pruning[EB/OL]. (2020-05-18)[2020-12-01]. . |
24 | MENG F, CHENG H, LI K, et al. Filter grafting for deep neural networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 6599-6607. |
25 | LUO J H, WU J. An Entropy-Based Pruning Method for CNN Compression[EB/OL]. (2017-07-19)[2020-12-01]. . |
26 | LI Y, LIN S, ZHANG B, et al. Exploiting kernel sparsity and entropy for interpretable CNN compression[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. California: IEEE, 2019: 2800-2809. |
27 | WANG L, XIONG Y, WANG Z, et al. Temporal segment networks: Towards good practices for deep action recognition[C]//Proceedings of the European Conference on Computer Vision, Amsterdam. Holland: Springer, 2016: 20-36. |
28 | KRIZHEVSKY A, HINTON G. Learning multiple layers of features from tiny images[J]. Technical Report, 2009, 1(1): 1-60. |
29 | NETZER Y, WANG T, COATES A, et al. Reading digits in natural images with unsupervised feature learning[J]. NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011, 1(1): 1-9. |
30 | ZHENG L, SHEN L, TIAN L, et al. Scalable person re-identification: A benchmark[C]//Proceedings of the IEEE International Conference on Computer Vision. Santiago: IEEE, 2015: 1116-1124. |
31 | RISTANI E, SOLERA F, ZOU R, et al. Performance measures and a data set for multi-target, multi-camera tracking[C]//Proceedings of the European Conference on Computer Vision. Amsterdam: Springer, 2016: 17-35. |
32 | HUANG Z, WANG N. Data-driven sparse structure selection for deep neural networks[C]//Proceedings of the European Conference on Computer Vision. Munich: Springer, 2018: 304-320. |
33 | LIN M, JI R, WANG Y, et al. HRank: Filter pruning using high-rank feature map[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 1529-1538. |
34 | CHIN T W, DING R, ZHANG C, et al. Towards efficient model compression via learned global ranking[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 1518-1528. |
35 | YU R, LI A, CHEN C F, et al. Nisp: Pruning networks using neuron importance score propagation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 9194-9203. |
36 | LI Y, GU S, MAYER C, et al. Group sparsity: The hinge between filter pruning and decomposition for network compression[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 8018-8027. |
[1] | 黄赟, 张帆, 郭威, 陈立, 羊光. 一种基于数据标准差的卷积神经网络量化方法[J]. 电子学报, 2023, 51(3): 639-647. |
[2] | 吕杭, 蒋明峰, 李杨, 张鞠成, 王志康. 基于混合时频域特征的卷积神经网络心律失常分类方法的研究[J]. 电子学报, 2023, 51(3): 701-711. |
[3] | 许新征, 李杉. 基于特征膨胀卷积模块的轻量化技术研究[J]. 电子学报, 2023, 51(2): 355-364. |
[4] | 丁琪, 田萱, 孙国栋. 基于注意力增强的热点感知新闻推荐模型[J]. 电子学报, 2023, 51(1): 93-104. |
[5] | 张永梅, 孙捷. 基于动静态特征双输入神经网络的咳嗽声诊断COVID-19算法[J]. 电子学报, 2023, 51(1): 202-212. |
[6] | 袁海英, 成君鹏, 曾智勇, 武延瑞. Mobile_BLNet:基于Big-Little Net的轻量级卷积神经网络优化设计[J]. 电子学报, 2023, 51(1): 180-191. |
[7] | 吴靖, 叶晓晶, 黄峰, 陈丽琼, 王志锋, 刘文犀. 基于深度学习的单帧图像超分辨率重建综述[J]. 电子学报, 2022, 50(9): 2265-2294. |
[8] | 毛国君, 王者浩, 黄山, 王翔. 基于剪边策略的图残差卷积深层网络模型[J]. 电子学报, 2022, 50(9): 2205-2214. |
[9] | 袁海英, 曾智勇, 成君鹏. 面向灵活并行度的稀疏卷积神经网络加速器[J]. 电子学报, 2022, 50(8): 1811-1818. |
[10] | 张志文, 刘天歌, 聂鹏举. 基于实景数据增强和双路径融合网络的实时街景语义分割算法[J]. 电子学报, 2022, 50(7): 1609-1620. |
[11] | 王相海, 赵晓阳, 王鑫莹, 赵克云, 宋传鸣. 非抽取小波边缘学习深度残差网络的单幅图像超分辨率重建[J]. 电子学报, 2022, 50(7): 1753-1765. |
[12] | 张波, 陆云杰, 秦东明, 邹国建. 一种卷积自编码深度学习的空气污染多站点联合预测模型[J]. 电子学报, 2022, 50(6): 1410-1427. |
[13] | 丁毅, 沈薇, 李海生, 钟琼慧, 田明宇, 李洁. 面向CNN的区块链可信隐私服务计算模型[J]. 电子学报, 2022, 50(6): 1399-1409. |
[14] | 张云, 化青龙, 姜义成, 徐丹. 基于混合型复数域卷积神经网络的三维转动舰船目标识别[J]. 电子学报, 2022, 50(5): 1042-1049. |
[15] | 廖勇, 李玉杰. 一种轻量化低复杂度的FDD大规模MIMO系统CSI反馈方法[J]. 电子学报, 2022, 50(5): 1211-1217. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||