Evaluation of the Effect of Neural Network Training Tricks on the Performance of High-Resolution Remote Sensing Image Scene Classification

ZHENG Hai-ying; WANG Feng; JIANG Wei; WANG Zhi-qiang; YAO Xi-wen

doi:10.12263/DZXB.20200961

您当前的位置：

首页 >

文章列表页 >

Evaluation of the Effect of Neural Network Training Tricks on the Performance of High-Resolution Remote Sensing Image Scene Classification

PAPERS | 更新时间：2025-12-08

- Evaluation of the Effect of Neural Network Training Tricks on the Performance of High-Resolution Remote Sensing Image Scene Classification
- ACTA ELECTRONICA SINICA Vol. 49, Issue 8, Pages: 1599-1614(2021)
- 作者机构：
  
  1.华北水利水电大学信息工程学院，河南郑州 450046
  2.西北工业大学自动化学院，陕西西安 710129
- 作者简介：
- 基金信息：
- DOI：10.12263/DZXB.20200961
  CLC： TP753;
- Received：31 August 2020，
  
  Revised：2020-12-14，
  
  Published：25 August 2021
- 稿件说明：
移动端阅览
郑海颖,王峰,姜维等.神经网络训练策略对高分辨率遥感图像场景分类性能影响的评估[J].电子学报,2021,49(08):1599-1614.

ZHENG Hai-ying,WANG Feng,JIANG Wei,et al.Evaluation of the Effect of Neural Network Training Tricks on the Performance of High-Resolution Remote Sensing Image Scene Classification[J].ACTA ELECTRONICA SINICA,2021,49(08):1599-1614.
郑海颖,王峰,姜维等.神经网络训练策略对高分辨率遥感图像场景分类性能影响的评估[J].电子学报,2021,49(08):1599-1614. DOI： 10.12263/DZXB.20200961.

ZHENG Hai-ying,WANG Feng,JIANG Wei,et al.Evaluation of the Effect of Neural Network Training Tricks on the Performance of High-Resolution Remote Sensing Image Scene Classification[J].ACTA ELECTRONICA SINICA,2021,49(08):1599-1614. DOI： 10.12263/DZXB.20200961.

摘要

机器学习方法在高分辨率遥感图像场景分类任务中已经得到大规模应用，但当前研究主要围绕数据特征和神经网络结构展开，极少提及神经网络训练策略对遥感图像分类性能的影响.因此，本文选取7种自然图像分类中常用的神经网络训练策略进行实验，根据其在3个规模较大的遥感图像数据集和4个广泛使用的神经网络模型上的实验表现，筛选出适用于遥感图像场景分类的神经网络训练策略.通过消融研究详细评估多个神经网络训练策略对遥感图像场景分类性能的影响，通过分析总体分类精度、混淆矩阵、Kappa系数得到有效的神经网络训练策略，并证明神经网络训练策略对遥感图像场景分类性能的有效性；根据叠加实验的结果分析，7种训练策略的组合可以在不同网络模型和数据集上表现出良好的适用性.

Abstract

Machine learning have been widely used in high-resolution remote sensing image scene classification task. However

the current research mainly focuses on data features and neural network structure

and the effect of neural network training tricks on remote sensing image classification performance is rarely mentioned. Therefore

this paper selects 7 neural network training tricks commonly used in natural image classification for experiments. According to their experimental performance in 3 large remote sensing image data sets and 4 widely used neural network models

neural network training tricks suitable for remote sensing image scene classification are selected. The effect of multiple neural network training tricks on the scene classification performance of remote sensing images was evaluated in detail through ablation experiment. An effective neural network training strategy was obtained by analyzing the overall accuracy

confusion matrix and Kappa coefficient

and the effectiveness of the neural network training strategy on the scene classification performance of remote sensing images was proved. According to the results of the stacking experiment

the combination of 7 training tricks can show good applicability in different network models and data sets.

关键词

Keywords

references

边小勇 , 费雄君 , 穆楠 . 基于尺度注意力网络的遥感图像场景分类 [J]. 计算机应用 , 2020 , 40 ( 3 ): 872 － 877 .

Bian X Y , Fei X J , Mu N . Remote sensing image scene classification based on scale-attention network [J]. Journal of Computer Applications , 2020 , 40 ( 3 ): 872 － 877 . (in Chinese)

许夙晖 , 慕晓冬 , 赵鹏 , 等 . 利用多尺度特征与深度网络对遥感影像进行场景分类 [J]. 测绘学报 , 2016 , 45 ( 7 ): 834 － 840 .

Xu S H , Mu X D , Zhao P , et al . Scene classification of remote sensing image based on multi-scale feature and deep neural network [J]. Acta Geodaetica et Cartographica Sinica , 2016 , 45 ( 7 ): 834 － 840 . (in Chinese)

钱晓亮 , 李佳 , 程塨 , 等 . 特征提取策略对高分辨率遥感图像场景分类性能影响的评估 [J]. 遥感学报 , 2018 , 22 ( 5 ): 758 － 776 .

Qian X L , Li J , Cheng G , et al . Evaluation of the effect of feature extraction strategy on the performance of high-resolution remote sensing image scene classification [J]. Journal of Remote Sensing , 2018 , 22 ( 5 ): 758 － 776 . (in Chinese)

Krizhevsky A , Sutskever I , Hinton G . Image net classification with deep convolutional neural networks [J]. Communications of the ACM , 2017 , 60 ( 6 ): 84 － 90 .

罗会兰 , 陈鸿坤 . 基于深度学习的目标检测研究综述 [J]. 电子学报 , 2020 , 48 ( 6 ): 1230 － 1239 .

Luo H L , Chen H K . Survey of object detection based on deep learning [J]. Acta Electronica Sinica , 2020 , 48 ( 6 ): 1230 － 1239 . (in Chinese)

刘颖 , 刘红燕 , 范九伦 , 等 . 基于深度学习的小目标检测研究与应用综述 [J]. 电子学报 , 2020 , 48 ( 3 ): 590 － 601 .

Liu Y , Liu H Y , Fan J L , et al . A survey of research and application of small object detection based on deep learning [J]. Acta Electronica Sinica , 2020 , 48 ( 3 ): 590 － 601 . (in Chinese)

Zhang W , Tang P , Zhao L . Remote sensing image scene classification using CNN-CapsNet [J]. Remote Sensing , 2019 , 11 ( 5 ): 494 .

Cheng G , Yang C , Yao X , et al . When deep learning meets metric learning:Remote sensing image scene classification via learning discriminative CNNs [J]. IEEE Transactions on Geoscience and Remote Sensing , 2018 , 56 ( 5 ): 2811 － 2821 .

Cheng G , Li Z , Yao X , et al . Remote sensing image scene classification using bag of convolutional features [J]. IEEE Geoscience and Remote Sensing Letters , 2017 , 14 ( 10 ): 1735 － 1739 .

陈雅琼 , 强振平 , 陈旭 , 等 . 利用微调卷积神经网络的土地利用场景分类 [J]. 遥感信息 , 2019 , 34 ( 3 ): 70 － 77 .

Chen Y Q , Qiang Z P , Chen X , et al . Classification of land use scenarios based on fine-tuning convolution neural network [J]. Remote Sensing Information , 2019 , 34 ( 3 ): 70 － 77 . (in Chinese)

柳潜 . 基于深度学习的遥感图像场景分类研究 [D]. 北京 : 北京邮电大学 , 2019 .

Liu Q . Research of remote sensing image scene classification based on deep learning [D]. Beijing, China : Beijing University of Posts and Telecommunications , 2019 . (in Chinese)

李金玲 . 基于预训练神经网络的遥感图像场景分类方法研究 [D]. 武汉 : 华中科技大学 , 2019 .

Li J L . Remote sensing image scene classification based on pre-trained CNN [D]. Wuhan, China : Huazhong University of Science and Technology , 2019 . (in Chinese)

黄鸿 , 徐科杰 , 石光耀 . 联合多尺度多特征的高分遥感图像场景分类 [J]. 电子学报 , 2020 , 48 ( 9 ): 1824 － 1833 .

Huang H , Xu K J , Shi G Y . Scene classification of high-resolution remote sensing image by multi-scale and multi-feature fusion [J]. Acta Electronica Sinica , 2020 , 48 ( 9 ): 1824 － 1833 . (in Chinese)

Goldblatt R , Stuhlmacher M , Tellman B , et al . Using Landsat and nighttime lights for supervised pixel-based image classification of urban land cover [J]. Remote Sensing of Environment , 2018 , 205 : 253 － 275 .

Du S , Zhang F , Zhang X . Semantic classification of urban buildings combining VHR image and GIS data: An improved random forest approach [J]. Isprs Journal of Photogrammetry and Remote Sensing , 2015 , 105 : 107 － 119 .

Yuan Y , Fang J , Lu X , et al . Remote sensing image scene classification using rearranged local features [J]. IEEE Transactions on Geoscience and Remote Sensing , 2019 , 57 ( 3 ): 1779 － 1792 .

Liu Q , Hang R , Song H , et al . Learning multiscale deep features for high-resolution satellite image scene classification [J]. IEEE Transactions on Geoscience and Remote Sensing , 2018 , 56 ( 1 ): 117 － 126 .

Zhu Q , Zhong Y , Zhang L , et al . Scene classification based on the fully sparse semantic topic model [J]. IEEE Transactions on Geoscience and Remote Sensing , 2017 , 55 ( 10 ): 5525 － 5538 .

Yan L , Zhu R , Mo N , et al . Cross-domain distance metric learning framework with limited target samples for scene classification of aerial images [J]. IEEE Transactions on Geoscience and Remote Sensing , 2019 , 57 ( 6 ): 3840 － 3857 .

Zhu Q , Zhong Y , Wu S , et al . Scene classification based on the sparse homogeneous–heterogeneous topic feature model [J]. IEEE Transactions on Geoscience and Remote Sensing , 2018 , 56 ( 5 ): 2689 － 2703 .

Zhu Q , Zhong Y , Zhang L , et al . Adaptive deep sparse semantic modeling framework for high spatial resolution image scene classification [J]. IEEE Transactions on Geoscience and Remote Sensing , 2018 , 56 ( 10 ): 6180 － 6195 .

Wang Y , Zhang L , Tong X , et al . LRAGE: Learning latent relationships with adaptive graph embedding for aerial scene classification [J]. IEEE Transactions on Geoscience and Remote Sensing , 2018 , 56 ( 2 ): 621 － 634 .

Zhong Z , Fan B , Ding K , et al . Efficient multiple feature fusion with hashing for hyperspectral imagery classification: A comparative study [J]. IEEE Transactions on Geoscience and Remote Sensing , 2016 , 54 ( 8 ): 4461 － 4478 .

Xu X , Li J , Huang X , et al . Multiple morphological component analysis based decomposition for remote sensing image classification [J]. IEEE Transactions on Geoscience and Remote Sensing , 2016 , 54 ( 5 ): 3083 － 3102 .

Pham M T , Lefèvre S , Aptoula E . Local feature-based attribute profiles for optical remote sensing image classification [J]. IEEE Transactions on Geoscience and Remote Sensing , 2017 , 56 ( 2 ): 1199 － 1212 .

Interdonato R , Ienco D , Gaetano R , et al . DuPLO: A DUal view point deep learning architecture for time series classification [J]. ISPRS Journal of Photogrammetry and Remote Sensing , 2019 , 149 : 91 － 104 .

Gavish , Yoni , O'connell , et al . Comparing the performance of flat and hierarchical Habitat/Land-Cover classification models in a NATURA 2000 site [J]. ISPRS Journal of Photogrammetry & Remote Sensing , 2018 , 136 : 1 － 12 .

Liu Y , Zhong Y , Qin Q . Scene classification based on multiscale convolutional neural network [J]. IEEE Transactions on Geoscience and Remote Sensing , 2018 , 56 ( 12 ): 7109 － 7121 .

Kemker R , Kanan C . Self-taught feature learning for hyperspectral image classification [J]. IEEE Transactions on Geoscience and Remote Sensing , 2017 , 55 ( 5 ): 2693 － 2705 .

Shu Y , Tang H , Li J , et al . Object-based unsupervised classification of VHR panchromatic satellite images by combining the HDP and IBP on multiple scenes [J]. IEEE Transactions on Geoscience and Remote Sensing , 2015 , 53 ( 11 ): 6148 － 6162 .

Zheng X , Yuan Y , Lu X . A deep scene representation for aerial scene classification [J]. IEEE Transactions on Geoscience and Remote Sensing , 2019 , 57 ( 7 ): 4799 － 4809 .

Romero A , Gatta C , Campsvalls G . Unsupervised deep feature extraction for remote sensing image classification [J]. IEEE Transactions on Geoscience and Remote Sensing , 2016 , 54 ( 3 ): 1349 － 1362 .

Zhong P , Gong Z , Li S , et al . Learning to diversify deep belief networks for hyperspectral image classification [J]. IEEE Transactions on Geoscience and Remote Sensing , 2017 , 55 ( 6 ): 3516 － 3530 .

Zhong Z , Li J , Luo Z , et al . Spectral-spatial residual network for hyperspectral image classification: A 3-D deep learning framework [J]. IEEE Transactions on Geoscience & Remote Sensing , 2017 , 56 ( 2 ): 847 － 858 .

Zhao W , Jiao L , Ma W , et al . Superpixel-based multiple local CNN for panchromatic and multispectral image classification [J]. IEEE Transactions on Geoscience and Remote Sensing , 2017 , 55 ( 7 ): 4141 － 4156 .

Wang Q , Liu S , Chanussot J , et al . Scene classification with recurrent attention of VHR remote sensing images [J]. IEEE Transactions on Geoscience and Remote Sensing , 2018 , 57 ( 2 ): 1155 － 1167 .

Gong Z , Zhong P , Yu Y , et al . Diversity-promoting deep structural metric learning for remote sensing scene classification [J]. IEEE Transactions on Geoscience and Remote Sensing , 2018 , 56 ( 1 ): 371 － 390 .

Cheng G , Han J , Zhou P , et al . Multi-class geospatial object detection and geographic image classification based on collection of part detectors [J]. ISPRS Journal of Photogrammetry and Remote Sensing , 2014 , 98 ( 98 ): 119 － 132 .

Wang W , Du L , Gao Y , et al . A discriminative learned CNN embedding for remote sensing image scene classification [J]. arXiv: Computer Vision and Pattern Recognition , 2019 : arXiv: 1911.12517 .

刘冰 , 李瑞麟 , 封举富 . 深度度量学习综述 [J]. 智能系统学报 , 2019 , 14 ( 6 ): 1064 － 1072 .

Liu B , Li R L , Feng J F . A brief introduction to deep metric learning [J]. CAAI Transactions on Intelligent Systems , 2019 , 14 ( 6 ): 1064 － 1072 . (in Chinese)

刘瑄 , 池明旻 . 基于多粒度特征蒸馏的遥感图像场景分类研究 [J/OL]. https://doi.org/10.19678/j.issn.1000-3428.0056798 https://doi.org/10.19678/j.issn.1000-3428.0056798 . [ 2020-12-07 ].

Liu X , Chi M M . Remote sensing image scene classification based on multi-granular feature distillation deep convolution neural networks [J/OL]. https://doi.org/10.19678/j.issn.1000-3428.0056798 https://doi.org/10.19678/j.issn.1000-3428.0056798 . [ 2020-12-07 ]. (in Chinese)

祁昆仑 . 基于视觉特征的高分辨率光学遥感影像多任务分类研究 [J]. 测绘学报 , 2017 , 46 ( 6 ): 802 .

Qi K L . Multi-task classification of high resolution optic remote sensing images based on visual features [J]. Acta Geodaetica et Cartographica Sinica , 2017 , 46 ( 6 ): 802 . (in Chinese)

张馨月 . 基于DCNN的高分辨率遥感图像场景分类 [D]. 长春 : 吉林大学 , 2019 .

Zhang X Y . Scene classification of high-resolution remotely sensed images based on DCNN [D]. Changchun, China : Jilin University , 2019 . (in Chinese)

崔先亮 , 陈立福 , 邢学敏 , 等 . 基于频带特征融合的GL-CNN遥感图像场景分类 [J]. 遥感技术与应用 , 2019 , 34 ( 4 ): 712 － 719 .

Cui X L , Chen L F , Xing X M , et al . Remote sensing image scene classification based on frequency band feature fusion and GL-CNN [J]. Remote Sensing Technology and Application , 2019 , 34 ( 4 ): 712 － 719 . (in Chinese)

Yang Y , Newsam S . Bag-of-visual-words and spatial extensions for land-use classification [A]. Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems [C]. New York, USA : ACM , 2010 . 270 － 279 .

Xia G S , Hu J , Hu F , et al . AID: A benchmark data set for performance evaluation of aerial scene classification [J]. IEEE Transactions on Geoscience and Remote Sensing , 2017 , 55 ( 7 ): 3965 － 3981 .

Cheng G , Han J , Lu X . Remote sensing image scene classification: Benchmark and state of the art [J]. Proceedings of the IEEE , 2017 , 105 ( 10 ): 1865 － 1883 .

He T , Zhang Z , Zhang H , et al . Bag of tricks for image classification with convolutional neural networks [A]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition [C]. Long Beach, USA : IEEE , 2019 . 558 － 567 .

Glorot X , Bengio Y . Understanding the difficulty of training deep feedforward neural networks [A]. Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics [C]. Sardinia, Italy : PMLR , 2010 . 249 － 256 .

Goyal P , Dollár P , Girshick R , et al . Accurate, large minibatch sgd: Training imagenet in 1 hour [J]. arXiv Preprint , 2017 , arXiv: 1706.02677 .

Jia X , Song S , He W , et al . Highly scalable deep learning training system with mixed-precision: Training imagenet in four minutes [J]. arXiv Preprint , 2018 , arXiv: 1807.11205 .

Szegedy C , Vanhoucke V , Ioffe S , et al . Rethinking the inception architecture for computer vision [A]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition [C]. Las Vegas, USA : IEEE , 2016 . 2818 － 2826 .

Zhong Z , Zheng L , Kang G , et al . Random Erasing Data Augmentation [J]. Computer Vision and Pattern Recognition , 2020 , arXiv: 1708.04896 .

DeVries T , Taylor G W . Improved regularization of convolutional neural networks with cutout [J]. arXiv Preprint , 2017 , arXiv: 1708.04552 .

Loshchilov I , Hutter F . Sgdr: Stochastic gradient descent with warm restarts [J]. arXiv Preprint , 2016 , arXiv: 1608.03983 .

Xiong R , Yang Y , He D , et al . On layer normalization in the transformer architecture [J]. arXiv Preprint , 2020 , arXiv: 2002.04745 .

Gotmare A , Keskar N S , Xiong C , et al . A closer look at deep learning heuristics: Learning rate restarts, warmup and distillation [J]. arXiv Preprint , 2018 , arXiv: 1810.13243 .

You Y , Gitman I , Ginsburg B . Large batch training of convolutional networks [J]. arXiv Preprint , 2017 , arXiv: 1708.03888 .

He K , Zhang X , Ren S , et al . Deep residual learning for image recognition [A]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition [C]. as Vegas, USA : IEEE , 2016 . 770 － 778 .

Müller R , Kornblith S , Hinton G E . When does label smoothing help? [A]. Advances in Neural Information Processing Systems [C]. Vancouver, Canada : DBLP , 2019 . 4694 － 4703 .

Liang D , Yang F , Zhang T , et al . Understanding mixup training methods [J]. IEEE Access , 2018 , 6 : 58774 － 58783 .

Zhang H , Cisse M , Dauphin Y N , et al . Mixup: Beyond empirical risk minimization [J]. arXiv Preprint , 2017 , arXiv: 1710.09412 .

Micikevicius P , Narang S , Alben J , et al . Mixed precision training [J]. arXiv Preprint , 2017 , arXiv: 1710.03740 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

A Robust and Accurate Object Tracking Algorithm Based on Convolutional Neural Network

Image Classification Network of Background Perception Mechanism

A Detection Method for Android Repackaged Malware Based on Sensitive Component Function Call Graph

Neural Network Based Image Style Transfer: A Survey

Related Author

LI Kang

LI Ya-min

HU Xue-min

SHAO Fang

Wang Jicheng

YUAN Heng

RAN Chao

ZHANG Sheng-chong

Related Institution

School of Computer Science and Information Engineering, Hubei University

Dept. of Computer Science and Engineering, Tongji University

200092

School of Software, Liaoning Technical University

Key Laboratory of Optoelectronic Information Control and Security Technology

⁰