Edge Guided and Dynamically Deformable Transformer Network for Remote Sensing Images Change Detection

LEI Tao; ZHAI Yu-jie; XU Ye-tong; WANG Ying-bo; GONG Mao-guo

doi:10.12263/DZXB.20230583

您当前的位置：

首页 >

文章列表页 >

Edge Guided and Dynamically Deformable Transformer Network for Remote Sensing Images Change Detection

PAPERS | 更新时间：2026-04-10

- Edge Guided and Dynamically Deformable Transformer Network for Remote Sensing Images Change Detection
- ACTA ELECTRONICA SINICA Vol. 52, Issue 1, Pages: 107-117(2024)
- 作者机构：
  
  1.陕西省人工智能联合实验室（陕西科技大学），陕西西安 710021
  2.陕西科技大学电子信息与人工智能学院，陕西西安 710021
  3.西安电子科技大学协同智能系统教育部重点实验室，陕西西安 710071
- 作者简介：
- 基金信息：
  
  National Natural Science Foundation of China(62271296;62201334);Shaanxi Province Outstanding Youth Science Fund(2021JC-47);Shaanxi Province Key Research and Development Program(2022GY-436;2021ZDLGY08-07);Shaanxi Province Natural Science Basic Research Program(2022JQ-634;2022JQ-018);Shaanxi Province Innovation Capability Support Program(2020SS-03)
- DOI：10.12263/DZXB.20230583
  CLC： TP391
- Received：26 June 2023，
  
  Revised：2024-01-16，
  
  Published：25 January 2024
- 稿件说明：
移动端阅览
雷涛,翟钰杰,许叶彤,等.基于边缘引导和动态可变形Transformer的遥感图像变化检测[J].电子学报,2024,52(01):107-117.

LEI Tao, ZHAI Yu-jie, XU Ye-tong, et al.Edge Guided and Dynamically Deformable Transformer Network for Remote Sensing Images Change Detection[J].Acta Electronica Sinica, 2024, 52(01): 107-117.
雷涛,翟钰杰,许叶彤,等.基于边缘引导和动态可变形Transformer的遥感图像变化检测[J].电子学报,2024,52(01):107-117. DOI：10.12263/DZXB.20230583

LEI Tao, ZHAI Yu-jie, XU Ye-tong, et al.Edge Guided and Dynamically Deformable Transformer Network for Remote Sensing Images Change Detection[J].Acta Electronica Sinica, 2024, 52(01): 107-117. DOI：10.12263/DZXB.20230583

摘要

卷积神经网络（Convolutional Neural Network，CNN）和Transformer的混合架构能够有效建模图像的局部与全局特征，已成为遥感图像变化检测任务的主流网络.然而这类网络仍面临着一些挑战.CNN分支中的卷积和池化运算通常会抑制遥感图像中的高频信息，降低目标边界的精度；此外，Transformer分支对图像像素进行等同长程依赖关系建模，忽略了变化目标的形状及语义关联信息，导致网络对变化目标特征的表达不足.为解决上述问题，提出了基于边缘引导和动态可变形Transformer的遥感图像变化检测网络.在CNN分支中设计了边缘信息引导模块，利用高频信息增强目标区域的边缘信息，从而改善变化目标的轮廓精度.同时设计了一种新颖的动态可变形Transformer，能够自适应地匹配形状不同的变化目标，选择与变化相关的特征建模长程依赖关系，以提高网络的特征表达能力.实验结果表明，提出的方法在三个公开数据集LEVIR-CD、CDD和DSIFN-CD上显著提高了检测精度，在变化目标的边界精度和内部完整性方面都明显优于当前的主流网络.

Abstract

The hybrid architecture of convolutional neural network (CNN) and Transformer can effectively model local and global features of images

and has emerged as the predominant choice for remote sensing images change detection tasks. Nevertheless

these networks still confront challenges. The convolution and pooling operations employed by the CNN branch typically suppress the high-frequency information of remote sensing images

resulting in decreased precision of object boundaries in change detection results. Additionally

the Transformer branch equivalently models long-range dependencies for all pixels in remote sensing images

thereby disregarding shape information and semantic associations of objects

which limits the network’s feature representation ability on changed objects in remote sensing images. To address these challenges

a remote sensing images change detection network is proposed based on edge guidance and dynamic deformable Transformer. In the CNN branch

an edge information guidance module (EIG) is designed to enhance the edge information of changed objects by leveraging the high-frequency details of images. This enhancement improves the edge accuracy of the changed objects. Simultaneously

an innovative dynamically deformable Transformer (DDaT) is designed to adaptively match changed objects with different shapes

selecting features relevant to changes to model long-range dependency relationships and enhance the network’s feature expression capability. Experimental results show that the proposed method significantly improves the detection accuracy on three public datasets: LEVIR-CD

CDD and DSIFN-CD

and is significantly better than the current mainstream networks in terms of edge accuracy and internal integrity of changed objects.

关键词

Keywords

references

SEBASTIAN A , TUMA T , PAPANDREOU N , et al . Temporal correlation detection using computational phase-change memory [J]. Nature Communications , 2017 , 8 ( 1 ): 1115 .

DEMIR B , BOVOLO F , BRUZZONE L . Updating land-cover maps by classification of image time series: A novel change-detection-driven transfer learning approach [J]. IEEE Transactions on Geoscience and Remote Sensing , 2012 , 51 ( 1 ): 300 - 312 .

FAN Y D , WEN Q , WANG W , et al . Quantifying disaster physical damage using remote sensing data—A technical work flow and case study of the 2014 Ludian earthquake in China [J]. International Journal of Disaster Risk Science , 2017 , 8 ( 4 ): 471 - 488 .

SINGH A . Change detection in the tropical forest environment of northeastern India using Landsat [J]. Remote Sensing and Tropical Land Management , 1986 , 44 : 273 - 254 .

HOWARTH P J , WICKWARE G M . Procedures for change detection using Landsat digital data [J]. International Journal of Remote Sensing , 1981 , 2 ( 3 ): 277 - 291 .

DENG J S , WANG K , DENG Y H , et al . PCA-based land-use change detection and analysis using multitemporal and multisensor satellite data [J]. International Journal of Remote Sensing , 2008 , 29 ( 16 ): 4823 - 4838 .

MARCHESI S , BRUZZONE L . ICA and kernel ICA for change detection in multispectral remote sensing images [C]// 2009 IEEE International Geoscience and Remote Sensing Symposium . Piscataway : IEEE , 2009 : II-980-II-983.

NIELSEN A A , CONRADSEN K , SIMPSON J J . Multivariate alteration detection (MAD) and MAF postprocessing in multispectral, bitemporal image data: New approaches to change detection studies [J]. Remote Sensing of Environment , 1998 , 64 ( 1 ): 1 - 19 .

XU G B , LI H F , ZANG Y W , et al . Change detection based on IR-MAD model for GF-5 remote sensing imagery [J]. IOP Conference Series: Materials Science and Engineering , 2020 , 768 ( 7 ): 072073 .

RONNEBERGER O , FISCHER P , BROX T . U-net: Convolutional networks for biomedical image segmentation [C]// Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015: 18th International Conference . Cham : Springer International Publishing , 2015 : 234 - 241 .

CAYE DAUDT R , LE SAUX B , BOULCH A . Fully convolutional Siamese networks for change detection [C]// 2018 25th IEEE International Conference on Image Processing (ICIP) . Piscataway : IEEE , 2018 : 4063 - 4067 .

ZHANG C X , YUE P , TAPETE D , et al . A deeply supervised image fusion network for change detection in high resolution bi-temporal remote sensing images [J]. ISPRS Journal of Photogrammetry and Remote Sensing , 2020 , 166 : 183 - 200 .

HOU B , LIU Q J , WANG H , et al . From W-Net to CDGAN: Bitemporal change detection via deep learning techniques [J]. IEEE Transactions on Geoscience and Remote Sensing , 2019 , 58 ( 3 ): 1790 - 1802 .

ZHANG M , SHI W Z . A feature difference convolutional neural network-based change detection method [J]. IEEE Transactions on Geoscience and Remote Sensing , 2020 , 58 ( 10 ): 7232 - 7246 .

CHENG H Q , WU H Y , ZHENG J , et al . A hierarchical self-attention augmented Laplacian pyramid expanding network for change detection in high-resolution remote sensing images [J]. ISPRS Journal of Photogrammetry and Remote Sensing , 2021 , 182 : 52 - 66 .

FANG S , LI K Y , SHAO J Y , et al . SNUNet-CD: A densely connected Siamese network for change detection of VHR images [J]. IEEE Geoscience and Remote Sensing Letters , 2021 , 19 : 1 - 5 .

LEI T , WANG J , NING H L , et al . Difference enhancement and spatial—Spectral nonlocal network for change detection in VHR remote sensing images [J]. IEEE Transactions on Geoscience and Remote Sensing , 2021 , 60 : 1 - 13 .

LEI T , GENG X Z , NING H L , et al . Ultralightweight spatial—Spectral feature cooperation network for change detection in remote sensing images [J]. IEEE Transactions on Geoscience and Remote Sensing , 2023 , 61 : 1 - 14 .

VASWANI A , SHAZEER N , PARMAR N , et al . Attention is all you need [C]// Proceedings of the 31st International Conference on Neural Information Processing Systems . New York : ACM , 2017 : 6000 - 6010 .

CHEN H , QI Z P , SHI Z W . Remote sensing image change detection with Transformers [J]. IEEE Transactions on Geoscience and Remote Sensing , 2021 , 60 : 1 - 14 .

BANDARA W G C , PATEL V M . A Transformer-based Siamese network for change detection [C]// IGARSS 2022-2022 IEEE International Geoscience and Remote Sensing Symposium . Piscataway : IEEE , 2022 : 207 - 210 .

LIU M X , CHAI Z Q , DENG H J , et al . A CNN-Transformer network with multiscale context aggregation for fine-grained cropland change detection [J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 2022 , 15 : 4297 - 4306 .

ZHANG C , WANG L J , CHENG S L , et al . SwinSUNet: Pure Transformer network for remote sensing image change detection [J]. IEEE Transactions on Geoscience and Remote Sensing , 2022 , 60 : 1 - 13 .

KE Q T , ZHANG P . Hybrid-TransCD: A hybrid Transformer remote sensing image change detection network via token aggregation [J]. ISPRS International Journal of Geo-Information , 2022 , 11 ( 4 ): 263 .

LI Q Y , ZHONG R F , DU X , et al . TransUNetCD: A hybrid Transformer network for change detection in optical remote-sensing images [J]. IEEE Transactions on Geoscience and Remote Sensing , 2022 , 60 : 1 - 19 .

LU D , MAUSEL P , BRONDÍZIO E , et al . Change detection techniques [J]. International Journal of Remote Sensing , 2004 , 25 ( 12 ): 2365 - 2401 .

CHEN H , WU C , DU B , et al . Change detection in multi-temporal VHR images based on deep siamese multi-scale convolutional networks [EB/OL]. ( 2020-07-10 )[ 2023-06-26 ]. https://arxiv.org/abs/1906.11479 https://arxiv.org/abs/1906.11479 .

LEI T , ZHANG Y X , LV Z Y , et al . Landslide inventory mapping from bitemporal images using deep convolutional neural networks [J]. IEEE Geoscience and Remote Sensing Letters , 2019 , 16 ( 6 ): 982 - 986 .

PENG D F , ZHANG Y J , GUAN H Y . End-to-end change detection for high resolution satellite images using improved UNet++ [J]. Remote Sensing , 2019 , 11 ( 11 ): 1382 .

CHEN H , SHI Z W . A spatial-temporal attention-based method and a new dataset for remote sensing image change detection [J]. Remote Sensing , 2020 , 12 ( 10 ): 1662 .

LIU Z , LIN Y T , CAO Y , et al . Swin Transformer: Hierarchical vision Transformer using shifted windows [C]// 2021 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2021 : 10012 - 10022 .

ZHENG Z , ZHONG Y F , TIAN S , et al . ChangeMask: Deep multi-task encoder-Transformer-decoder architecture for semantic change detection [J]. ISPRS Journal of Photogrammetry and Remote Sensing , 2022 , 183 : 228 - 239 .

FENG Y C , XU H H , JIANG J W , et al . ICIF-Net: Intra-scale cross-interaction and inter-scale feature fusion network for bitemporal remote sensing images change detection [J]. IEEE Transactions on Geoscience and Remote Sensing , 2022 , 60 : 1 - 13 .

YANG L , ZHANG R Y , LI L D , et al . SimAM: A simple, parameter-free attention module for convolutional neural networks [C]// International Conference on Machine Learning . New York : PMLR , 2021 : 11863 - 11874 .

LUO T , MA Z , XU Z Q J , et al . Theory of the frequency principle for general deep neural networks [EB/OL]. ( 2019-07-02 )[ 2023-06-26 ]. https://arxiv.org/abs/1906.09235 https://arxiv.org/abs/1906.09235 .

AL-SUMAIDAEE S , ABDULLAH M , AL-NIMA R , et al . Multi-gradient features and elongated quinary pattern encoding for image-based facial expression recognition [J]. Pattern Recognition , 2017 , 71 : 249 - 263 .

DAI J F , QI H Z , XIONG Y W , et al . Deformable convolutional networks [C]// 2017 IEEE International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2017 : 764 - 773 .

ZHU X , SU W , LU L , et al . Deformable DETR: Deformable Transformers for end-to-end object detection [EB/OL].( 2021-03-18 )[ 2023-06-26 ]. https://arxiv.org/abs/2010.04159 https://arxiv.org/abs/2010.04159 .

CHEN Y P , DAI X Y , LIU M C , et al . Dynamic convolution: Attention over convolution kernels [C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2020 : 11030 - 11039 .

LEBEDEV M A , VIZILTER Y V , VYGOLOV O V , et al . Change detection in remote sensing images using conditional adversarial networks [J]. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences , 2018 , 42 : 565 - 571 .

NANDI A K . From multiple independent metrics to single performance measure based on objective function [J]. IEEE Access , 2023 , 11 : 3899 - 3913 .

BAHDANAU D , CHO K , BENGIO Y . Neural machine translation by jointly learning to align and translate [EB/OL]. ( 2016-05-19 )[ 2023-06-26 ]. https://arxiv.org/abs/1409.0473 https://arxiv.org/abs/1409.0473 .

LI X , SUN X , MENG Y , et al . Dice loss for data-imbalanced NLP tasks [EB/OL]. ( 2019-11-07 )[ 2023-06-26 ]. https://arxiv.org/abs/1911.02855 https://arxiv.org/abs/1911.02855 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

A Remote Sensing Image Text Retrieval Method Based on the Shared Prompt and Mamba Adapter

MAFNet: A Multi-Path Asymmetric Fusion Network for Metric-Based Change Detection in High-Resolution Remote Sensing Images

Invariant Features Based Ship Detection Model for Multi-source Remote Sensing Images

Efficient Object Detection in Remote Sensing Images: Advances in Deep Network Compression and Acceleration from Algorithmic and Model-Scale Perspectives

Related Author

LIU Bing

DU Wen-liang

XU Xiao-yu

ZHAO Jia-qi

ZHOU Yong

YAN Li

LI Xi

YANG Xi

Related Institution

Mine Digitization Engineering Research Center of the Ministry of Education

School of Computer Science and Technology/School of Artificial Intelligence, China University of Mining and Technology

School of Geodesy and Geomatics， Wuhan University

Chongqing Key Laboratory of Image Cognition, Chongqing University of Posts and Telecommunications

State Key Laboratory of Integrated Services Networks, Xidian University

⁰