A Spatiotemporal Fusion Algorithm of Remote Sensing Images Based on Cross-Scale Similarity Prior

FANG Shuai; WAN Qi; CAO Yang

doi:10.12263/DZXB.20221147

您当前的位置：

首页 >

文章列表页 >

A Spatiotemporal Fusion Algorithm of Remote Sensing Images Based on Cross-Scale Similarity Prior

PAPERS | 更新时间：2025-12-11

- A Spatiotemporal Fusion Algorithm of Remote Sensing Images Based on Cross-Scale Similarity Prior
- ACTA ELECTRONICA SINICA Vol. 52, Issue 6, Pages: 2037-2052(2024)
- 作者机构：
  
  1.合肥工业大学计算机与信息学院,安徽合肥 230000
  2.工业安全与应急技术安徽省重点实验室,安徽合肥 230000
  3.大数据知识工程教育部重点实验室(合肥工业大学),安徽合肥 230000
  4.中国科学技术大学,安徽合肥 230000
- 作者简介：
- 基金信息：
  
  National Natural Science Foundation of China(61872327;61175033)
- DOI：10.12263/DZXB.20221147
  CLC： TP751.1;
- Received：11 October 2022，
  
  Revised：2023-03-10，
  
  Published：25 June 2024
- 稿件说明：
移动端阅览
方帅, 万旗, 曹洋. 基于跨尺度相似先验的遥感图像时空融合算法[J]. 电子学报, 2024, 52(06): 2037-2052.

FANG Shuai, WAN Qi, CAO Yang. A Spatiotemporal Fusion Algorithm of Remote Sensing Images Based on Cross-Scale Similarity Prior[J]. Acta Electronica Sinica, 2024, 52(06): 2037-2052.
方帅, 万旗, 曹洋. 基于跨尺度相似先验的遥感图像时空融合算法[J]. 电子学报, 2024, 52(06): 2037-2052. DOI：10.12263/DZXB.20221147

FANG Shuai, WAN Qi, CAO Yang. A Spatiotemporal Fusion Algorithm of Remote Sensing Images Based on Cross-Scale Similarity Prior[J]. Acta Electronica Sinica, 2024, 52(06): 2037-2052. DOI：10.12263/DZXB.20221147

摘要

遥感卫星图像在空间分辨率和时间分辨率之间权衡导致图像序列的时空矛盾.时空图像融合提供了一个生成高空间分辨率和高时间分辨率图像的解决方案，以满足各种地球观测应用.基于稀疏表示的时空融合算法通过联合训练字典和稀疏编码表示建立高低空间分辨率图像之间的关系，为物候变化、类型变化等各种情况提供了统一的融合框架.然而，多源遥感图像来自于不同的传感器，高低空间分辨率图像之间关系模型暗含有传感器映射关系，导致模型设备依赖.针对该问题，本文提出将多源遥感图像时空融合过程分解为传感器偏差校正和时空融合两个子问题，即设备依赖部分和设备无关部分.传感器偏差校正部分可以作为时空融合预处理模块，提高融合精度，并且使得后续的融合模型更加具有普适性.当高低空间分辨率图像空间分辨率差异较大时，“高低空间分辨率图像稀疏系数一致”的假设带来的融合误差非常突出.针对该问题，本文提出基于跨尺度相似先验的遥感图像时空融合算法，利用跨尺度相似块构建稀疏结构先验的正则项，优化稀疏表示的目标函数，并构建中间尺度图像，降低跨尺度相似块的二义性.本文分别使用3组典型场景的实验数据集与其他算法进行对比，实验结果表明，在BOREAS数据集上，与次优的指标相比，本文算法的结构相似度（Structural SIMilarity，SSIM）提高了4.2%，光谱角（Spectral Angle Mapper，SAM）提高了4.6%；在CIA数据集上，与次优的指标相比，本文算法的SSIM提高了2.7%，SAM提高了12.8%；在LGC数据集上，与次优的指标相比，本文算法的SSIM提高了7.1%，SAM提高了16.3%；证明本文算法在空间和光谱特性上表现出优秀的特性.

Abstract

The trade-off between spatial and temporal resolution of satellite images leads to spatial and temporal contradictions in image sequences. Spatiotemporal image fusion provides a solution to generate high spatial resolution and high temporal resolution images to satisfy various earth observation applications. The spatiotemporal fusion algorithm based on sparse representation establishes the relationship between high and low spatial resolution images by jointly training the dictionary and sparse coding representation

which provides a unified fusion framework for phenological change and type change. However

the multi-source remote sensing images come from different sensors

and the relationship model between high and low spatial resolution images implies the sensor mapping. This inevitably leads to that the model is device dependent. To solve the problem

we decompose the multi-source remote sensing spatiotemporal fusion process into two sub-problems

device dependent sensor bias correction and device independent spatiotemporal fusion. The sensor bias correction can be used as a preprocessing module to improve the universality and accuracy of subsequent fusion models. When there are large space scale gaps between high and low spatial resolution image

the assumption that “the sparse coefficients of high and low spatial resolution images are the same” will bring about very significant fusion errors. To solve the problem

we optimize the objective function of sparse representation using cross-scale similarity prior. Intermediate-scale images are constructed to reduce ambiguity of cross-scale similar patches and improve the accuracy of cross-scale similar patches. Experimental results in three typical scenarios demonstrate the generalization ability of our algorithm. The contrastive experiments show that on the BOREAS dataset

compared to suboptimal indicators

SSIM (Structural SIMilarity) is improved by 4.2%

SAM (Spectral Angle Mapper) is increased by 4.6%; On the CIA dataset

compared to suboptimal indicators

SSIM is increased by 2.7%

and SAM is increased by 12.8%; On the LGC dataset

compared to suboptimal indicators

SSIM is increased by 7.1%

and SAM is increased by 16.3%. Our algorithm is superior to other compared methods in spatial and spectral performance.

关键词

Keywords

references

VOGELMANN J E , HOWARD S M , YANG L , et al . Completion of the 1990s National Land Cover Data set for the conterminous United States from Landsat Thematic Mapper data and ancillary data sources [J ] . Photogrammetric Engineering and Remote Sensing , 2001 , 67 ( 6 ): 650 - 662 .

MASEK J G , HUANG C Q , WOLFE R , et al . North American forest disturbance mapped from a decadal Landsat record [J ] . Remote Sensing of Environment , 2008 , 112 ( 6 ): 2914 - 2926 .

WOODCOCK C E , OZDOGAN M . Trends in land cover mapping and monitoring [M ] // Land Change Science . Dordrecht : Springer , 2012 : 367 - 377 .

GAO F , MASEK J , SCHWALLER M , et al . On the blending of the Landsat and MODIS surface reflectance: Predicting daily Landsat surface reflectance [J ] . IEEE Transactions on Geoscience and Remote Sensing , 2006 , 44 ( 8 ): 2207 - 2218 .

HILKER T , WULDER M A , COOPS N C , et al . A new data fusion model for high spatial- and temporal-resolution mapping of forest disturbance based on Landsat and MODIS [J ] . Remote Sensing of Environment , 2009 , 113 ( 8 ): 1613 - 1627 .

ZHU X L , CHEN J , GAO F , et al . An enhanced spatial and temporal adaptive reflectance fusion model for complex heterogeneous regions [J ] . Remote Sensing of Environment , 2010 , 114 ( 11 ): 2610 - 2623 .

ZHUKOV B , OERTEL D , LANZL F , et al . Unmixing-based multisensor multiresolution image fusion [J ] . IEEE Transactions on Geoscience and Remote Sensing , 1999 , 37 ( 3 ): 1212 - 1226 .

ZURITA-MILLA R , CLEVERS J G P W , SCHAEPMAN M E . Unmixing-based Landsat TM and MERIS FR data fusion [J ] . IEEE Geoscience and Remote Sensing Letters , 2008 , 5 ( 3 ): 453 - 457 .

WU M Q , NIU Z , WANG C Y , et al . Use of MODIS and Landsat time series data to generate high-resolution temporal synthetic Landsat data using a spatial and temporal reflectance fusion model [J ] . Journal of Applied Remote Sensing , 2012 , 6 ( 1 ): 063507 .

WU M Q , HUANG W J , NIU Z , et al . Generating daily synthetic Landsat imagery by combining landsat and MODIS data [J ] . Sensors , 2015 , 15 ( 9 ): 24002 - 24025 .

ZHU X L , HELMER E H , GAO F , et al . A flexible spatiotemporal method for fusing satellite images with different resolutions [J ] . Remote Sensing of Environment , 2016 , 172 : 165 - 177 .

SONG H H , LIU Q S , WANG G J , et al . Spatiotemporal satellite image fusion using deep convolutional neural networks [J ] . IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 2018 , 11 ( 3 ): 821 - 829 .

ZHANG H Y , SONG Y Y , HAN C , et al . Remote sensing image spatiotemporal fusion using a generative adversarial network [J ] . IEEE Transactions on Geoscience and Remote Sensing , 2021 , 59 ( 5 ): 4273 - 4286 .

LI Y F , LI J , HE L , et al . A new sensor bias-driven spatio-temporal fusion model based on convolutional neural networks [J ] . Science China Information Sciences , 2020 , 63 ( 4 ): 140302 .

TAN Z Y , YUE P , DI L P , et al . Deriving high spatiotemporal remote sensing images using deep convolutional network [J ] . Remote Sensing , 2018 , 10 ( 7 ): 1066 .

TAN Z Y , DI L P , ZHANG M D , et al . An enhanced deep convolutional model for spatiotemporal image fusion‍ [J ] . Remote Sensing , 2019 , 11 ( 24 ): 2898 .

LI W S , ZHANG X Y , PENG Y D , et al . DMNet: A network architecture using dilated convolution and multiscale mechanisms for spatiotemporal fusion of remote sensing images [J ] . IEEE Sensors Journal , 2020 , 20 ( 20 ): 12190 - 12202 .

JIA D , CHENG C X , SONG C Q , et al . A hybrid deep learning-based spatiotemporal fusion method for combining satellite images with different resolutions [J ] . Remote Sensing , 2021 , 13 ( 4 ): 645 .

HUANG B , SONG H H . Spatiotemporal reflectance fusion via sparse representation [J ] . IEEE Transactions on Geoscience and Remote Sensing , 2012 , 50 ( 10 ): 3707 - 3716 .

SONG H H , HUANG B . Spatiotemporal satellite image fusion through one-pair image learning [J ] . IEEE Transactions on Geoscience and Remote Sensing , 2013 , 51 ( 4 ): 1883 - 1896 .

LIU X , DENG C W , ZHAO B J . Spatiotemporal reflectance fusion based on location regularized sparse representation [C ] // 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS) . Piscataway : IEEE , 2016 : 2562 - 2565 .

WU B , HUANG B , ZHANG L P . An error-bound-regularized sparse coding for spatiotemporal reflectance fusion [J ] . IEEE Transactions on Geoscience and Remote Sensing , 2015 , 53 ( 12 ): 6791 - 6803 .

WEI J B , WANG L Z , LIU P , et al . Spatiotemporal fusion of remote sensing images with structural sparsity and semi-coupled dictionary learning [J ] . Remote Sensing , 2017 , 9 ( 1 ): 21 .

WEI J B , WANG L Z , LIU P , et al . Spatiotemporal fusion of MODIS and Landsat-7 reflectance images via compressed sensing [J ] . IEEE Transactions on Geoscience and Remote Sensing , 2017 , 55 ( 12 ): 7126 - 7139 .

YANG W M , YUAN T R , WANG W , et al . Single-image super-resolution by subdictionary coding and kernel regression [J ] . IEEE Transactions on Systems, Man, and Cybernetics: Systems , 2017 , 47 ( 9 ): 2478 - 2488 .

ASEFPOUR VAKILIAN A , SARADJIAN M R . An object-based sparse representation model for spatiotemporal image fusion [J ] . Scientific Reports , 2022 , 12 : 5021 .

SHEN H F , WU P H , LIU Y L , et al . A spatial and temporal reflectance fusion model considering sensor observation differences [J ] . International Journal of Remote Sensing , 2013 , 34 ( 12 ): 4367 - 4383 .

TAN Z Y , GAO M L , LI X H , et al . A flexible reference-insensitive spatiotemporal fusion model for remote sensing images using conditional generative adversarial network [J ] . IEEE Transactions on Geoscience and Remote Sensing , 2022 , 60 : 5601413 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Cross-View Context-Aware Semi-Supervised Semantic Segmentation for High-Resolution Remote Sensing Images

Shared Super-Resolution Dual-Branch Network for Spatiotemporal Fusion of Remote Sensing Images

Sparse ISAR Imaging Combined with Nearest Neighbor Graph Model

Sparse Representation Theory and Application in Underwater Acoustic Signal Processing

A TCAD-DNN-Based Total-Ionizing-Dose Effect Model for FinFET Devices

Related Author

LÜ Liang

LAN Jie

LAN Meng

LU Xian-kai

ZHANG Le-fei

FANG Shuai

ZHANG Xiao-xi

ZHANG Jing

Related Institution

School of Computer Science, Wuhan University

Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology

School of Software, Shandong University

College of Electronic Information Engineering， Wuxi University

Key Laboratory of Radar Imaging and Microwave Photonics of the Ministry of Education， Nanjing University of Aeronautics and Astronautics

⁰