基于跨尺度相似先验的遥感图像时空融合算法

方帅; 万旗; 曹洋

doi:10.12263/DZXB.20221147

您当前的位置：

首页 >

文章列表页 >

基于跨尺度相似先验的遥感图像时空融合算法

学术论文 | 更新时间：2025-12-11

- 基于跨尺度相似先验的遥感图像时空融合算法
- A Spatiotemporal Fusion Algorithm of Remote Sensing Images Based on Cross-Scale Similarity Prior
- 电子学报 2024年52卷第6期页码：2037-2052
- 作者机构：
  
  1.合肥工业大学计算机与信息学院,安徽合肥 230000
  2.工业安全与应急技术安徽省重点实验室,安徽合肥 230000
  3.大数据知识工程教育部重点实验室(合肥工业大学),安徽合肥 230000
  4.中国科学技术大学,安徽合肥 230000
- 作者简介：
  
  [ "方帅女，合肥工业大学计算机与信息学院教授，博士生导师.研究方向为计算机视觉、图像复原等.主持国家自然科学基金青年基金1项，面上基金1项，博士专项基金1项，学院杰出青年科学基金人才培育计划1项，企业合作项目多项，作为主要成员参加了国防973专项1项，国家自然科学基金3项，博士学科点专项科研基金1项，安徽省自然科学基金1项，近年来共发表论文30余篇. E-mail: fangshuai@hfut.edu.cn" ]
  万旗男，合肥工业大学计算机与信息学院硕士.研究方向为计算机视觉. E-mail: 2020171223@mail.hfut.edu.cn
  [ "曹洋男，中国科学技术大学信息科学技术学院自动化系副教授，博士生导师.研究方向为底层视觉处理及表征学习等.近3年来，在CVPR、ICCV、NeuRIPS、ICLR等人工智能领域顶尖学术会议以及IJCV、TNNLS、TIP等国际权威期刊上发表文章30余篇，承担国家重点研发计划、国家自然科学基金、安徽省重大专项等科研项目10余项，作为主要完成人获安徽省技术发明奖一等奖、安徽省自然科学奖二等奖和中国自动化学会科学技术一等奖. E-mail: forrest@ustc.edu.cn" ]
- 基金信息：
  
  国家自然科学基金(61872327;61175033)
- DOI：10.12263/DZXB.20221147
  中图分类号： TP751.1;
- 收稿：2022-10-11，
  
  修回：2023-03-10，
  
  纸质出版：2024-06-25
- 稿件说明：
移动端阅览
方帅, 万旗, 曹洋. 基于跨尺度相似先验的遥感图像时空融合算法[J]. 电子学报, 2024, 52(06): 2037-2052.

FANG Shuai, WAN Qi, CAO Yang. A Spatiotemporal Fusion Algorithm of Remote Sensing Images Based on Cross-Scale Similarity Prior[J]. Acta Electronica Sinica, 2024, 52(06): 2037-2052.
方帅, 万旗, 曹洋. 基于跨尺度相似先验的遥感图像时空融合算法[J]. 电子学报, 2024, 52(06): 2037-2052. DOI：10.12263/DZXB.20221147

FANG Shuai, WAN Qi, CAO Yang. A Spatiotemporal Fusion Algorithm of Remote Sensing Images Based on Cross-Scale Similarity Prior[J]. Acta Electronica Sinica, 2024, 52(06): 2037-2052. DOI：10.12263/DZXB.20221147

摘要

遥感卫星图像在空间分辨率和时间分辨率之间权衡导致图像序列的时空矛盾.时空图像融合提供了一个生成高空间分辨率和高时间分辨率图像的解决方案，以满足各种地球观测应用.基于稀疏表示的时空融合算法通过联合训练字典和稀疏编码表示建立高低空间分辨率图像之间的关系，为物候变化、类型变化等各种情况提供了统一的融合框架.然而，多源遥感图像来自于不同的传感器，高低空间分辨率图像之间关系模型暗含有传感器映射关系，导致模型设备依赖.针对该问题，本文提出将多源遥感图像时空融合过程分解为传感器偏差校正和时空融合两个子问题，即设备依赖部分和设备无关部分.传感器偏差校正部分可以作为时空融合预处理模块，提高融合精度，并且使得后续的融合模型更加具有普适性.当高低空间分辨率图像空间分辨率差异较大时，“高低空间分辨率图像稀疏系数一致”的假设带来的融合误差非常突出.针对该问题，本文提出基于跨尺度相似先验的遥感图像时空融合算法，利用跨尺度相似块构建稀疏结构先验的正则项，优化稀疏表示的目标函数，并构建中间尺度图像，降低跨尺度相似块的二义性.本文分别使用3组典型场景的实验数据集与其他算法进行对比，实验结果表明，在BOREAS数据集上，与次优的指标相比，本文算法的结构相似度（Structural SIMilarity，SSIM）提高了4.2%，光谱角（Spectral Angle Mapper，SAM）提高了4.6%；在CIA数据集上，与次优的指标相比，本文算法的SSIM提高了2.7%，SAM提高了12.8%；在LGC数据集上，与次优的指标相比，本文算法的SSIM提高了7.1%，SAM提高了16.3%；证明本文算法在空间和光谱特性上表现出优秀的特性.

Abstract

The trade-off between spatial and temporal resolution of satellite images leads to spatial and temporal contradictions in image sequences. Spatiotemporal image fusion provides a solution to generate high spatial resolution and high temporal resolution images to satisfy various earth observation applications. The spatiotemporal fusion algorithm based on sparse representation establishes the relationship between high and low spatial resolution images by jointly training the dictionary and sparse coding representation

which provides a unified fusion framework for phenological change and type change. However

the multi-source remote sensing images come from different sensors

and the relationship model between high and low spatial resolution images implies the sensor mapping. This inevitably leads to that the model is device dependent. To solve the problem

we decompose the multi-source remote sensing spatiotemporal fusion process into two sub-problems

device dependent sensor bias correction and device independent spatiotemporal fusion. The sensor bias correction can be used as a preprocessing module to improve the universality and accuracy of subsequent fusion models. When there are large space scale gaps between high and low spatial resolution image

the assumption that “the sparse coefficients of high and low spatial resolution images are the same” will bring about very significant fusion errors. To solve the problem

we optimize the objective function of sparse representation using cross-scale similarity prior. Intermediate-scale images are constructed to reduce ambiguity of cross-scale similar patches and improve the accuracy of cross-scale similar patches. Experimental results in three typical scenarios demonstrate the generalization ability of our algorithm. The contrastive experiments show that on the BOREAS dataset

compared to suboptimal indicators

SSIM (Structural SIMilarity) is improved by 4.2%

SAM (Spectral Angle Mapper) is increased by 4.6%; On the CIA dataset

compared to suboptimal indicators

SSIM is increased by 2.7%

and SAM is increased by 12.8%; On the LGC dataset

compared to suboptimal indicators

SSIM is increased by 7.1%

and SAM is increased by 16.3%. Our algorithm is superior to other compared methods in spatial and spectral performance.

关键词

Keywords

references

VOGELMANN J E , HOWARD S M , YANG L , et al . Completion of the 1990s National Land Cover Data set for the conterminous United States from Landsat Thematic Mapper data and ancillary data sources [J ] . Photogrammetric Engineering and Remote Sensing , 2001 , 67 ( 6 ): 650 - 662 .

MASEK J G , HUANG C Q , WOLFE R , et al . North American forest disturbance mapped from a decadal Landsat record [J ] . Remote Sensing of Environment , 2008 , 112 ( 6 ): 2914 - 2926 .

WOODCOCK C E , OZDOGAN M . Trends in land cover mapping and monitoring [M ] // Land Change Science . Dordrecht : Springer , 2012 : 367 - 377 .

GAO F , MASEK J , SCHWALLER M , et al . On the blending of the Landsat and MODIS surface reflectance: Predicting daily Landsat surface reflectance [J ] . IEEE Transactions on Geoscience and Remote Sensing , 2006 , 44 ( 8 ): 2207 - 2218 .

HILKER T , WULDER M A , COOPS N C , et al . A new data fusion model for high spatial- and temporal-resolution mapping of forest disturbance based on Landsat and MODIS [J ] . Remote Sensing of Environment , 2009 , 113 ( 8 ): 1613 - 1627 .

ZHU X L , CHEN J , GAO F , et al . An enhanced spatial and temporal adaptive reflectance fusion model for complex heterogeneous regions [J ] . Remote Sensing of Environment , 2010 , 114 ( 11 ): 2610 - 2623 .

ZHUKOV B , OERTEL D , LANZL F , et al . Unmixing-based multisensor multiresolution image fusion [J ] . IEEE Transactions on Geoscience and Remote Sensing , 1999 , 37 ( 3 ): 1212 - 1226 .

ZURITA-MILLA R , CLEVERS J G P W , SCHAEPMAN M E . Unmixing-based Landsat TM and MERIS FR data fusion [J ] . IEEE Geoscience and Remote Sensing Letters , 2008 , 5 ( 3 ): 453 - 457 .

WU M Q , NIU Z , WANG C Y , et al . Use of MODIS and Landsat time series data to generate high-resolution temporal synthetic Landsat data using a spatial and temporal reflectance fusion model [J ] . Journal of Applied Remote Sensing , 2012 , 6 ( 1 ): 063507 .

WU M Q , HUANG W J , NIU Z , et al . Generating daily synthetic Landsat imagery by combining landsat and MODIS data [J ] . Sensors , 2015 , 15 ( 9 ): 24002 - 24025 .

ZHU X L , HELMER E H , GAO F , et al . A flexible spatiotemporal method for fusing satellite images with different resolutions [J ] . Remote Sensing of Environment , 2016 , 172 : 165 - 177 .

SONG H H , LIU Q S , WANG G J , et al . Spatiotemporal satellite image fusion using deep convolutional neural networks [J ] . IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 2018 , 11 ( 3 ): 821 - 829 .

ZHANG H Y , SONG Y Y , HAN C , et al . Remote sensing image spatiotemporal fusion using a generative adversarial network [J ] . IEEE Transactions on Geoscience and Remote Sensing , 2021 , 59 ( 5 ): 4273 - 4286 .

LI Y F , LI J , HE L , et al . A new sensor bias-driven spatio-temporal fusion model based on convolutional neural networks [J ] . Science China Information Sciences , 2020 , 63 ( 4 ): 140302 .

TAN Z Y , YUE P , DI L P , et al . Deriving high spatiotemporal remote sensing images using deep convolutional network [J ] . Remote Sensing , 2018 , 10 ( 7 ): 1066 .

TAN Z Y , DI L P , ZHANG M D , et al . An enhanced deep convolutional model for spatiotemporal image fusion‍ [J ] . Remote Sensing , 2019 , 11 ( 24 ): 2898 .

LI W S , ZHANG X Y , PENG Y D , et al . DMNet: A network architecture using dilated convolution and multiscale mechanisms for spatiotemporal fusion of remote sensing images [J ] . IEEE Sensors Journal , 2020 , 20 ( 20 ): 12190 - 12202 .

JIA D , CHENG C X , SONG C Q , et al . A hybrid deep learning-based spatiotemporal fusion method for combining satellite images with different resolutions [J ] . Remote Sensing , 2021 , 13 ( 4 ): 645 .

HUANG B , SONG H H . Spatiotemporal reflectance fusion via sparse representation [J ] . IEEE Transactions on Geoscience and Remote Sensing , 2012 , 50 ( 10 ): 3707 - 3716 .

SONG H H , HUANG B . Spatiotemporal satellite image fusion through one-pair image learning [J ] . IEEE Transactions on Geoscience and Remote Sensing , 2013 , 51 ( 4 ): 1883 - 1896 .

LIU X , DENG C W , ZHAO B J . Spatiotemporal reflectance fusion based on location regularized sparse representation [C ] // 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS) . Piscataway : IEEE , 2016 : 2562 - 2565 .

WU B , HUANG B , ZHANG L P . An error-bound-regularized sparse coding for spatiotemporal reflectance fusion [J ] . IEEE Transactions on Geoscience and Remote Sensing , 2015 , 53 ( 12 ): 6791 - 6803 .

WEI J B , WANG L Z , LIU P , et al . Spatiotemporal fusion of remote sensing images with structural sparsity and semi-coupled dictionary learning [J ] . Remote Sensing , 2017 , 9 ( 1 ): 21 .

WEI J B , WANG L Z , LIU P , et al . Spatiotemporal fusion of MODIS and Landsat-7 reflectance images via compressed sensing [J ] . IEEE Transactions on Geoscience and Remote Sensing , 2017 , 55 ( 12 ): 7126 - 7139 .

YANG W M , YUAN T R , WANG W , et al . Single-image super-resolution by subdictionary coding and kernel regression [J ] . IEEE Transactions on Systems, Man, and Cybernetics: Systems , 2017 , 47 ( 9 ): 2478 - 2488 .

ASEFPOUR VAKILIAN A , SARADJIAN M R . An object-based sparse representation model for spatiotemporal image fusion [J ] . Scientific Reports , 2022 , 12 : 5021 .

SHEN H F , WU P H , LIU Y L , et al . A spatial and temporal reflectance fusion model considering sensor observation differences [J ] . International Journal of Remote Sensing , 2013 , 34 ( 12 ): 4367 - 4383 .

TAN Z Y , GAO M L , LI X H , et al . A flexible reference-insensitive spatiotemporal fusion model for remote sensing images using conditional generative adversarial network [J ] . IEEE Transactions on Geoscience and Remote Sensing , 2022 , 60 : 5601413 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于跨视图上下文感知的高分辨率遥感图像半监督语义分割方法

共享超分的双分支遥感图像时空融合网络

结合最近邻图模型的稀疏ISAR成像方法

水声信号处理中的稀疏表示理论及应用

特征掩码与对比学习融合多维度去过度相关的序列推荐