Estimating Illumination Condition and Spectral Reflectance Based on the Color Variation Between Shadowed Region and Its Surrounding Regions

WANG Xiao; YAO Si-yuan; DAI Peng-wen; WANG Rui; CAO Xiao-chun

doi:10.12263/DZXB.20211291

您当前的位置：

首页 >

文章列表页 >

Estimating Illumination Condition and Spectral Reflectance Based on the Color Variation Between Shadowed Region and Its Surrounding Regions

PAPERS | 更新时间：2025-12-08

- Estimating Illumination Condition and Spectral Reflectance Based on the Color Variation Between Shadowed Region and Its Surrounding Regions
- ACTA ELECTRONICA SINICA Vol. 51, Issue 8, Pages: 2098-2109(2023)
- 作者机构：
  
  1.中国科学院信息工程研究所信息安全国家重点实验室,北京 100093
  2.之江实验室,浙江杭州 311100
  3.中国科学院大学网络空间安全学院,北京 100049
- 作者简介：
- 基金信息：
  
  National Natural Science Foundation of China Under Grants(U20B2066);The National Key R&D Program of China(2020AAA0109304);Open Research Projects of Zhejiang Lab(2021KB0AB01)
- DOI：10.12263/DZXB.20211291
  CLC： TP183;
- Received：21 September 2021，
  
  Revised：2021-11-17，
  
  Published：25 August 2023
- 稿件说明：
移动端阅览
王潇,尧思远,代朋纹等.基于阴影和周边区域色差的光照条件和光谱反射率计算方法[J].电子学报,2023,51(08):2098-2109.

WANG Xiao,YAO Si-yuan,DAI Peng-wen,et al.Estimating Illumination Condition and Spectral Reflectance Based on the Color Variation Between Shadowed Region and Its Surrounding Regions[J].ACTA ELECTRONICA SINICA,2023,51(08):2098-2109.
王潇,尧思远,代朋纹等.基于阴影和周边区域色差的光照条件和光谱反射率计算方法[J].电子学报,2023,51(08):2098-2109. DOI： 10.12263/DZXB.20211291.

WANG Xiao,YAO Si-yuan,DAI Peng-wen,et al.Estimating Illumination Condition and Spectral Reflectance Based on the Color Variation Between Shadowed Region and Its Surrounding Regions[J].ACTA ELECTRONICA SINICA,2023,51(08):2098-2109. DOI： 10.12263/DZXB.20211291.

摘要

光照条件和光谱反射率对增强现实和场景渲染任务非常有用，但是通过仪器采集比较困难，因此根据普通图像计算光照条件和光谱反射率是重要的计算机视觉任务.本文设计了一种通过分析场景中的阴影区域和周边区域颜色之间的色差，无需训练数据即可计算场景的光照条件和光谱反射率的算法.在自然场景中，阴影区域由于遮挡只能接收和反射天空光，而其周边的非阴影区域则同时受到太阳光和天空光的影响.当阴影和非阴影区域有着相同的光谱反射率时，两块区域颜色之间的色差反映了被遮挡光源的信息.两块区域之间的颜色差异由光照条件造成，又以光谱反射率为联系.因此，可以利用这两种颜色之间的关系计算光照条件和光谱反射率.本文使用先验知识和模型公式化表达某一区域在光照下呈现在图像中的颜色，然后基于两种颜色以及两者之间的色差设计了一种优化算法，通过计算最优解来计算场景的光照条件以及光谱反射率.通过在多个数据集上的一系列对比实验，以及一个使用光照条件和光谱反射率来去除阴影区域的应用实验，证明了该算法的准确性.

Abstract

The illumination condition and the spectral reflectance are very useful for scene rendering and augmented reality but difficult to collect. Thus

it is an important task to get them from ordinary image. In this paper

we proposed a method that estimates illumination condition and spectral reflectance of natural scene without training process by analyzing the color variation between shadowed region and its surrounding regions. In natural scenes

the shadowed region was only lighted by skylight while its surrounding unshadowed regions were lighted by both sunlight and skylight. The color variation between them was the effect of shielded light source when they had the same spectral reflectance. The difference between these two colors was caused by the illumination condition of captured scene

while they were connected by their spectral reflectance. These colors provided the cue to calculate the illumination condition and the spectral reflectance. We formulized the whole process of image generation using prior knowledge and model. Then

we designed an optimization function based on the relationship between these colors to estimate the illumination condition and the spectral reflectance. The results of comparison experiment and the application experiment on shadow removal showed that the proposed method performed favorably against the state-of-the-art methods.

关键词

Keywords

references

HOLD-GEOFFROY Y , ATHAWALE A , LALONDE J F . Deep sky modeling for single image outdoor lighting estimation [C ] // 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2020 : 6920 - 6928 .

TARABALKA Y , CHANUSSOT J , BENEDIKTSSON J A . Segmentation and classification of hyperspectral images using watershed transformation [J ] . Pattern Recognition , 2010 , 43 ( 7 ): 2367 - 2379 .

XU X , LI J , HUANG X , et al . Multiple morphological component analysis based decomposition for remote sensing image classification [J ] . IEEE Transactions on Geoscience and Remote Sensing , 2016 , 54 ( 5 ): 3083 - 3102 .

孙乐 , 吴泽彬 , 冯灿 , 等 . 一种新的两分类器融合的空谱联合高光谱分类方法 [J ] . 电子学报 , 2015 , 43 ( 11 ): 2210 - 2217 .

SUN L , WU Z B , FENG C , et al . A novel two-classifier fusion method for spectral-spatial hyperspectral classification [J ] . Acta Electronica Sinica , 2015 , 43 ( 11 ): 2210 - 2217 . (in Chinese)

陈允杰 , 马辰阳 , 孙乐 , 等 . 基于边缘修正的高光谱图像超像素空谱核分类方法 [J ] . 电子学报 , 2019 , 47 ( 1 ): 73 - 81 .

CHEN Y J , MA C Y , SUN L , et al . Edge-modified superpixel based spectral-spatial kernel method for hyperspectral image classification [J ] . Acta Electronica Sinica , 2019 , 47 ( 1 ): 73 - 81 . (in Chinese)

VAN NGUYEN H , BANERJEE A , CHELLAPPA R . Tracking via object reflectance using a hyperspectral video camera [C ] // 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops . Piscataway : IEEE , 2010 : 44 - 51 .

AKHTAR N , MIAN A . Nonparametric coupled Bayesian dictionary and classifier learning for hyperspectral classification [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2018 , 29 ( 9 ): 4038 - 4050 .

BIOUCAS-DIAS J M , PLAZA A , CAMPS-VALLS G , et al . Hyperspectral remote sensing data analysis and future challenges [J ] . IEEE Geoscience and Remote Sensing Magazine , 2013 , 1 ( 2 ): 6 - 36 .

AKHTAR N , SHAFAIT F , MIAN A . Bayesian sparse representation for hyperspectral image super resolution [C ] // 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2015 : 3631 - 3640 .

PANAGOPOULOS A , WANG C H , SAMARAS D , et al . Simultaneous cast shadows, illumination and geometry inference using hypergraphs [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2013 , 35 ( 2 ): 437 - 449 .

PANAGOPOULOS A , VICENTE T F Y , SAMARAS D . Illumination estimation from shadow borders [C ] // 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops) . Piscataway : IEEE , 2012 : 798 - 805 .

LALONDE J F , EFROS A A , NARASIMHAN S G . Estimating the natural illumination conditions from a single outdoor image [J ] . International Journal of Computer Vision , 2012 , 98 ( 2 ): 123 - 145 .

HOLD-GEOFFROY Y , SUNKAVALLI K , HADAP S , et al . Deep outdoor illumination estimation [C ] // 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2017 : 2373 - 2382 .

GEORGOULIS S , REMATAS K , RITSCHEL T , et al . Reflectance and natural illumination from single-material specular objects using deep learning [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2018 , 40 ( 8 ): 1932 - 1947 .

BELL S , BALA K , SNAVELY N . Intrinsic images in the wild [J ] . ACM Transactions on Graphics , 2014 , 33 ( 4 ): 159 .

Horn B K P , Brooks M J . Shape from shading [J ] . Computer Vision, Graphics, and Image Processing , 1990 , 50 ( 2 ): 255 - 255 .

LOMBARDI S , NISHINO K . Reflectance and natural illumination from a single image [C ] // European Conference on Computer Vision . Berlin, Heidelberg : Springer , 2012 : 582 - 595 .

PANAGOPOULOS A , WANG C H , SAMARAS D , et al . Illumination estimation and cast shadow detection through a higher-order graphical model [C ] // 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2011 : 673 - 680 .

PHONG B T . Illumination for computer generated pictures [J ] . Communications of the ACM , 1975 , 18 ( 6 ): 311 - 317 .

WAGADARIKAR A , JOHN R , WILLETT R , et al . Single disperser design for coded aperture snapshot spectral imaging [J ] . Applied Optics , 2008 , 47 ( 10 ): B44 - B51 .

FORSYTH D A , PONCE J . Computer Vision - A Modern Approach [M ] . Second Edition . London : Pitman , 2012 .

PREETHAM A J , SHIRLEY P , SMITS B . A practical analytic model for daylight [C ] // Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques . New York : ACM , 1999 : 91 - 100 .

FINLAYSON G D , HORDLEY S D , LU C , et al . On the removal of shadows from images [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2006 , 28 ( 1 ): 59 - 68 .

HUANG X , HUA G , TUMBLIN J , et al . What characterizes a shadow boundary under the Sun and sky? [C ] // 2011 International Conference on Computer Vision . Piscataway : IEEE , 2012 : 898 - 905 .

YASUMA F , MITSUNAGA T , ISO D , et al . Generalized assorted pixel camera: Postcapture control of resolution, dynamic range, and spectrum [J ] . IEEE Transactions on Image Processing , 2010 , 19 ( 9 ): 2241 - 2253 .

CHAKRABARTI A , ZICKLER T . Statistics of real-world hyperspectral images [C ] // 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2011 : 193 - 200 .

WANG J F , LI X , YANG J . Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal [C ] // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2018 : 1788 - 1797 .

QU L Q , TIAN J D , HE S F , et al . DeshadowNet: A multi-context embedding deep network for shadow removal [C ] // 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2017 : 2308 - 2316 .

LE H , SAMARAS D . Shadow removal via shadow image decomposition [C ] // 2019 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2020 : 8577 - 8586 .

AKHTAR N , MIAN A . Hyperspectral recovery from RGB images using Gaussian processes [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2020 , 42 ( 1 ): 100 - 113 .

ARAD B , BEN-SHAHAR O . Sparse recovery of hyperspectral signal from natural RGB images [C ] // European Conference on Computer Vision . Cham : Springer , 2016 : 19 - 34 .

WU J Q , AESCHBACHER J , TIMOFTE R . In defense of shallow learned spectral reconstruction from RGB images [C ] // 2017 IEEE International Conference on Computer Vision Workshops (ICCVW) . Piscataway : IEEE , 2018 : 471 - 479 .

ZHANG S P , WANG L Z , ZHANG L , et al . Learning tensor low-rank prior for hyperspectral image reconstruction [C ] // 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2021 : 12001 - 12010 .

GUO R Q , DAI Q Y , HOIEM D . Paired regions for shadow detection and removal [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2013 , 35 ( 12 ): 2956 - 2967 .

GONG H , COSKER D . Interactive shadow removal and ground truth for variable scene categories [C ] // Proceedings of the British Machine Vision Conference 2014 . Nottingham , 2014 : 1 - 11 .

HU X W , ZHU L , FU C W , et al . Direction-aware spatial context features for shadow detection [C ] // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2018 : 7454 - 7462 .

HU X W , JIANG Y T , FU C W , et al . Mask-ShadowGAN: Learning to remove shadows from unpaired data [C ] // 2019 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2020 : 2472 - 2481 .

FU L , ZHOU C Q , GUO Q , et al . Auto-exposure fusion for single-image shadow removal [C ] // 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2021 : 10566 - 10575 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

No data

Related Author

WANG Xiao

YAO Si-yuan

DAI Peng-wen

WANG Rui

CAO Xiao-chun

Related Institution

State Key Laboratory of Information Security， Institute of Information Engineering， CAS

之江实验室

⁰