A Priori Information-Based Lightweight Convolutional Neural Network for Video Denoising

SHENTU Min-jian; ZHU Qiang; ZHU Shu-yuan; MENG Xian-dong

doi:10.12263/DZXB.20210679

您当前的位置：

首页 >

文章列表页 >

A Priori Information-Based Lightweight Convolutional Neural Network for Video Denoising

PAPERS | 更新时间：2025-12-08

- A Priori Information-Based Lightweight Convolutional Neural Network for Video Denoising
- ACTA ELECTRONICA SINICA Vol. 51, Issue 6, Pages: 1510-1517(2023)
- 作者机构：
  
  1.电子科技大学信息与通信工程学院，四川成都 611731
  2.鹏城实验室，广东深圳 518000
- 作者简介：
- 基金信息：
  
  Foundation Item(s):　National Natural Science Foundation of China(U20A20184)
- DOI：10.12263/DZXB.20210679
  CLC： TP391;
- Received：26 May 2021，
  
  Revised：2022-05-31，
  
  Published：25 June 2023
- 稿件说明：
移动端阅览
申屠敏健,朱强,朱树元等.基于视频先验信息的轻量化去噪卷积神经网络[J].电子学报,2023,51(06):1510-1517.

SHENTU Min-jian,ZHU Qiang,ZHU Shu-yuan,et al.A Priori Information-Based Lightweight Convolutional Neural Network for Video Denoising[J].ACTA ELECTRONICA SINICA,2023,51(06):1510-1517.
申屠敏健,朱强,朱树元等.基于视频先验信息的轻量化去噪卷积神经网络[J].电子学报,2023,51(06):1510-1517. DOI： 10.12263/DZXB.20210679.

SHENTU Min-jian,ZHU Qiang,ZHU Shu-yuan,et al.A Priori Information-Based Lightweight Convolutional Neural Network for Video Denoising[J].ACTA ELECTRONICA SINICA,2023,51(06):1510-1517. DOI： 10.12263/DZXB.20210679.

摘要

提出了一种基于视频先验信息的轻量化去噪卷积神经网络.先验信息从近邻的多帧视频图像中获取，采用了基于预去噪的视频运动补偿方法消除噪声和运动偏移对信息获取准确度的影响.为降低卷积神经网络复杂度，构建了基于双路处理的卷积神经网络用于去除视频噪声，特别是设计了双路稠密连接单元，实现了网络的轻量化.双路稠密连接单元通过高、低分辨率特征分解和特征拼接，有效降低了网络复杂度.实验结果表明：采用本文方法去除视频噪声能够获得较好的客观评价结果和主观视觉结果.此外，在减少网络参数、降低浮点运算次数和提升运行速度方面均体现出了良好性能.

Abstract

A priori information-based lightweight convolutional neural network (CNN) for video denoising is proposed in this paper. More specifically

the priori information is obtained from the adjacent frames and the pre-denosing-based motion compensation is applied to effectively collect the priori information. Meanwhile

the dual-path processing-based CNN is designed to remove the video noise. Specifically

the dual-path cross connection unit (DCU) is proposed to simplify the feature extraction for the design of a lightweight network. With DCU

the high-resolution features and the low-resolution features are generated and they are concatenated by shorter connections

which achieve low complexity. The experimental results demonstrate that our proposed method offers good objective results as well as subjective results. Moreover

it effectively reduces network parameters and floating point operations

and achieves a fast CNN-based video denosing.

关键词

Keywords

references

DABOV K , FOI A , KATKOVNIK V , et al . Image denoising by sparse 3-D transform-domain collaborative filtering [J]. IEEE Transactions on Image Processing , 2007 , 16 ( 8 ): 2080 - 2095 .

GU S H , ZHANG L , ZUO W M , et al . Weighted nuclear norm minimization with application to image denoising [C]// 2014 IEEE Conference on Computer Vision and Pattern Recognition . Columbus : IEEE , 2014 : 2862 - 2869 .

MAIRAL J , BACH F , PONCE J , et al . Non-local sparse models for image restoration [C]// 2009 IEEE 12th International Conference on Computer Vision . Kyoto : IEEE , 2009 : 2272 - 2279 .

DONG W S , ZHANG L , SHI G M , et al . Nonlocally centralized sparse representation for image restoration [J]. IEEE Transactions on Image Processing , 2013 , 22 ( 4 ): 1620 - 1630 .

BUADES A , COLL B , MOREL J M . Nonlocal image and movie denoising [J]. International Journal of Computer Vision , 2008 , 76 ( 2 ): 123 - 139 .

WEISS Y , FREEMAN W T . What makes a good model of natural images? [C]// 2007 IEEE Conference on Computer Vision and Pattern Recognition . Minneapolis : IEEE , 2007 : 1 - 8 .

DABOV K , FOI A , EGIAZARIAN K . Video denoising by sparse 3D transform-domain collaborative filtering [C]// Proceedings of the 2007 European Signal Processing Conference . Poznan : IEEE , 2007 : 145 - 149 .

MAGGIONI M , BORACCHI G , FOI A , et al . Video denoising, deblocking, and enhancement through separable 4-D nonlocal spatiotemporal transforms [J]. IEEE Transactions on Image Processing , 2012 , 21 ( 9 ): 3952 - 3966 .

ARIAS P , MOREL J M . Towards a Bayesian Video Denoising Method [M]// Advanced Concepts for Intelligent Vision Systems . Cham : Springer International Publishing , 2015 : 107 - 117 .

ZHANG K , ZUO W M , CHEN Y J , et al . Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising [J]. IEEE Transactions on Image Processing , 2017 , 26 ( 7 ): 3142 - 3155 .

DAVY A , EHRET T , MOREL J M , et al . A non-local CNN for video denoising [C]// 2019 IEEE International Conference on Image Processing . Taipei : IEEE , 2019 : 2409 - 2413 .

IANDOLA F N , MOSKEWICZ M W , ASHRAF K , et al . SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and < 0 . 5 MB model size[EB/OL]. ( 2016 )[2021]. http://arxiv.org/abs/1602.07360 http://arxiv.org/abs/1602.07360 .

HOWARD A , ZHU M , CHEN B , et al . MobileNets: Efficient convolutional neural networks for mobile vision applications [EB/OL]. ( 2017 )[2021]. http://arxiv.org/abs/1704.04861 http://arxiv.org/abs/1704.04861 .

CHOLLET F . Xception: Deep learning with depthwise separable convolutions [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition . Honolulu : IEEE , 2017 : 1800 - 1807 .

ZHANG X , ZHOU X , LIN M , et al . ShuffleNet: An extremely efficient convolutional neural network for mobile devices [EB/OL]. ( 2017 )[2021]. http://arxiv.org/abs/1707.01083 http://arxiv.org/abs/1707.01083 .

HAN K , WANG Y H , TIAN Q , et al . GhostNet: More features from cheap operations [C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) . Seattle : IEEE , 2020 : 1577 - 1586 .

孟琭 , 徐磊 , 郭嘉阳 . 一种基于改进的MobileNetV2网络语义分割算法 [J]. 电子学报 , 2020 , 48 ( 9 ): 1769 - 1776 .

MENG L , XU L , GUO J Y . Semantic segmentation algorithm based on improved MobileNetV2 [J]. Acta Electronica Sinica , 2020 , 48 ( 9 ): 1769 - 1776 . (in Chinese)

IOFFE S , SZEGEDY C . Batch normalization: Accelerating deep network training by reducing internal covariate shift [C]// Proceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37 . New York : ACM , 2015 : 448 - 456 .

KRIZHEVSKY A , SUTSKEVER I , HINTON G . ImageNet classification with deep convolutional neural networks [C]// Proceedings of the 2012 International Conference on Neural Information Processing Systems . Doha : Curran Associates Inc. , 2012 : 1097 - 1105 .

TASSANO M , DELON J , VEIT T . FastDVDnet: Towards real-time deep video denoising without flow estimation [C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Seattle : IEEE , 2020 : 1351 - 1360 .

PONT-TUSET J , PERAZZI F , CAELLES S , et al . The 2017 DAVIS challenge on video object segmentation [EB/OL]. ( 2018 )[2021]. https://arxiv.org/abs/1803.00557 https://arxiv.org/abs/1803.00557 .

KINGMA D , BA J . Adam: A method for stochastic optimization [EB/OL]. ( 2017 )[2021]. https://128.84.21.199/abs/1412.6980v4 https://128.84.21.199/abs/1412.6980v4 .

HE K M , ZHANG X Y , REN S Q , et al . Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification [C]// 2015 IEEE International Conference on Computer Vision . Santiago : IEEE , 2015 : 1026 - 1034 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Operator Fusion Method and Hardware Architecture Design Based on Non-Standard Operators

Shared Super-Resolution Dual-Branch Network for Spatiotemporal Fusion of Remote Sensing Images

Lightweight Fully-Connected Tensorial Mapping Network for Hyperspectral Image Classification

Cross-CNN: An Animation Cross-Frame Sketch Colorization Algorithm Based on Hybrid Model with CNN and Transformer

Related Author

WANG Ying

GAO Lan

ZHANG Zhe

LIU Xin

WU Yi-xiong

ZHANG Wei-gong

FANG Shuai

ZHANG Xiao-xi

Related Institution

College of Information Engineering, Capital Normal University

School of Mathematical Science, Capital Normal University

Faculty of Software Technologics, Shanxi Agricultural University

School of Computer and Information, Hefei University of Technology

Anhui Province Key Laboratory of Industry Safety and Emergency Technology

⁰