基于视频先验信息的轻量化去噪卷积神经网络

申屠敏健; 朱强; 朱树元; 孟现东

doi:10.12263/DZXB.20210679

您当前的位置：

首页 >

文章列表页 >

基于视频先验信息的轻量化去噪卷积神经网络

学术论文 | 更新时间：2025-12-08

- 基于视频先验信息的轻量化去噪卷积神经网络
- A Priori Information-Based Lightweight Convolutional Neural Network for Video Denoising
- 电子学报 2023年51卷第6期页码：1510-1517
- 作者机构：
  
  1.电子科技大学信息与通信工程学院，四川成都 611731
  2.鹏城实验室，广东深圳 518000
- 作者简介：
  
  [ "申屠敏健男，1995年9月出生，浙江东阳人.电子科技大学信息与通信工程学院硕士研究生.主要研究方向为深度学习与视频去噪.E-mail: shentu_mj@163.com" ]
  [ "朱强男，1997年2月出生，甘肃定西人.电子科技大学信息与通信工程学院博士研究生.主要研究方向为图像及视频超分辨率.E-mail: zhuqiang@std.uestc.edu.cn" ]
  [ "朱树元男，1980年1月出生，河南平顶山人.电子科技大学信息与通信工程学院教授.主要研究方向为图像视频处理、图像视频编码等.E-mail: eezsy@uestc.edu.cn" ]
- 基金信息：
  
  国家自然科学基金(U20A20184)
- DOI：10.12263/DZXB.20210679
  中图分类号： TP391;
- 收稿：2021-05-26，
  
  修回：2022-05-31，
  
  纸质出版：2023-06-25
- 稿件说明：
移动端阅览
申屠敏健,朱强,朱树元等.基于视频先验信息的轻量化去噪卷积神经网络[J].电子学报,2023,51(06):1510-1517.

SHENTU Min-jian,ZHU Qiang,ZHU Shu-yuan,et al.A Priori Information-Based Lightweight Convolutional Neural Network for Video Denoising[J].ACTA ELECTRONICA SINICA,2023,51(06):1510-1517.
申屠敏健,朱强,朱树元等.基于视频先验信息的轻量化去噪卷积神经网络[J].电子学报,2023,51(06):1510-1517. DOI： 10.12263/DZXB.20210679.

SHENTU Min-jian,ZHU Qiang,ZHU Shu-yuan,et al.A Priori Information-Based Lightweight Convolutional Neural Network for Video Denoising[J].ACTA ELECTRONICA SINICA,2023,51(06):1510-1517. DOI： 10.12263/DZXB.20210679.

摘要

提出了一种基于视频先验信息的轻量化去噪卷积神经网络.先验信息从近邻的多帧视频图像中获取，采用了基于预去噪的视频运动补偿方法消除噪声和运动偏移对信息获取准确度的影响.为降低卷积神经网络复杂度，构建了基于双路处理的卷积神经网络用于去除视频噪声，特别是设计了双路稠密连接单元，实现了网络的轻量化.双路稠密连接单元通过高、低分辨率特征分解和特征拼接，有效降低了网络复杂度.实验结果表明：采用本文方法去除视频噪声能够获得较好的客观评价结果和主观视觉结果.此外，在减少网络参数、降低浮点运算次数和提升运行速度方面均体现出了良好性能.

Abstract

A priori information-based lightweight convolutional neural network (CNN) for video denoising is proposed in this paper. More specifically

the priori information is obtained from the adjacent frames and the pre-denosing-based motion compensation is applied to effectively collect the priori information. Meanwhile

the dual-path processing-based CNN is designed to remove the video noise. Specifically

the dual-path cross connection unit (DCU) is proposed to simplify the feature extraction for the design of a lightweight network. With DCU

the high-resolution features and the low-resolution features are generated and they are concatenated by shorter connections

which achieve low complexity. The experimental results demonstrate that our proposed method offers good objective results as well as subjective results. Moreover

it effectively reduces network parameters and floating point operations

and achieves a fast CNN-based video denosing.

关键词

Keywords

references

DABOV K , FOI A , KATKOVNIK V , et al . Image denoising by sparse 3-D transform-domain collaborative filtering [J]. IEEE Transactions on Image Processing , 2007 , 16 ( 8 ): 2080 - 2095 .

GU S H , ZHANG L , ZUO W M , et al . Weighted nuclear norm minimization with application to image denoising [C]// 2014 IEEE Conference on Computer Vision and Pattern Recognition . Columbus : IEEE , 2014 : 2862 - 2869 .

MAIRAL J , BACH F , PONCE J , et al . Non-local sparse models for image restoration [C]// 2009 IEEE 12th International Conference on Computer Vision . Kyoto : IEEE , 2009 : 2272 - 2279 .

DONG W S , ZHANG L , SHI G M , et al . Nonlocally centralized sparse representation for image restoration [J]. IEEE Transactions on Image Processing , 2013 , 22 ( 4 ): 1620 - 1630 .

BUADES A , COLL B , MOREL J M . Nonlocal image and movie denoising [J]. International Journal of Computer Vision , 2008 , 76 ( 2 ): 123 - 139 .

WEISS Y , FREEMAN W T . What makes a good model of natural images? [C]// 2007 IEEE Conference on Computer Vision and Pattern Recognition . Minneapolis : IEEE , 2007 : 1 - 8 .

DABOV K , FOI A , EGIAZARIAN K . Video denoising by sparse 3D transform-domain collaborative filtering [C]// Proceedings of the 2007 European Signal Processing Conference . Poznan : IEEE , 2007 : 145 - 149 .

MAGGIONI M , BORACCHI G , FOI A , et al . Video denoising, deblocking, and enhancement through separable 4-D nonlocal spatiotemporal transforms [J]. IEEE Transactions on Image Processing , 2012 , 21 ( 9 ): 3952 - 3966 .

ARIAS P , MOREL J M . Towards a Bayesian Video Denoising Method [M]// Advanced Concepts for Intelligent Vision Systems . Cham : Springer International Publishing , 2015 : 107 - 117 .

ZHANG K , ZUO W M , CHEN Y J , et al . Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising [J]. IEEE Transactions on Image Processing , 2017 , 26 ( 7 ): 3142 - 3155 .

DAVY A , EHRET T , MOREL J M , et al . A non-local CNN for video denoising [C]// 2019 IEEE International Conference on Image Processing . Taipei : IEEE , 2019 : 2409 - 2413 .

IANDOLA F N , MOSKEWICZ M W , ASHRAF K , et al . SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and < 0 . 5 MB model size[EB/OL]. ( 2016 )[2021]. http://arxiv.org/abs/1602.07360 http://arxiv.org/abs/1602.07360 .

HOWARD A , ZHU M , CHEN B , et al . MobileNets: Efficient convolutional neural networks for mobile vision applications [EB/OL]. ( 2017 )[2021]. http://arxiv.org/abs/1704.04861 http://arxiv.org/abs/1704.04861 .

CHOLLET F . Xception: Deep learning with depthwise separable convolutions [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition . Honolulu : IEEE , 2017 : 1800 - 1807 .

ZHANG X , ZHOU X , LIN M , et al . ShuffleNet: An extremely efficient convolutional neural network for mobile devices [EB/OL]. ( 2017 )[2021]. http://arxiv.org/abs/1707.01083 http://arxiv.org/abs/1707.01083 .

HAN K , WANG Y H , TIAN Q , et al . GhostNet: More features from cheap operations [C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) . Seattle : IEEE , 2020 : 1577 - 1586 .

孟琭 , 徐磊 , 郭嘉阳 . 一种基于改进的MobileNetV2网络语义分割算法 [J]. 电子学报 , 2020 , 48 ( 9 ): 1769 - 1776 .

MENG L , XU L , GUO J Y . Semantic segmentation algorithm based on improved MobileNetV2 [J]. Acta Electronica Sinica , 2020 , 48 ( 9 ): 1769 - 1776 . (in Chinese)

IOFFE S , SZEGEDY C . Batch normalization: Accelerating deep network training by reducing internal covariate shift [C]// Proceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37 . New York : ACM , 2015 : 448 - 456 .

KRIZHEVSKY A , SUTSKEVER I , HINTON G . ImageNet classification with deep convolutional neural networks [C]// Proceedings of the 2012 International Conference on Neural Information Processing Systems . Doha : Curran Associates Inc. , 2012 : 1097 - 1105 .

TASSANO M , DELON J , VEIT T . FastDVDnet: Towards real-time deep video denoising without flow estimation [C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Seattle : IEEE , 2020 : 1351 - 1360 .

PONT-TUSET J , PERAZZI F , CAELLES S , et al . The 2017 DAVIS challenge on video object segmentation [EB/OL]. ( 2018 )[2021]. https://arxiv.org/abs/1803.00557 https://arxiv.org/abs/1803.00557 .

KINGMA D , BA J . Adam: A method for stochastic optimization [EB/OL]. ( 2017 )[2021]. https://128.84.21.199/abs/1412.6980v4 https://128.84.21.199/abs/1412.6980v4 .

HE K M , ZHANG X Y , REN S Q , et al . Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification [C]// 2015 IEEE International Conference on Computer Vision . Santiago : IEEE , 2015 : 1026 - 1034 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于非一般类算子融合方法及硬件架构设计

共享超分的双分支遥感图像时空融合网络

基于轻量级全连接张量映射网络的高光谱图像分类方法

Cross-CNN：基于CNN和Transformer混合模型的动画跨帧线稿着色算法