全参考图像质量评价回顾与展望

doi:10.12263/DZXB.20200780

电子学报 ›› 2021, Vol. 49 ›› Issue (11): 2261-2272.DOI: 10.12263/DZXB.20200780

全参考图像质量评价回顾与展望

高敏娟¹, 党宏社¹, 魏立力², 刘国军², 张选德³

^1.陕西科技大学电气与控制工程学院，陕西西安 710021
^2.宁夏大学数学统计学院，宁夏银川 750021
^3.陕西科技大学电子信息与人工智能学院，陕西西安 710021

收稿日期:2020-07-27 修回日期:2021-03-09 出版日期:2021-11-25
- 作者简介:
- 高敏娟女，1984年生，山西忻州人.博士研究生，主要研究方向为图像处理、图像质量评价.E-mail:gaominjuan1984@163.com
  党宏社男，1962年生，陕西武功人.教授、博士生导师，主要研究方向为工业过程与优化、计算机控制和图像处理. E-mail:danghs@sust.edu.cn
  魏立力男，1965年生，甘肃天水人.教授，主要研究方向为应用统计与数据分析. E-mail:liliwei@nxu.edu.cn
  刘国军男，1978年生，宁夏吴忠人.教授，主要研究方向为小波分析、偏微分方程图像处理和图像质量评价. E-mail:liugj@nxu.edu.cn
  张选德（通讯作者）男，1979年生，宁夏固原人.教授、博士生导师，主要研究方向为图像恢复、图像质量评价、稀疏表示和低秩逼近理论. E-mail:zhangxuande@sust.edu.cn
- 基金资助:
- 国家自然科学基金 (61871260); 陕西省自然科学基金 (2020JM-509)

Review and Prospect of Full Reference Image Quality Assessment

GAO Min-juan¹, DANG Hong-she¹, WEI Li-li², LIU Guo-jun², ZHANG Xuan-de³

^1.School of Electrical and Control Engineering, Shaanxi University of Science and Technology, Xi’an, Shaanxi 710021, China
^2.School of Mathematics and Statistics, Ningxia University, Yinchuan, Ningxia 750021, China
^3.School of Electronic Information and Artificial Intelligence, Shaanxi University of Science and Technology, Xi’an, Shaanxi 710021, China

Received:2020-07-27 Revised:2021-03-09 Online:2021-11-25 Published:2021-11-25
- Supported by:
- National Natural Science Foundation of China (61871260); Natural Science Foundation of Shaanxi Province, China (2020JM-509)

摘要/Abstract

摘要：

全参考图像质量评价(Full Reference Image Quality Assessment, FR-IQA)是IQA领域广为研究的类型之一.本文回顾了FR-IQA的发展历程，对FR-IQA应用现状和通用FR-IQA问题的构建进行综述，以及对FR-IQA算法进行总结和梳理.并在此基础上，重点分析了现有研究中存在的问题，包括问题构建的合理性、建模的全面性问题、知识驱动与数据驱动结合的问题等.基于对主观评价过程的深入分析，结合现有研究存在的问题，探讨了主观评分采用模糊建模和知识与数据联合驱动构建算法两个可能的研究方向，以期对后续的研究者提供参考.

关键词: 图像质量评价, FR-IQA, 主观评分, 模糊建模, 知识与数据联合驱动

Abstract:

Full reference image quality assessment (FR-IQA) is one of the types widely studied in the field of IQA. This paper reviews the development of FR-IQA, summarizes the application status of FR-IQA and the construction of general FR-IQA problems, and summarizes and combs FR-IQA algorithms. And on this basis, it focuses on the problems existing in the existing research, including the rationality of the problem construction, the comprehensive problem of modeling, and the problem of the combination of knowledge-driven and data-driven. Finally, based on the in-depth analysis of the subjective evaluation process and the existing problems in the existing research, the two possible research directions of subjective scoring using fuzzy modeling and knowledge-data-driven construction algorithm are discussed, in order to provide reference for subsequent researchers.

Key words: image quality assessment, FR-IQA, subjective rating, fuzzy modeling, knowledge-data joint driven

中图分类号:

TP391

高敏娟, 党宏社, 魏立力, 刘国军, 张选德. 全参考图像质量评价回顾与展望[J]. 电子学报, 2021, 49(11): 2261-2272.

GAO Min-juan, DANG Hong-she, WEI Li-li, LIU Guo-jun, ZHANG Xuan-de. Review and Prospect of Full Reference Image Quality Assessment[J]. Acta Electronica Sinica, 2021, 49(11): 2261-2272.

图/表 3

参考文献 78

1	ChandlerD M. Seven challenges in image quality assessment: Past, present, and future research[J]. ISRN Signal Processing, 2013, 2013: 1－53.
2	MohammadiP, Ebrahimi-MoghadamA, ShiraniS. Subjective and objective quality assessment of image: A survey[J]. Majlesi Journal of Electrical Engineering, 2014, 9(1): 419－423.
3	MannosJ, SakrisonD. The effects of a visual fidelity criterion of the encoding of images[J]. IEEE Transactions on Information Theory, 1974, 20(4): 525－536.
4	BuadesA, CollB, MorelJ M. A review of image denoising algorithms, with a new one[J]. Multiscale Modeling & Simulation, 2005, 4(2): 490－530.
5	李亮亮. 基于非下采样剪切波变换的图像增强算法研究[D]. 长春: 吉林大学, 2019.
	LiL L. The research of image enhancement algorithm based on nonsubsampled shearlet transform[D]. Changchun, China: Jilin University, 2019. (in Chinese)
6	WangZ, BovikA C, SheikhH R, et al. Image quality assessment: From error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600－612.
7	沈瑜, 陈小朋, 刘成, 等. 基于混合模型驱动的红外与可见光图像融合[J]. 控制与决策, 2021, 36(9): 2143－2151.
	ShenY, ChenX P, LiuC, et al. Infrared and visible image fusion based on hybrid model driving[J]. Control and Decision, 2021, 36(9): 2143－2151. (in Chinese)
8	DingK Y, MaK D, WangS Q, et al. Comparison of full-reference image quality models for optimization of image processing systems[J]. International Journal of Computer Vision, 2021, 129(4): 1258－1281.
9	ZhaoH, GalloO, FrosioI, et al. Loss functions for image restoration with neural networks[J]. IEEE Transactions on Computational Imaging, 2017, 3(1): 47－57.
10	WangZ, SimoncelliE P, BovikA C. Multiscale structural similarity for image quality assessment[A]. The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers[C]. Pacific Grove, CA, USA: IEEE, 2003. 1398－1402.
11	LaparraV, BalléJ, BerardinoA, et al. Perceptual image quality assessment using a normalized Laplacian pyramid[J]. Electronic Imaging, 2016, 2016(16): 1－6.
12	LaparraV, BerardinoA, BalléJ, et al. Perceptually optimized image rendering[J]. Journal of the Optical Society of America A: Optics, Image Science, and Vision, 2017, 34(9): 1511－1525.
13	MaJ J, NakarmiU, KinC Y S, et al. Diagnostic image quality assessment and classification in medical imaging: Opportunities and challenges[A]. 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI)[C]. Iowa City, IA, USA: IEEE, 2020. 337－340.
14	郭从洲, 李可, 李贺, 等. 遥感图像质量等级分类的深度卷积神经网络方法[J/OL]. 武汉大学学报(信息科学版), 2020, DOI:10.13203/j.whugis20200292.
	GUOC Z, LIK, LIH, et al. A deep convolution -al neural network method for remote sensing image quality classification [J/OL]. Geomatics and Information Sinence of Wuhan University, 2020, DOI:10.13203/j.whugis20200292.(in Chinese)
15	李毅红, 韩焱, 潘晋孝, 等. 基于递变能量线性约束的X射线图像质量评价方法[J]. 电子学报, 2017, 45(3): 669－673.
	LiY H, HanY, PanJ X, et al. X-ray image quality evaluation based on linear constraint with variable energy[J]. Acta Electronica Sinica, 2017, 45(3): 669－673. (in Chinese)
16	SheikhH R, WangZ, BovikA C, et al. Image and video quality assessment research at LIVE[EB/OL]. , 2020.
17	LarsonE C, ChandlerD M. Categorical subjective image quality database[EB/OL]. , 2020.
18	PonomarenkoN, EgiazarianK. Tampere image database TID2008[EB/OL]. , 2020.
19	PonomarenkoN, JinL N, IeremeievO, et al. Image database TID2013: Peculiarities, results and perspectives[J]. Signal Processing: Image Communication, 2015, 30: 57－77.
20	NinassiA, CalletP L, AutrusseauF. Subjective quality assessment IVC database 2005[EB/OL]. , 2020.
21	ChandlerD M, HemamiS S. A57Database 2007[EB/OL]. , 2020.
22	MaK D, DuanmuZ F, WuQ B, et al. Waterloo exploration database: New challenges for image quality assessment models[J]. IEEE Transactions on Image Processing, 2017, 26(2): 1004－1016.
23	ZhangR, IsolaP, EfrosA A, et al. The unreasonable effectiveness of deep features as a perceptual metric[A]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition[C]. Salt Lake City, UT, USA: IEEE, 2018. 586－595.
24	LinH H, HosuV, SaupeD. KADID-10k: A large-scale artificially distorted IQA database[A]. 2019 Eleventh International Conference on Quality of Multimedia Experience (QoMEX)[C]. Berlin, Germany: IEEE, 2019. 1－3.
25	WangZ, SimoncelliE P. Maximum differentiation (MAD) competition: A methodology for comparing computational models of perceptual quantities[J]. Journal of Vision, 2008, 8(12): 1－8.
26	MaK D, DuanmuZ F, WangZ, et al. Group maximum differentiation competition: Model comparison with few samples[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(4): 851－864.
27	SheikhH R, BovikA C, de VecianaG. An information fidelity criterion for image quality assessment using natural scene statistics[J]. IEEE Transactions on Image Processing, 2005, 14(12): 2117－2128.
28	SheikhH R, BovikA C. Image information and visual quality[J]. IEEE Transactions on Image Processing, 2006, 15(2): 430－444.
29	SheikhH R, SabirM F, BovikA C. A statistical evaluation of recent full reference image quality assessment algorithms[J]. IEEE Transactions on Image Processing, 2006, 15(11): 3440－3451.
30	ShnaydermanA, GusevA, EskiciogluA M. An SVD-based grayscale image quality measure for local and global assessment[J]. IEEE Transactions on Image Processing, 2006, 15(2): 422－429.
31	WangZ, ShangX L. Spatial pooling strategies for perceptual image quality assessment[A]. 2006 International Conference on Image Processing[C]. Atlanta, GA, USA: IEEE, 2006. 2945－2948.
32	ChenG H, YangC L, XieS L. Gradient-based structural similarity for image quality assessment[A]. 2006 International Conference on Image Processing[C]. Atlanta, GA, USA: IEEE, 2006. 2929－2932.
33	ChandlerD M, HemamiS S. VSNR: A wavelet-based visual signal-to-noise ratio for natural images[J]. IEEE Transactions on Image Processing, 2007, 16(9): 2284－2298.
34	SampatM P, WangZ, GuptaS, et al. Complex wavelet structural similarity: A new image similarity index[J]. IEEE Transactions on Image Processing, 2009, 18(11): 2385－2401.
35	MoorthyA K, BovikA C. Visual importance pooling for image quality assessment[J]. IEEE Journal of Selected Topics in Signal Processing, 2009, 3(2): 193－201.
36	LarsonE C, ChandlerD M. Most apparent distortion: Full-reference image quality assessment and the role of strategy[J]. Journal of Electronic Imaging, 2010, 19(1): 011006.
37	ZhangL, ZhangL, MouX Q. RFSIM: A feature based image quality assessment metric using Riesz transforms[A]. 2010 IEEE International Conference on Image Processing[C]. Hong Kong, China: IEEE, 2010. 321－324.
38	WangZ, LiQ. Information content weighting for perceptual image quality assessment[J]. IEEE Transactions on Image Processing, 2011, 20(5): 1185－1198.
39	ZhangL, ZhangL, MouX Q, et al. FSIM: A feature similarity index for image quality assessment[J]. IEEE Transactions on Image Processing, 2011, 20(8): 2378－2386.
40	LiuA M, LinW S, NarwariaM. Image quality assessment based on gradient similarity[J]. IEEE Transactions on Image Processing, 2012, 21(4): 1500－1512.
41	ZhuJ Y, WangN C. Image quality assessment by visual gradient similarity[J]. IEEE Transactions on Image Processing, 2012, 21(3): 919－933.
42	CapodiferroL, JacovittiG, Di ClaudioE D. Two-dimensional approach to full-reference image quality assessment based on positional structural information[J]. IEEE Transactions on Image Processing, 2012, 21(2): 505－516.
43	XueW F, ZhangL, MouX Q, et al. Gradient magnitude similarity deviation: A highly efficient perceptual image quality index[J]. IEEE Transactions on Image Processing, 2014, 23(2): 684－695.
44	ZhangX D, FengX C, WangW W, et al. Edge strength similarity for image quality assessment[J]. IEEE Signal Processing Letters, 2013, 20(4): 319－322.
45	WuJ J, LinW S, ShiG M, et al. Perceptual quality metric with internal generative mechanism[J]. IEEE Transactions on Image Processing, 2013, 22(1): 43－54.
46	ChangH W, YangH, GanY, et al. Sparse feature fidelity for perceptual image quality assessment[J]. IEEE Transactions on Image Processing, 2013, 22(10): 4007－4018.
47	ZhangL, ShenY, LiH Y. VSI: A visual saliency-induced index for perceptual image quality assessment[J]. IEEE Transactions on Image Processing, 2014, 23(10): 4270－4281.
48	PreissJ, FernandesF, UrbanP. Color-image quality assessment: From prediction to optimization[J]. IEEE Transactions on Image Processing, 2014, 23(3): 1366－1378.
49	PeiS C, ChenL H. Image quality assessment using human visual DOG model fused with random forest[J]. IEEE Transactions on Image Processing, 2015, 24(11): 3282－3292.
50	LeeD, PlataniotisK N. Towards a full-reference quality assessment for color images using directional statistics[J]. IEEE Transactions on Image Processing, 2015, 24(11): 3950－3965.
51	NafchiH Z, ShahkolaeiA, HedjamR, et al. Mean deviation similarity index: Efficient and reliable full-reference image quality evaluator[J]. IEEE Access, 2016, 4: 5579－5590.
52	BaeS H, KimM. A novel image quality assessment with globally and locally consilient visual quality perception[J]. IEEE Transactions on Image Processing, 2016, 25(5): 2392－2406.
53	ReisenhoferR, BosseS, KutyniokG, et al. A Haar wavelet-based perceptual similarity index for image quality assessment[J]. Signal Processing: Image Communication, 2018, 61: 33－43.
54	LiL D, CaiH, ZhangY B, et al. Sparse representation-based image quality index with adaptive sub-dictionaries[J]. IEEE Transactions on Image Processing, 2016, 25(8): 3775－3786.
55	DingL, HuangH, ZangY. Image quality assessment using directional anisotropy structure measurement[J]. IEEE Transactions on Image Processing, 2017, 26(4): 1799－1809.
56	GuK, LiL D, LuH, et al. A fast reliable image quality predictor by fusing micro- and macro-structures[J]. IEEE Transactions on Industrial Electronics, 2017, 64(5): 3903－3912.
57	SunW, LiaoQ M, XueJ H, et al. SPSIM: A superpixel-based similarity index for full-reference image quality assessment[J]. IEEE Transactions on Image Processing, 2018, 27(9): 4232－4244.
58	AharA, BarriA, SchelkensP. From sparse coding significance to perceptual quality: A new approach for image quality assessment[J]. IEEE Transactions on Image Processing, 2018, 27(2): 879－893.
59	Di ClaudioE D, JacovittiG. A detail-based method for linear full reference image quality prediction[J]. IEEE Transactions on Image Processing, 2018, 27(1): 179－193.
60	JiaH Z, ZhangL, WangT H. Contrast and visual saliency similarity-induced index for assessing image quality[J]. IEEE Access, 2018, 6: 65885－65893.
61	TemelD, AlRegibG. Perceptual image quality assessment through spectral analysis of error representations[J]. Signal Processing: Image Communication, 2019, 70: 37－46.
62	KimW, NguyenA D, LeeS, et al. Dynamic receptive field generation for full-reference image quality assessment[J]. IEEE Transactions on Image Processing, 2020, 29: 4219－4231.
63	LingW Y, HuY. Machine learning to design full-reference image quality assessment algorithm[J]. TELKOMNIKA Indonesian Journal of Electrical Engineering, 2013, 11(6): 3439－3444.
64	NarwariaM, LinW S. SVD-based quality metric for image and video using machine learning[J]. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2012, 42(2): 347－364.
65	LiuT J, LinW S, KuoC C J. Image quality assessment using multi-method fusion[J]. IEEE Transactions on Image Processing, 2013, 22(5): 1793－1807.
66	GastaldoP, ZuninoR, RediJ. Supporting visual quality assessment with machine learning[J]. EURASIP Journal on Image and Video Processing, 2013, 2013(1): 1－15.
67	GuhaT, NezhadaryaE, WardR K. Learning sparse models for image quality assessment[A]. 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)[C]. Florence, Italy: IEEE, 2014. 151－155.
68	LiangY D, WangJ J, WanX Y, et al. Image quality assessment using similar scene as reference[A]. Computer Vision－ECCV 2016[C]. Cham, GER: Springer International Publishing, 2016. 3－18.
69	LiuT J, LiuK H, LinJ Y, et al. A ParaBoost method to image quality assessment[J]. IEEE Transactions on Neural Networks and Learning Systems, 2017, 28(1): 107－121.
70	GaoF, WangY, LiP P, et al. DeepSim: Deep similarity for image quality assessment[J]. Neurocomputing, 2017, 257: 104－114.
71	WangS G, DengC W, LinW S, et al. NMF-based image quality assessment using extreme learning machine[J]. IEEE Transactions on Cybernetics, 2017, 47(1): 232－243.
72	KimJ, LeeS. Deep learning of human visual sensitivity in image quality assessment framework[A]. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)[C]. Florence, Italy: IEEE, 2017. 1969－1977.
73	BosseS, ManiryD, MüllerK R, et al. Deep neural networks for no-reference and full-reference image quality assessment[J]. IEEE Transactions on Image Processing, 2017, 27(1): 206－219.
74	MaK D, DuanmuZ F, WangZ. Geometric transformation invariant image quality assessment using convolutional neural networks[A]. 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)[C]. Calgary, AB, Canada: IEEE, 2018. 6732－6736.
75	PrashnaniE, CaiH, MostofiY, et al. PieAPP: Perceptual image-error assessment through pairwise preference[A]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition[C]. Salt Lake City, UT, USA: IEEE, 2018. 1808－1817.
76	DingK Y, MaK D, WangS Q, et al. Image quality assessment: Unifying structure and texture similarity[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 5810, PP(99): 1.
77	HanS, MaoH, DallyW. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding[EB/OL].,
	2020.
78	SunY, WangX G, TangX O. Sparsifying neural network connections for face recognition[A]. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)[C]. Las Vegas, NV, USA: IEEE, 2016. 4856－4864.

[1]	陈扬, 李旦, 张建秋. 互补色小波域图像质量盲评价方法[J]. 电子学报, 2019, 47(4): 775-783.
[2]	李毅红, 韩焱, 潘晋孝, 陈平. 基于递变能量线性约束的X射线图像质量评价方法[J]. 电子学报, 2017, 45(3): 669-673.
[3]	沈军民, 李俊峰, 戴文战. 结合结构信息和亮度统计的无参考图像质量评价[J]. 电子学报, 2016, 44(4): 804-812.
[4]	刘延伟, 刘金霞, 慈松, 要瑞霄, 赵平华, 王遵义. 3DQoE评价方法及其模型研究进展综述[J]. 电子学报, 2015, 43(3): 568-576.
[5]	米曾真. 小波域中CSF频率与方向加权的图像质量评价方法[J]. 电子学报, 2014, 42(7): 1273-1276.
[6]	郑江云;江巨浪. 基于小波第二级系数误差的图像质量评价模型[J]. 电子学报, 2012, 40(3): 559-563.
[7]	杨春玲;高文瑞. 基于结构相似的小波域图像质量评价方法的研究[J]. 电子学报, 2009, 37(4): 845-849.
[8]	张慧, 张海滨, 李琼, 牛夏牧. 基于人类视觉系统的图像感知哈希算法[J]. 电子学报, 2008, 36(S1): 30-34.
[9]	叶盛楠;苏开娜;肖创柏;段娟. 基于结构信息提取的图像质量评价[J]. 电子学报, 2008, 36(5): 856-861.
[10]	路文, 高新波, 王体胜. 一种基于WBCT的自然图像质量评价方法[J]. 电子学报, 2008, 36(2): 303-308.
[11]	陈得宝, 赵春霞. 复数自适应进化规划及模糊规则基的自动提取[J]. 电子学报, 2007, 35(2): 341-344.
[12]	胡良梅, 高隽, 何柯峰. 图像融合质量评价方法的研究[J]. 电子学报, 2004, 32(S1): 222-225.
[13]	余英林;田菁;蔡志峰. 图像视觉感知信息的初步研究[J]. 电子学报, 2001, 29(10): 1373-1375.
[14]	杜文吉;谢维信;刘源;李隐峰. 基于局部线性度量的模糊建模[J]. 电子学报, 2000, 28(1): 64-66.

全参考图像质量评价回顾与展望

Review and Prospect of Full Reference Image Quality Assessment

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 3

参考文献 78

相关文章 14

编辑推荐

Metrics