多色彩通道特征融合的GAN合成图像检测方法

乔通; 陈彧星; 谢世闯; 姚恒; 罗向阳

doi:10.12263/DZXB.20220711

您当前的位置：

首页 >

文章列表页 >

多色彩通道特征融合的GAN合成图像检测方法

学术论文 | 更新时间：2025-12-11

- 多色彩通道特征融合的GAN合成图像检测方法
- GAN Synthetic Image Detection Using Fused Features in the Multi-Color Channels
- 电子学报 2024年52卷第3期页码：924-936
- 作者机构：
  
  1.杭州电子科技大学网络空间安全学院，浙江杭州 310016
  2.中国科学院信息工程研究所信息安全国家重点实验室，北京 100093
  3.河南省网络空间态势感知重点实验室，河南郑州 450001
  4.上海理工大学光电信息与计算机工程学院，上海 200093
- 作者简介：
  
  [ "乔通男，1986年10月出生于河南省新乡市.现为杭州电子科技大学网络空间安全学院副教授、硕士生导师.主要研究方向为数字图像取证、信息隐藏.主持多项国家级、省部级项目，首批“浙江省高校领军人才培养计划”-青年优秀人才，在国内外发表学术论文60余篇. E-mail: tong.qiao@hdu.edu.cn" ]
  [ "陈彧星女，1998年8月出生于江西省新余市.2020年本科毕业于杭州电子科技大学网络空间安全学院，现为杭州电子科技大学网络空间安全学院硕士，主要研究方向为多媒体信息安全、人工智能安全. E-mail: cyx299@hdu.edu.cn" ]
  [ "谢世闯男，1997年10月出生于河南省驻马店市.2020年本科毕业于河南大学计算机科学与技术专业，现为杭州电子科技大学网络空间安全学院硕士，主要研究方向为多媒体信息安全、人工智能安全. E-mail: shichuang_xie@hdu.edu.cn" ]
  [ "姚恒男，1982年9月出生于安徽省芜湖市.现为上海理工大学光电信息与计算机工程学院副教授，硕士生导师.主要研究方向为多媒体安全、图像处理和模式识别，主持了两项国家自然科学基金项目，在国内外发表学术论文60余篇. E-mail: hyao@usst.edu.cn" ]
  [ "罗向阳男，1978年2月出生于湖北省钟祥市.现为信息工程大学教授、博导，主要研究方向为网络与信息安全，在国内外发表学术论文200余篇. E-mail: luoxy_ieu@sina.com" ]
- 基金信息：
  
  浙江省属高校基本科研业务费专项资金(GK219909299001-007);模式识别国家重点实验室开放课题基金(202200026);河南省网络空间态势感知重点实验室开放课题基金(HNTS2022016);国家重点研发计划(2022YFB3102900);国家自然科学基金(U1804263;62172435;62172281);中原科技创新领军人才项目(214200510019)
- DOI：10.12263/DZXB.20220711
  中图分类号： TP391.41
- 收稿：2022-06-20，
  
  修回：2023-02-23，
  
  纸质出版：2024-03-25
- 稿件说明：
移动端阅览
乔通,陈彧星,谢世闯,等.多色彩通道特征融合的GAN合成图像检测方法[J].电子学报,2024,52(03):924-936.

QIAO Tong, CHEN Yu-xing, XIE Shi-chuang, et al.GAN Synthetic Image Detection Using Fused Features in the Multi-Color Channels[J].Acta Electronica Sinica, 2024, 52(03): 924-936.
乔通,陈彧星,谢世闯,等.多色彩通道特征融合的GAN合成图像检测方法[J].电子学报,2024,52(03):924-936. DOI：10.12263/DZXB.20220711

QIAO Tong, CHEN Yu-xing, XIE Shi-chuang, et al.GAN Synthetic Image Detection Using Fused Features in the Multi-Color Channels[J].Acta Electronica Sinica, 2024, 52(03): 924-936. DOI：10.12263/DZXB.20220711

摘要

当前，生成对抗网络（Generative Adversarial Networks， GAN）合成的逼真图像难以识别，严重危害国家网络安全及社会稳定.与此同时，多数基于深度神经网络模型设计的检测器需要大规模训练样本，且存在模型可解释度不高、泛化性能差等问题.为了克服上述亟待解决的关键性难题，本文提出一种多色彩通道特征融合的GAN合成图像检测方法.首先，探索分析真实自然图像和GAN合成图像在不同色彩空间相邻像素之间的差异，并设计差异度量算法，完成色彩通道选择.其次，利用图像像素间的高度相关性，在八个方向上通过二阶马尔可夫链对相邻像素之间的差分数组进行建模，提取差分像素邻接矩阵特征.最后，利用上述特征，设计一种简单且高效的集成分类器完成GAN合成图像的检测任务.在基于StyleGAN模型合成的伪造人脸数据集中，所提出方法的检测准确率高达100.00%；在小样本训练约束条件下，正负样本对数仅仅为2时，检测准确率高达99.65%；在单类样本训练约束条件下，正样本数仅仅为50时，检测准确率高达92.84%.在基于更先进的StyleGAN2和PGGAN模型合成的伪造场景数据集中，所提出方法的检测准确率达到99.96%以上.以上大量实验表明，本文所提出的方法明显优于比较的GAN合成图像检测方法.本文方法已经开源：

https：//github.com/cyxcyx559/ccss

https://github.com/cyxcyx559/ccss

Abstract

Currently

it is very difficult to identify the images synthesized by generative adversarial networks (GAN)

which severely poses the threat on national cyber security and social stability. Meanwhile

most classifiers based on deep neural networks require large-scale samples for training

where the problems such as low model interpretability and poor generalization performance are less addressed. To overcome the limitations

we propose to design the ensemble classifier using fused features in the multi-color channels. First of all

by studying the discrimination of adjacent pixels in the multi-color channels between natural and GAN synthetic images

the difference metric is designed based on the correlation of adjacent pixels

in order to select the optimal color channels. Secondly

by utilizing the highly-correlated relationship among pixels

the difference a

rray between adjacent pixels are modeled through a second-order Markov chain along eight directions

and meanwhile the subtractive pixel adjacency matrix features are successfully extracted. Finally

based on the extracted features

a simple but efficient detector for identifying GAN synthetic images is constructed. In the image dataset synthesized by the StyleGAN model

the results show that the accuracy of the proposed detector can reach 100.00%. It can also identify GAN synthetic images very well when the pair number of positive and negative training samples is 2 (99.65% accuracy) or only 50 positive training samples are provided (92.84% accuracy). The accuracy can also reach more than 99.96% in the image dataset synthesized by StyleGAN2 and PGGAN models. Numerous experiments show that the proposed method in this paper is better than the compared forensic methods. Our code is available at

https://github.com/cyxcyx559/ccss

关键词

Keywords

references

梁瑞刚 , 吕培卓 , 赵月 , 等 . 视听觉深度伪造检测技术研究综述 [J ] . 信息安全学报 , 2020 , 5 ( 2 ): 1 - 17 .

LIANG R G , LV P Z , ZHAO Y , et al . A survey of audiovisual deepfake detection techniques [J ] . Journal of Cyber Security , 2020 , 5 ( 2 ): 1 - 17 . (in Chinese)

GOODFELLOW I , POUGET-ABADIE J , MIRZA M , et al . Generative adversarial networks [J ] . Communications of the ACM , 2020 , 63 ( 11 ): 139 - 144 .

GU J X , WANG Z H , KUEN J S , et al . Recent advances in convolutional neural networks [J ] . Pattern Recognition , 2018 , 77 : 354 - 377 .

PEVNÝ T , BAS P , FRIDRICH J . Steganalysis by subtractive pixel adjacency matrix [C ] // Proceedings of the 11th ACM workshop on Multimedia and security . New York : ACM , 2009 : 75 - 84 .

DENTON E , CHINTALA S , SZLAM A , et al . Deep generative image models using a Laplacian pyramid of adversarial networks [C ] // Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 1 . New York : ACM , 2015 : 1486 - 1494 .

ZHANG H , XU T , LI H S , et al . StackGAN: Text to photo-realistic image synthesis with stacked generative adversarial networks [C ] // 2017 IEEE International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2017 : 5907 - 5915 .

ZHANG H , XU T , LI H S , et al . StackGAN++: Realistic image synthesis with stacked generative adversarial networks [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2019 , 41 ( 8 ): 1947 - 1962 .

KARRAS T , AILA T M , LAINE S , et al . Progressive growing of GANs for improved quality, stability, and variation [EB/OL ] . ( 2017-10-27 )[ 2022-05-20 ] . https://arxiv.org/abs/1710.10196v2 https://arxiv.org/abs/1710.10196v2 .

GULRAJANI I , AHMED F , ARJOVSKY M , et al . Improved training of wasserstein GANs [C ] // Proceedings of the 31st International Conference on Neural Information Processing Systems . Red Hook : Curran Associates Inc. 2017 : 5769 - 5779 .

ZHANG Z Z , XIE Y P , YANG L . Photographic text-to-image synthesis with a hierarchically-nested adversarial network [C ] // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2018 : 6199 - 6208 .

BROCK A , DONAHUE J , SIMONYAN K . Large scale GAN training for high fidelity natural image synthesis [EB/OL ] . ( 2018-09-28 )[ 2022-05-20 ] . https://arxiv.org/abs/1809.11096 https://arxiv.org/abs/1809.11096 .

LIN C H , CHANG C C , CHEN Y S , et al . COCO-GAN: Generation by parts via conditional coordinating [C ] // 2019 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2019 : 4512 - 4521 .

KARRAS T , LAINE S , AILA T M . A style-based generator architecture for generative adversarial networks [C ] // 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2019 : 4401 - 4410 .

KARRAS T , LAINE S , AITTALA M , et al . Analyzing and improving the image quality of StyleGAN [C ] // 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway : IEEE , 2020 : 8110 - 8119 .

何沛松 , 李伟创 , 张婧媛 , 等 . 面向GAN生成图像的被动取证及反取证技术综述 [J ] . 中国图象图形学报 , 2022 , 27 ( 1 ): 88 - 110 .

HE P S , LI W C , ZHANG J Y , et al . Overview of passive forensics and anti-forensics techniques for GAN-generated image [J ] . Journal of Image and Graphics , 2022 , 27 ( 1 ): 88 - 110 . (in Chinese)

郑远攀 , 李广阳 , 李晔 . 深度学习在图像识别中的应用研究综述 [J ] . 计算机工程与应用 , 2019 , 55 ( 12 ): 20 - 36 .

ZHENG Y P , LI G Y , LI Y . Survey of application of deep learning in image recognition [J ] . Computer Engineering and Applications , 2019 , 55 ( 12 ): 20 - 36 . (in Chinese)

MO H X , CHEN B L , LUO W Q . Fake faces identification via convolutional neural network [C ] // Proceedings of the 6th ACM Workshop on Information Hiding and Multimedia Security . New York : ACM , 2018 : 43 - 47 .

DANG L , HASSAN S , IM S , et al . Deep learning based computer generated face identification using convolutional neural network [J ] . Applied Sciences , 2018 , 8 ( 12 ): 2610 .

ZHANG X , KARAMAN S , CHANG S F . Detecting and simulating artifacts in GAN fake images [C ] // 2019 IEEE International Workshop on Information Forensics and Security (WIFS) . Piscataway : IEEE , 2019 : 1 - 6 .

YU N , DAVIS L , FRITZ M . Attributing fake images to GANs: Learning and analyzing GAN fingerprints [C ] // 2019 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway : IEEE , 2019 : 7556 - 7566 .

ZHUANG Y X , HSU C C . Detecting generated image based on a coupled network with two-step pairwise learning [C ] // 2019 IEEE International Conference on Image Processing (ICIP) . Piscataway : IEEE , 2019 : 3212 - 3216 .

FU Y , SUN T F , JIANG X H , et al . Robust GAN-face detection based on dual-channel CNN network [C ] // 2019 12th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI) . Piscataway : IEEE , 2019 : 1 - 5 .

李旭嵘 , 纪守领 , 吴春明 , 等 . 深度伪造与检测技术综述 [J ] . 软件学报 , 2021 , 32 ( 2 ): 496 - 518 .

LI X R , JI S L , WU C M , et al . Survey on deepfakes and detection techniques [J ] . Journal of Software , 2021 , 32 ( 2 ): 496 - 518 . (in Chinese)

MCCLOSKEY S , ALBRIGHT M . Detecting GAN-generated imagery using saturation cues [C ] // 2019 IEEE International Conference on Image Processing (ICIP) . Piscataway : IEEE , 2019 : 4584 - 4588 .

NATARAJ L , MOHAMMED T M , MANJUNATH B S , et al . Detecting GAN generated fake images using co-occurrence matrices [J ] . Electronic Imaging , 2019 , 31 ( 5 ): 532-1- 532 - 7 .

BARNI M , KALLAS K , NOWROOZI E , et al . CNN detection of GAN-generated face images based on cross-band Co-occurrences analysis [C ] // 2020 IEEE International Workshop on Information Forensics and Security (WIFS) . Piscataway : IEEE , 2020 : 1 - 6 .

LI H D , LI B , TAN S Q , . Identification of deep network generated images using disparities in color components [J ] . Signal Processing , 2020 , 174 : 107616 .

FRIDRICH J , KODOVSKY J . Rich models for steganalysis of digital images [J ] . IEEE Transactions on Information Forensics and Security , 2012 , 7 ( 3 ): 868 - 882 .

MARRA F , GRAGNANIELLO D , COZZOLINO D , et al . Detection of GAN-generated fake images over social networks [C ] // 2018 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR) . Piscataway : IEEE , 2018 : 384 - 389 .

KODOVSKÝ J , FRIDRICH J . Steganalysis of JPEG images using rich models [C ] // Proc SPIE 8303, Media Watermarking, Security, and Forensics 2012 . Burlingame : SPIE , 2012 : 81 - 93 .

GAN Y F , YANG J X , LAI W D . Video object forgery detection algorithm based on VGG-11 convolutional neural network [C ] // 2019 International Conference on Intelligent Computing, Automation and Systems (ICICAS) . Piscataway : IEEE , 2019 : 575 - 580 .

苏兆品 , 吴张倩 , 岳峰 , 等 . 自然环境背景噪声下基于低维深度特征的手机来源识别 [J ] . 电子学报 , 2021 , 49 ( 4 ): 637 - 646 .

SU Z P , WU Z Q , YUE F , et al . Source cell-phone identification under background noise based on low-dimensional deep features [J ] . Acta Electronica Sinica , 2021 , 49 ( 4 ): 637 - 646 . (in Chinese)

SERFOZO R . Basics of Applied Stochastic Processes [M ] . Berlin : Springer , 2009 .

BONNIER N . Contribution to Spatial Gamut Mapping Algorithms [D ] . Paris : Telecom Paristech , 2008 .

CASTLEMAN K R . Digital Image Processing [M ] . Englewood Cliffs : Prentice Hall , 1996 .

CHANG C C , LIN C J . LIBSVM: A library for support vector machines [J ] . ACM Transactions on Intelligent Systems and Technology , 2 ( 3 ): 1 - 27 .

KODOVSKY J , FRIDRICH J , HOLUB V . Ensemble classifiers for steganalysis of digital media [J ] . IEEE Transactions on Information Forensics and Security , 2012 , 7 ( 2 ): 432 - 444 .

HART P E , STORK D G , DUDA R O . Pattern Classification [M ] . Hoboken : Wiley , 2000 .

MATERN F , RIESS C , STAMMINGER M . Exploiting visual artifacts to expose deepfakes and face manipulations [C ] // 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW) . Piscataway : IEEE , 2019 : 83 - 92 .

NGUYEN H H , YAMAGISHI J , ECHIZEN I . Capsule-forensics: Using capsule networks to detect forged images and videos [C ] // ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . Piscataway : IEEE , 2019 : 2307 - 2311 .

YU F , ZHANG Y D , SONG S R , et al . Construction of a large-scale image dataset using deep learning with humans in the loop [EB/OL ] . ( 2015-06-10 )[ 2022-04-20 ] . https://arxiv.org/abs/1506.03365 https://arxiv.org/abs/1506.03365 .

潘兴甲 , 张旭龙 , 董未名 , 等 . 小样本目标检测的研究现状 [J ] . 南京信息工程大学学报(自然科学版) , 2019 , 11 ( 6 ): 698 - 705 .

PAN X J , ZHANG X L , DONG W M , et al . A survey of few-shot object detection [J ] . Journal of Nanjing University of Information Science & Technology (Natural Science Edition) , 2019 , 11 ( 6 ): 698 - 705 . (in Chinese)

葛轶洲 , 刘恒 , 王言 , 等 . 小样本困境下的深度学习图像识别综述 [J ] . 软件学报 , 2022 , 33 ( 1 ): 193 - 210 .

GE Y Z , LIU H , WANG Y , et al . Survey on deep learning image recognition in dilemma of small samples [J ] . Journal of Software , 2022 , 33 ( 1 ): 193 - 210 . (in Chinese)

TAX D M J , DUIN R P W . Support vector domain description [J ] . Pattern Recognition Letters , 1999 , 20 ( 11/12/13 ): 1191 - 1199 .

乔通 , 姚宏伟 , 潘彬民 , 等 . 基于深度学习的数字图像取证技术研究进展 [J ] . 网络与信息安全学报 , 2021 , 7 ( 5 ): 13 - 28 .

QIAO T , YAO H W , PAN B M , et al . Research progress of digital image forensic techniques based on deep learning [J ] . Chinese Journal of Network and Information Security , 2021 , 7 ( 5 ): 13 - 28 . (in Chinese)

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于分频式生成对抗网络的非成对水下图像增强

融合图像与文本特征的组合检索方法

隐空间采样与隐蔽特征提取的CR-GAN复杂无线信道建模

基于离散小波包变换与胶囊生成对抗网络的语音超分辨率算法