电子学报 ›› 2022, Vol. 50 ›› Issue (8): 2003-2017.DOI: 10.12263/DZXB.20210639

• 综述评论 • 上一篇    下一篇

基于深度学习的表情动作单元识别综述

邵志文1,2, 周勇1,2, 谭鑫3, 马利庄3,4, 刘兵1,2, 姚睿1,2   

  1. 1.中国矿业大学计算机科学与技术学院,江苏 徐州 221116
    2.矿山数字化教育部工程研究中心,江苏 徐州 221116
    3.上海交通大学计算机科学与工程系,上海 200240
    4.华东师范大学计算机科学与技术学院,上海 200062
  • 收稿日期:2021-05-18 修回日期:2021-09-16 出版日期:2022-08-25
    • 通讯作者:
    • 周勇
    • 作者简介:
    • 邵志文 男,1994年生,安徽马鞍山人. 2020年获得上海交通大学博士学位.现为中国矿业大学计算机科学与技术学院准聘副教授.曾主持国家自然科学基金青年项目、江苏省“双创博士”项目、中央高校基本科研业务费青年项目等.研究方向为计算机视觉和深度学习,涵盖人脸表情识别、人脸表情合成、人脸配准等领域.E-mail: zhiwen_shao@cumt.edu.cn
      周 勇 男,1974年生,江苏徐州人.现为中国矿业大学计算机科学与技术学院教授、博士生导师.曾主持国家自然科学基金面上项目、国家863计划子课题、江苏省“333人才工程”和“六大人才高峰”项目等.研究方向为数据挖掘、机器学习和人工智能.E-mail: yzhou@cumt.edu.cn
    • 基金资助:
    • 国家自然科学基金(62106268);江苏省自然科学基金(BK20201346);江苏省“六大人才高峰”项目(2015-DZXX-010);中央高校基本科研基金(2021QN1072)

Survey of Expression Action Unit Recognition Based on Deep Learning

SHAO Zhi-wen1,2, ZHOU Yong1,2, TAN Xin3, MA Li-zhuang3,4, LIU Bing1,2, YAO Rui1,2   

  1. 1.School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, Jiangsu 221116, China
    2.Engineering Research Center of Mine Digitization, Ministry of Education of the People’s Republic of China, Xuzhou, Jiangsu 221116, China
    3.Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai 200240, China
    4.School of Computer Science and Technology, East China Normal University, Shanghai 200062, China
  • Received:2021-05-18 Revised:2021-09-16 Online:2022-08-25 Published:2022-09-08
    • Corresponding author:
    • ZHOU Yong
    • Supported by:
    • National Natural Science Foundation of China(62106268);Natural Science Foundation of Jiangsu Province(BK20201346);Six Talents Peaks in Jiangsu Province(2015-DZXX-010);Fundamental Research Funds for the Central Universities(2021QN1072)

摘要:

基于深度学习的表情动作单元识别是计算机视觉与情感计算领域的热点课题.每个动作单元描述了一种人脸局部表情动作,其组合可定量地表示任意表情.当前动作单元识别主要面临标签稀缺、特征难捕捉和标签不均衡3个挑战因素.基于此,本文将已有的研究分为基于迁移学习、基于区域学习和基于关联学习的方法,对各类代表性方法进行评述和总结.最后,本文对不同方法进行了比较和分析,并在此基础上探讨了未来动作单元识别的研究方向.

关键词: 表情动作单元识别, 标签稀缺性, 特征难捕捉性, 标签不均衡性, 迁移学习, 区域学习, 关联学习

Abstract:

Expression action unit(AU) recognition based on deep learning is a hot topic in the fields of computer vision and affective computing. Each AU describes a facial local expression action, and the combinations of AUs can quantitatively represent any expression. Current AU recognition mainly faces three challenging factors, scarcity of labels, difficulty of feature capture, and imbalance of labels. On this basis, this paper categorizes the existing researches into transfer learning based, region learning based, and relation learning based methods, and comments and summarizes each category of representative methods. Finally, this paper compares and analyzes different methods, and further discusses the future research directions of AU recognition.

Key words: expression action unit recognition, scarcity of labels, difficulty of feature capture, imbalance of labels, transfer learning, region learning, relation learning

中图分类号: