基于伪孪生神经网络的低纹理工业零件6D位姿估计

doi:10.12263/DZXB.20211688

电子学报 ›› 2023, Vol. 51 ›› Issue (1): 192-201.DOI: 10.12263/DZXB.20211688

基于伪孪生神经网络的低纹理工业零件6D位姿估计

王神龙, 雍宇, 吴晨睿

上海理工大学机械工程学院，上海 200093

收稿日期:2021-12-23 修回日期:2022-03-23 出版日期:2023-01-25
- 通讯作者:
- 吴晨睿
- 作者简介:
- 王神龙男，1989年8月出生于安徽省安庆市.上海理工大学机械工程学院副教授、硕士生导师.主要研究方向为随机动力学与控制、机器视觉与机器学习等.E-mail: shenlongwang@usst.edu.cn
  雍宇男，1996年11月出生于江苏省扬州市.上海理工大学机械工程学院硕士研究生.研究方向为目标检测、6D位姿估计.E-mail: 1219817191@qq.com
  吴晨睿（通讯作者）男，1989年9月出生于黑龙江省哈尔滨市.上海理工大学机械工程学院讲师、硕士生导师.主要研究方向为机器人视觉伺服控制、工业零件目标位姿估计等.E-mail: wuchenrui@usst.edu.cn
- 基金资助:
- 国家自然科学基金青年项目(52105525);National Natural Science of China(52105525);国家自然科学基金面上项目(12172226);National Natural Science of China(12172226)

6D Pose Estimation of Low Texture Industrial Parts Based on Pseudo-Siamese Neural Network

WANG Shen-long, YONG Yu, WU Chen-rui

College of Mechanical Engineering, University of Shanghai for Science and Technology, Shanghai 200093, China

Received:2021-12-23 Revised:2022-03-23 Online:2023-01-25 Published:2023-02-23
- Corresponding author:
- WU Chen-rui

摘要/Abstract

摘要：

从单帧RGB图像中获取目标物体的6D位姿信息在机器人抓取、虚拟现实、自动驾驶等领域应用广泛.本文针对低纹理物体位姿估计精度不足的问题，提出一种基于伪孪生神经网络的位姿估计方法.首先，通过渲染CAD模型的方式，获取不同观察角度下的RGB图作为训练样本，解决了深度学习中数据集获取与标注较为繁琐的问题.其次，利用伪孪生神经网络结构学习二维图像特征和物体的三维网格模型特征之间的相似性，即分别采用全卷积网络和三维点云语义分割网络构成伪孪生神经网络，提取二维图像和三维模型的高维深层特征，使用网络推断密集的二维-三维对应关系.最后，通过PnP-RANSAC方法恢复物体的位姿.仿真数据集的实验结果表明，本文提出的方法具有较高的准确性和鲁棒性.

关键词: 深度学习, 6D位姿估计, 仿真数据集, 伪孪生神经网络, 点向密集匹配

Abstract:

Obtaining the 6D pose information of the target object from a single frame RGB image is widely used in the fields of robot capture, virtual reality, automatic driving, and so on. Aiming at the problem of insufficient accuracy of pose estimation of low texture objects, a pose estimation method based on pseudo-siamese neural network is proposed in this paper. Firstly, RGB images from different viewing angles are obtained as training samples by rendering CAD models, which solves the cumbersome problem of data set acquisition and annotation in deep learning. Secondly, the pseudo-siamese neural network structure is used to learn the similarity between the two-dimensional image features and the three-dimensional mesh model features of the object, that is, the full convolution network and the three-dimensional point cloud semantic segmentation network are used to form the pseudo-siamese neural network, extract the high-dimensional deep features of the two-dimensional image and the three-dimensional model, and use the network to infer the dense two-dimensional three-dimensional correspondence. Finally, the pose of the object is restored by PNP-RANSAC method. The experimental results of simulation data sets show that the proposed method has high accuracy and robustness.

Key words: deep learning, 6D pose estimation, simulation data set, pseudo-siamese neural network, dense matching of points

中图分类号:

TP391.41

王神龙, 雍宇, 吴晨睿. 基于伪孪生神经网络的低纹理工业零件6D位姿估计[J]. 电子学报, 2023, 51(1): 192-201.

Shen-long WANG, Yu YONG, Chen-rui WU . 6D Pose Estimation of Low Texture Industrial Parts Based on Pseudo-Siamese Neural Network[J]. Acta Electronica Sinica, 2023, 51(1): 192-201.

图/表 12

图1 本文研究技术路线图

图2 工业零件仿真数据集的建立(a) 零件实物图 (b) 三维建模图 (c) 点云模型图 (d) 仿真渲染图

图3 目标的三维坐标轴以及虚拟观察采样点

图4 仿真数据集(a) 工业零件 (b) 黄鸭 (c) 相机 (d) 花洒 (e) T-Less 零件

图5 位姿估计网络结构图

图6 二维平面中的重心坐标以及像素分配原理(a) 重心坐标原理 (b) 像素分配原理

图7 像素密集的二维-三维对应点匹配说明(a) 重心坐标原理 (b) 像素分配原理

表1 不同位姿估计方法在ADD指标下的测试结果 (%)

估计对象	方法
估计对象	PVNET	CDPN	YOLO6D	DPVL	DPOD	PVN3D	本文方法
工业零件	92.0	88.3	57.4	86.7	74.3	88.4	89.1
花洒	95.5	95.9	68.8	98.5	52.6	99.5	96.3
相机	86.9	91.7	36.6	94.1	24.2	99.6	88.6
黄鸭	52.6	66.8	27.2	63.5	26.1	98.2	67.2

图8 目标对象位姿估计的可视化结果

表2 工业零件数据集在不同度量指标以及方法下的测试对比结果 (%)

度量指标	方法
度量指标	PVNET	CDPN	YOLO6D	DPVL	DPOD	PVN3D	本文方法
5°5 cm	73.4	94.3	—	—	—	—	94.9
二维投影度量	99.0	98.1	90.4	99.4	—	—	92.7
网络参数量	12.9	26.7	7.8	—	13.5	13.2	10.1
计算成本	74.3	136.4	32.1	—	75.2	75.8	62.5

图9 不同目标对象下位姿估计的准确率

图10 不同采样点数下位姿估计的准确率

参考文献 28

1	XIANG Y, MOTTAGHI R, SAVARESE S. Beyond PASCAL: A benchmark for 3D object detection in the wild[C]//IEEE Winter Conference on Applications of Computer Vision. Piscataway: IEEE, 2014: 75-82.
2	HE Z X, WU C R, ZHANG S Y, et al. Moment-based 2.5-D visual servoing for textureless planar part grasping[J]. IEEE Transactions on Industrial Electronics, 2019, 66(10): 7821-7830.
3	BORGHI G, VENTURELLI M, VEZZANI R, et al. POSEidon: face-from-depth for driver pose estimation[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 5494-5503.
4	MENG Y, LU Y, RAJ A, et al, Signet: Semantic instance aided unsupervised 3d geometry perception[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway: IEEE, 2020: 9802-9812.
5	XIANG Y, SCHMIDT T, NARAYANAN V, et al. PoseCNN: A convolutional neural network for 6D object pose estimation in cluttered scenes[EB/OL]. (2017-11-07)[2022-12]. .
6	KENDALL A, GRIMES M, CIPOLLA R. PoseNet: A convolutional network for real-time 6-DOF camera relocalization[C]//2015 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2015: 2938-2946.
7	KEHL W, MANHARDT F, TOMBARI F, et al. SSD-6D: Making RGB-based 3D detection and 6D pose estimation great again[C]//2017 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2017: 1530-1538.
8	PENG S D, LIU Y, HUANG Q X, et al. PVNet: Pixel-wise voting network for 6DoF pose estimation[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway: IEEE, 2019: 4556-4565.
9	YU X, ZHUANG Z Y, KONIUSZ P, et al. 6DoF object pose estimation via differentiable proxy voting loss[EB/OL]. (2020-02-10)[2021-12]. .
10	LI Z G, WANG G, JI X Y. CDPN: Coordinates-based disentangled pose network for real-time RGB-based 6-DoF object pose estimation[C]//2019 IEEE/CVF International Conference on Computer Vision(ICCV). Piscataway: IEEE, 2019: 7677-7686.
11	ZAKHAROV S, SHUGUROV I, ILIC S. DPOD: 6D pose object detector and refiner[C]//2019 IEEE/CVF International Conference on Computer Vision(ICCV). Piscataway: IEEE, 2019: 1941-1950.
12	WU C R, CHEN L, HE Z X, et al. Pseudo-Siamese graph matching network for textureless objects’ 6-D pose estimation[J]. IEEE Transactions on Industrial Electronics, 2022, 69(3): 2718-2727.
13	TEKIN B, SINHA S N, FUA P. Real-time seamless single shot 6D object pose prediction[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 292-301.
14	QI C R, YI L, SU H, et al. PointNet++: Deep hierarchical feature learning on point sets in a metric space[EB/OL]. (2017-06-07)[2021-12]. .
15	GAO G, LAURI M, HU X L, et al. CloudAAE: Learning 6D object pose regression with on-line data synthesis on point clouds[C]//2021 IEEE International Conference on Robotics and Automation. Piscataway: IEEE, 2021: 11081-11087.
16	HE Y S, SUN W, HUANG H B, et al. PVN3D: A deep point-wise 3D keypoints voting network for 6DoF pose estimation[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway: IEEE, 2020: 11629-11638.
17	CHEN W, DUAN J M, BASEVI H, et al. PointPoseNet: Point pose network for robust 6D object pose estimation[C]//2020 IEEE Winter Conference on Applications of Computer Vision. Piscataway: IEEE, 2020: 2813-2822.
18	HE Y S, HUANG H B, FAN H Q, et al. FFB6D: A full flow bidirectional fusion network for 6D pose estimation[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway: IEEE, 2021: 3002-3012.
19	KE Y, SUKTHANKAR R. PCA-SIFT: A more distinctive representation for local image descriptors[C]//Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2004, 2: II-II.
20	LI S Q, XU C, XIE M. A robust O(n) solution to the perspective-n-point problem[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(7): 1444-1450.
21	HU Q Y, YANG B, XIE L H, et al. RandLA-net: Efficient semantic segmentation of large-scale point clouds[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway: IEEE, 2020: 11105-11114.
22	DENNINGER M, SUNDERMEYER M, WINKELBAUER D, et al. BlenderProc[EB/OL]. (2019-10-25)[2021-12]. .
23	BERTINETTO L, VALMADRE J, HENRIQUES J F, et al. Fully-convolutional Siamese networks for object tracking[EB/OL]. (2016-06-30)[2021-12]. .
24	SKALA V. Barycentric coordinates computation in homogeneous coordinates[J]. Computers & Graphics, 2008, 32(1): 120-127.
25	HODAN T, HALUZA P, OBDRŽÁLEK Š, et al. T-LESS: An RGB-D dataset for 6D pose estimation of texture-less objects[C]//2017 IEEE Winter Conference on Applications of Computer Vision. Piscataway: IEEE, 2017: 880-888.
26	FEY M, LENSSEN J E, WEICHERT F, et al. SplineCNN: Fast geometric deep learning with continuous B-spline kernels[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 869-877.
27	HINTERSTOISSER S, LEPETIT V, ILIC S, et al. Model based training, detection and pose estimation of texture-less 3d objects in heavily cluttered scenes[C]//Proceedings of the 11th Asian conference on Computer Vision. New York: ACM, 2012: 548-562.
28	左国玉, 张成威, 刘洪星, 等. 低质量渲染图像的目标物体6D姿态估计[J]. 控制与决策, 2022, 37(1): 135-141.
	ZUO G Y, ZHANG C W, LIU H X, et al. 6D object pose estimation for low-quality rendering images[J]. Control and Decision, 2022, 37(1): 135-141.

[1]	李钦, 刘伟, 牛朝阳, 宝音图, 惠周勃. 低信噪比下基于分裂EfficientNet网络的雷达信号调制方式识别[J]. 电子学报, 2023, 51(3): 675-686.
[2]	范兵兵, 何庭建, 张聪炫, 陈震, 黎明. 联合遮挡约束与残差补偿的特征金字塔光流计算方法[J]. 电子学报, 2023, 51(3): 648-657.
[3]	张聿远, 闫文君, 张立民. 基于多模态特征融合网络的空时分组码识别算法[J]. 电子学报, 2023, 51(2): 489-498.
[4]	许新征, 李杉. 基于特征膨胀卷积模块的轻量化技术研究[J]. 电子学报, 2023, 51(2): 355-364.
[5]	张永梅, 孙捷. 基于动静态特征双输入神经网络的咳嗽声诊断COVID-19算法[J]. 电子学报, 2023, 51(1): 202-212.
[6]	袁海英, 成君鹏, 曾智勇, 武延瑞. Mobile_BLNet：基于Big-Little Net的轻量级卷积神经网络优化设计[J]. 电子学报, 2023, 51(1): 180-191.
[7]	李滔, 董秀成, 林宏伟. 基于深监督跨尺度注意力网络的深度图像超分辨率重建[J]. 电子学报, 2023, 51(1): 128-138.
[8]	郭晓轩, 冯其波, 冀振燕, 郑发家, 杨燕燕. 多线激光光条图像缺陷分割模型研究[J]. 电子学报, 2023, 51(1): 172-179.
[9]	贾童瑶, 卓力, 李嘉锋, 张菁. 基于深度学习的单幅图像去雾研究进展[J]. 电子学报, 2023, 51(1): 231-245.
[10]	何滢婕, 刘月峰, 边浩东, 郭威, 张小燕. 基于Informer的电池荷电状态估算及其稀疏优化方法[J]. 电子学报, 2023, 51(1): 50-56.
[11]	吴靖, 叶晓晶, 黄峰, 陈丽琼, 王志锋, 刘文犀. 基于深度学习的单帧图像超分辨率重建综述[J]. 电子学报, 2022, 50(9): 2265-2294.
[12]	李雪莹, 王田路, 梁鹏, 王翀. 基于系统模型的用户评论中非功能需求的自动分类[J]. 电子学报, 2022, 50(9): 2079-2089.
[13]	琚长瑞, 秦晓燕, 袁广林, 李豪, 朱虹. 尺度敏感损失与特征融合的快速小目标检测方法[J]. 电子学报, 2022, 50(9): 2119-2126.
[14]	张志昌, 于沛霖, 庞雅丽, 朱林, 曾扬扬. SMGN：用于对话状态跟踪的状态记忆图网络[J]. 电子学报, 2022, 50(8): 1851-1858.
[15]	张亚洲, 俞洋, 朱少林, 陈锐, 戎璐, 梁辉. 一种量子概率启发的对话讽刺识别网络模型[J]. 电子学报, 2022, 50(8): 1885-1893.

基于伪孪生神经网络的低纹理工业零件6D位姿估计

6D Pose Estimation of Low Texture Industrial Parts Based on Pseudo-Siamese Neural Network

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 12

参考文献 28

相关文章 15

编辑推荐

Metrics