基于伪孪生神经网络的低纹理工业零件6D位姿估计

王神龙; 雍宇; 吴晨睿

doi:10.12263/DZXB.20211688

您当前的位置：

首页 >

文章列表页 >

基于伪孪生神经网络的低纹理工业零件6D位姿估计

学术论文 | 更新时间：2025-07-02

- 基于伪孪生神经网络的低纹理工业零件6D位姿估计
- 6D Pose Estimation of Low Texture Industrial Parts Based on Pseudo-Siamese Neural Network
- 电子学报 2023年51卷第1期页码：192-201
- 作者机构：
  
  上海理工大学机械工程学院，上海 200093
- 作者简介：
  
  [ "王神龙男，1989年8月出生于安徽省安庆市.上海理工大学机械工程学院副教授、硕士生导师.主要研究方向为随机动力学与控制、机器视觉与机器学习等.E-mail: shenlongwang@usst.edu.cn" ]
  [ "雍宇男，1996年11月出生于江苏省扬州市.上海理工大学机械工程学院硕士研究生.研究方向为目标检测、6D位姿估计.E-mail: 1219817191@qq.com" ]
  [ "吴晨睿（通讯作者）男，1989年9月出生于黑龙江省哈尔滨市.上海理工大学机械工程学院讲师、硕士生导师.主要研究方向为机器人视觉伺服控制、工业零件目标位姿估计等.E-mail: wuchenrui@usst.edu.cn" ]
- 基金信息：
  
  国家自然科学基金青年项目(52105525);国家自然科学基金面上项目(12172226)
- DOI：10.12263/DZXB.20211688
  中图分类号： TP391.41;
- 收稿：2021-12-23，
  
  修回：2022-03-23，
  
  纸质出版：2023-01-25
- 稿件说明：
移动端阅览
王神龙,雍宇,吴晨睿.基于伪孪生神经网络的低纹理工业零件6D位姿估计[J].电子学报,2023,51(01):192-201.

WANG Shen-long,YONG Yu,WU Chen-rui.6D Pose Estimation of Low Texture Industrial Parts Based on Pseudo-Siamese Neural Network[J].ACTA ELECTRONICA SINICA,2023,51(01):192-201.
王神龙,雍宇,吴晨睿.基于伪孪生神经网络的低纹理工业零件6D位姿估计[J].电子学报,2023,51(01):192-201. DOI： 10.12263/DZXB.20211688.

WANG Shen-long,YONG Yu,WU Chen-rui.6D Pose Estimation of Low Texture Industrial Parts Based on Pseudo-Siamese Neural Network[J].ACTA ELECTRONICA SINICA,2023,51(01):192-201. DOI： 10.12263/DZXB.20211688.

摘要

从单帧RGB图像中获取目标物体的6D位姿信息在机器人抓取、虚拟现实、自动驾驶等领域应用广泛.本文针对低纹理物体位姿估计精度不足的问题，提出一种基于伪孪生神经网络的位姿估计方法.首先，通过渲染CAD模型的方式，获取不同观察角度下的RGB图作为训练样本，解决了深度学习中数据集获取与标注较为繁琐的问题.其次，利用伪孪生神经网络结构学习二维图像特征和物体的三维网格模型特征之间的相似性，即分别采用全卷积网络和三维点云语义分割网络构成伪孪生神经网络，提取二维图像和三维模型的高维深层特征，使用网络推断密集的二维-三维对应关系.最后，通过PnP-RANSAC方法恢复物体的位姿.仿真数据集的实验结果表明，本文提出的方法具有较高的准确性和鲁棒性.

Abstract

Obtaining the 6D pose information of the target object from a single frame RGB image is widely used in the fields of robot capture

virtual reality

automatic driving

and so on. Aiming at the problem of insufficient accuracy of pose estimation of low texture objects

a pose estimation method based on pseudo-siamese neural network is proposed in this paper. Firstly

RGB images from different viewing angles are obtained as training samples by rendering CAD models

which solves the cumbersome problem of data set acquisition and annotation in deep learning. Secondly

the pseudo-siamese neural network structure is used to learn the similarity between the two-dimensional image features and the three-dimensional mesh model features of the object

that is

the full convolution network and the three-dimensional point cloud semantic segmentation network are used to form the pseudo-siamese neural network

extract the high-dimensional deep features of the two-dimensional image and the three-dimensional model

and use the network to infer the dense two-dimensional three-dimensional correspondence. Finally

the pose of the object is restored by PNP-RANSAC method. The experimental results of simulation data sets show that the proposed method has high accuracy and robustness.

关键词

Keywords

references

XIANG Y , MOTTAGHI R , SAVARESE S . Beyond PASCAL: A benchmark for 3D object detection in the wild [C]// IEEE Winter Conference on Applications of Computer Vision . Piscataway : IEEE , 2014 : 75 - 82 .

HE Z X , WU C R , ZHANG S Y , et al . Moment-based 2.5-D visual servoing for textureless planar part grasping [J]. IEEE Transactions on Industrial Electronics , 2019 , 66 ( 10 ): 7821 - 7830 .

BORGHI G , VENTURELLI M , VEZZANI R , et al . POSEidon: face-from-depth for driver pose estimation [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2017 : 5494 - 5503 .

MENG Y , LU Y , RAJ A , al et , Signet: Semantic instance aided unsupervised 3d geometry perception [C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) . Piscataway : IEEE , 2020 : 9802 - 9812 .

XIANG Y , SCHMIDT T , NARAYANAN V , et al . PoseCNN: A convolutional neural network for 6D object pose estimation in cluttered scenes [EB/OL]. ( 2017-11-07 )[ 2022-12 ]. https://arxiv.org/abs/1711.00199 https://arxiv.org/abs/1711.00199 .

KENDALL A , GRIMES M , CIPOLLA R . PoseNet: A convolutional network for real-time 6-DOF camera relocalization [C]// 2015 IEEE International Conference on Computer Vision . Piscataway : IEEE , 2015 : 2938 - 2946 .

KEHL W , MANHARDT F , TOMBARI F , et al . SSD-6D: Making RGB-based 3D detection and 6D pose estimation great again [C]// 2017 IEEE International Conference on Computer Vision . Piscataway : IEEE , 2017 : 1530 - 1538 .

PENG S D , LIU Y , HUANG Q X , et al . PVNet: Pixel-wise voting network for 6DoF pose estimation [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) . Piscataway : IEEE , 2019 : 4556 - 4565 .

YU X , ZHUANG Z Y , KONIUSZ P , et al . 6DoF object pose estimation via differentiable proxy voting loss [EB/OL]. ( 2020-02-10 )[ 2021-12 ]. https://arxiv.org/abs/2002.03923 https://arxiv.org/abs/2002.03923 .

LI Z G , WANG G , JI X Y . CDPN: Coordinates-based disentangled pose network for real-time RGB-based 6-DoF object pose estimation [C]// 2019 IEEE/CVF International Conference on Computer Vision(ICCV) . Piscataway : IEEE , 2019 : 7677 - 7686 .

ZAKHAROV S , SHUGUROV I , ILIC S . DPOD: 6D pose object detector and refiner [C]// 2019 IEEE/CVF International Conference on Computer Vision(ICCV) . Piscataway : IEEE , 2019 : 1941 - 1950 .

WU C R , CHEN L , HE Z X , et al . Pseudo-Siamese graph matching network for textureless objects’ 6-D pose estimation [J]. IEEE Transactions on Industrial Electronics , 2022 , 69 ( 3 ): 2718 - 2727 .

TEKIN B , SINHA S N , FUA P . Real-time seamless single shot 6D object pose prediction [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2018 : 292 - 301 .

QI C R , YI L , SU H , et al . PointNet++: Deep hierarchical feature learning on point sets in a metric space [EB/OL]. ( 2017-06-07 )[ 2021-12 ]. https://arxiv.org/abs/1706.02413 https://arxiv.org/abs/1706.02413 .

GAO G , LAURI M , HU X L , et al . CloudAAE: Learning 6D object pose regression with on-line data synthesis on point clouds [C]// 2021 IEEE International Conference on Robotics and Automation . Piscataway : IEEE , 2021 : 11081 - 11087 .

HE Y S , SUN W , HUANG H B , et al . PVN3D: A deep point-wise 3D keypoints voting network for 6DoF pose estimation [C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) . Piscataway : IEEE , 2020 : 11629 - 11638 .

CHEN W , DUAN J M , BASEVI H , et al . PointPoseNet: Point pose network for robust 6D object pose estimation [C]// 2020 IEEE Winter Conference on Applications of Computer Vision . Piscataway : IEEE , 2020 : 2813 - 2822 .

HE Y S , HUANG H B , FAN H Q , et al . FFB6D: A full flow bidirectional fusion network for 6D pose estimation [C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) . Piscataway : IEEE , 2021 : 3002 - 3012 .

KE Y , SUKTHANKAR R . PCA-SIFT: A more distinctive representation for local image descriptors [C]// Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2004 , 2: II-II.

LI S Q , XU C , XIE M . A robust O(n) solution to the perspective-n-point problem [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2012 , 34 ( 7 ): 1444 - 1450 .

HU Q Y , YANG B , XIE L H , et al . RandLA-net: Efficient semantic segmentation of large-scale point clouds [C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) . Piscataway : IEEE , 2020 : 11105 - 11114 .

DENNINGER M , SUNDERMEYER M , WINKELBAUER D , et al . BlenderProc [EB/OL]. ( 2019-10-25 )[ 2021-12 ]. https://arxiv.org/abs/1911.01911 https://arxiv.org/abs/1911.01911 .

BERTINETTO L , VALMADRE J , HENRIQUES J F , et al . Fully-convolutional Siamese networks for object tracking [EB/OL]. ( 2016-06-30 )[ 2021-12 ]. https://arxiv.org/abs/1606.09549 https://arxiv.org/abs/1606.09549 .

SKALA V . Barycentric coordinates computation in homogeneous coordinates [J]. Computers & Graphics , 2008 , 32 ( 1 ): 120 - 127 .

HODAN T , HALUZA P , OBDRŽÁLEK Š , et al . T-LESS: An RGB-D dataset for 6D pose estimation of texture-less objects [C]// 2017 IEEE Winter Conference on Applications of Computer Vision . Piscataway : IEEE , 2017 : 880 - 888 .

FEY M , LENSSEN J E , WEICHERT F , et al . SplineCNN: Fast geometric deep learning with continuous B-spline kernels [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2018 : 869 - 877 .

HINTERSTOISSER S , LEPETIT V , ILIC S , et al . Model based training, detection and pose estimation of texture-less 3d objects in heavily cluttered scenes [C]// Proceedings of the 11th Asian conference on Computer Vision . New York : ACM , 2012 : 548 - 562 .

左国玉 , 张成威 , 刘洪星 , 等 . 低质量渲染图像的目标物体6D姿态估计 [J]. 控制与决策 , 2022 , 37 ( 1 ): 135 - 141 .

ZUO G Y , ZHANG C W , LIU H X , et al . 6D object pose estimation for low-quality rendering images [J]. Control and Decision , 2022 , 37 ( 1 ): 135 - 141 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

面向时序异常检测的可变视距多向扫描方法

基于稀疏平滑自蒸馏的差分隐私深度学习方法

基于非一般类算子融合方法及硬件架构设计

基于注意力融合多尺度特征的解压缩点云质量增强方法