6D Pose Estimation of Low Texture Industrial Parts Based on Pseudo-Siamese Neural Network

WANG Shen-long; YONG Yu; WU Chen-rui

doi:10.12263/DZXB.20211688

您当前的位置：

首页 >

文章列表页 >

6D Pose Estimation of Low Texture Industrial Parts Based on Pseudo-Siamese Neural Network

PAPERS | 更新时间：2025-07-02

- 6D Pose Estimation of Low Texture Industrial Parts Based on Pseudo-Siamese Neural Network
- ACTA ELECTRONICA SINICA Vol. 51, Issue 1, Pages: 192-201(2023)
- 作者机构：
  
  上海理工大学机械工程学院，上海 200093
- 作者简介：
- 基金信息：
- DOI：10.12263/DZXB.20211688
  CLC： TP391.41;
- Received：23 December 2021，
  
  Revised：2022-03-23，
  
  Published：25 January 2023
- 稿件说明：
移动端阅览
王神龙,雍宇,吴晨睿.基于伪孪生神经网络的低纹理工业零件6D位姿估计[J].电子学报,2023,51(01):192-201.

WANG Shen-long,YONG Yu,WU Chen-rui.6D Pose Estimation of Low Texture Industrial Parts Based on Pseudo-Siamese Neural Network[J].ACTA ELECTRONICA SINICA,2023,51(01):192-201.
王神龙,雍宇,吴晨睿.基于伪孪生神经网络的低纹理工业零件6D位姿估计[J].电子学报,2023,51(01):192-201. DOI： 10.12263/DZXB.20211688.

WANG Shen-long,YONG Yu,WU Chen-rui.6D Pose Estimation of Low Texture Industrial Parts Based on Pseudo-Siamese Neural Network[J].ACTA ELECTRONICA SINICA,2023,51(01):192-201. DOI： 10.12263/DZXB.20211688.

摘要

从单帧RGB图像中获取目标物体的6D位姿信息在机器人抓取、虚拟现实、自动驾驶等领域应用广泛.本文针对低纹理物体位姿估计精度不足的问题，提出一种基于伪孪生神经网络的位姿估计方法.首先，通过渲染CAD模型的方式，获取不同观察角度下的RGB图作为训练样本，解决了深度学习中数据集获取与标注较为繁琐的问题.其次，利用伪孪生神经网络结构学习二维图像特征和物体的三维网格模型特征之间的相似性，即分别采用全卷积网络和三维点云语义分割网络构成伪孪生神经网络，提取二维图像和三维模型的高维深层特征，使用网络推断密集的二维-三维对应关系.最后，通过PnP-RANSAC方法恢复物体的位姿.仿真数据集的实验结果表明，本文提出的方法具有较高的准确性和鲁棒性.

Abstract

Obtaining the 6D pose information of the target object from a single frame RGB image is widely used in the fields of robot capture

virtual reality

automatic driving

and so on. Aiming at the problem of insufficient accuracy of pose estimation of low texture objects

a pose estimation method based on pseudo-siamese neural network is proposed in this paper. Firstly

RGB images from different viewing angles are obtained as training samples by rendering CAD models

which solves the cumbersome problem of data set acquisition and annotation in deep learning. Secondly

the pseudo-siamese neural network structure is used to learn the similarity between the two-dimensional image features and the three-dimensional mesh model features of the object

that is

the full convolution network and the three-dimensional point cloud semantic segmentation network are used to form the pseudo-siamese neural network

extract the high-dimensional deep features of the two-dimensional image and the three-dimensional model

and use the network to infer the dense two-dimensional three-dimensional correspondence. Finally

the pose of the object is restored by PNP-RANSAC method. The experimental results of simulation data sets show that the proposed method has high accuracy and robustness.

关键词

Keywords

references

XIANG Y , MOTTAGHI R , SAVARESE S . Beyond PASCAL: A benchmark for 3D object detection in the wild [C]// IEEE Winter Conference on Applications of Computer Vision . Piscataway : IEEE , 2014 : 75 - 82 .

HE Z X , WU C R , ZHANG S Y , et al . Moment-based 2.5-D visual servoing for textureless planar part grasping [J]. IEEE Transactions on Industrial Electronics , 2019 , 66 ( 10 ): 7821 - 7830 .

BORGHI G , VENTURELLI M , VEZZANI R , et al . POSEidon: face-from-depth for driver pose estimation [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2017 : 5494 - 5503 .

MENG Y , LU Y , RAJ A , al et , Signet: Semantic instance aided unsupervised 3d geometry perception [C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) . Piscataway : IEEE , 2020 : 9802 - 9812 .

XIANG Y , SCHMIDT T , NARAYANAN V , et al . PoseCNN: A convolutional neural network for 6D object pose estimation in cluttered scenes [EB/OL]. ( 2017-11-07 )[ 2022-12 ]. https://arxiv.org/abs/1711.00199 https://arxiv.org/abs/1711.00199 .

KENDALL A , GRIMES M , CIPOLLA R . PoseNet: A convolutional network for real-time 6-DOF camera relocalization [C]// 2015 IEEE International Conference on Computer Vision . Piscataway : IEEE , 2015 : 2938 - 2946 .

KEHL W , MANHARDT F , TOMBARI F , et al . SSD-6D: Making RGB-based 3D detection and 6D pose estimation great again [C]// 2017 IEEE International Conference on Computer Vision . Piscataway : IEEE , 2017 : 1530 - 1538 .

PENG S D , LIU Y , HUANG Q X , et al . PVNet: Pixel-wise voting network for 6DoF pose estimation [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) . Piscataway : IEEE , 2019 : 4556 - 4565 .

YU X , ZHUANG Z Y , KONIUSZ P , et al . 6DoF object pose estimation via differentiable proxy voting loss [EB/OL]. ( 2020-02-10 )[ 2021-12 ]. https://arxiv.org/abs/2002.03923 https://arxiv.org/abs/2002.03923 .

LI Z G , WANG G , JI X Y . CDPN: Coordinates-based disentangled pose network for real-time RGB-based 6-DoF object pose estimation [C]// 2019 IEEE/CVF International Conference on Computer Vision(ICCV) . Piscataway : IEEE , 2019 : 7677 - 7686 .

ZAKHAROV S , SHUGUROV I , ILIC S . DPOD: 6D pose object detector and refiner [C]// 2019 IEEE/CVF International Conference on Computer Vision(ICCV) . Piscataway : IEEE , 2019 : 1941 - 1950 .

WU C R , CHEN L , HE Z X , et al . Pseudo-Siamese graph matching network for textureless objects’ 6-D pose estimation [J]. IEEE Transactions on Industrial Electronics , 2022 , 69 ( 3 ): 2718 - 2727 .

TEKIN B , SINHA S N , FUA P . Real-time seamless single shot 6D object pose prediction [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2018 : 292 - 301 .

QI C R , YI L , SU H , et al . PointNet++: Deep hierarchical feature learning on point sets in a metric space [EB/OL]. ( 2017-06-07 )[ 2021-12 ]. https://arxiv.org/abs/1706.02413 https://arxiv.org/abs/1706.02413 .

GAO G , LAURI M , HU X L , et al . CloudAAE: Learning 6D object pose regression with on-line data synthesis on point clouds [C]// 2021 IEEE International Conference on Robotics and Automation . Piscataway : IEEE , 2021 : 11081 - 11087 .

HE Y S , SUN W , HUANG H B , et al . PVN3D: A deep point-wise 3D keypoints voting network for 6DoF pose estimation [C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) . Piscataway : IEEE , 2020 : 11629 - 11638 .

CHEN W , DUAN J M , BASEVI H , et al . PointPoseNet: Point pose network for robust 6D object pose estimation [C]// 2020 IEEE Winter Conference on Applications of Computer Vision . Piscataway : IEEE , 2020 : 2813 - 2822 .

HE Y S , HUANG H B , FAN H Q , et al . FFB6D: A full flow bidirectional fusion network for 6D pose estimation [C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) . Piscataway : IEEE , 2021 : 3002 - 3012 .

KE Y , SUKTHANKAR R . PCA-SIFT: A more distinctive representation for local image descriptors [C]// Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2004 , 2: II-II.

LI S Q , XU C , XIE M . A robust O(n) solution to the perspective-n-point problem [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2012 , 34 ( 7 ): 1444 - 1450 .

HU Q Y , YANG B , XIE L H , et al . RandLA-net: Efficient semantic segmentation of large-scale point clouds [C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR) . Piscataway : IEEE , 2020 : 11105 - 11114 .

DENNINGER M , SUNDERMEYER M , WINKELBAUER D , et al . BlenderProc [EB/OL]. ( 2019-10-25 )[ 2021-12 ]. https://arxiv.org/abs/1911.01911 https://arxiv.org/abs/1911.01911 .

BERTINETTO L , VALMADRE J , HENRIQUES J F , et al . Fully-convolutional Siamese networks for object tracking [EB/OL]. ( 2016-06-30 )[ 2021-12 ]. https://arxiv.org/abs/1606.09549 https://arxiv.org/abs/1606.09549 .

SKALA V . Barycentric coordinates computation in homogeneous coordinates [J]. Computers & Graphics , 2008 , 32 ( 1 ): 120 - 127 .

HODAN T , HALUZA P , OBDRŽÁLEK Š , et al . T-LESS: An RGB-D dataset for 6D pose estimation of texture-less objects [C]// 2017 IEEE Winter Conference on Applications of Computer Vision . Piscataway : IEEE , 2017 : 880 - 888 .

FEY M , LENSSEN J E , WEICHERT F , et al . SplineCNN: Fast geometric deep learning with continuous B-spline kernels [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2018 : 869 - 877 .

HINTERSTOISSER S , LEPETIT V , ILIC S , et al . Model based training, detection and pose estimation of texture-less 3d objects in heavily cluttered scenes [C]// Proceedings of the 11th Asian conference on Computer Vision . New York : ACM , 2012 : 548 - 562 .

左国玉 , 张成威 , 刘洪星 , 等 . 低质量渲染图像的目标物体6D姿态估计 [J]. 控制与决策 , 2022 , 37 ( 1 ): 135 - 141 .

ZUO G Y , ZHANG C W , LIU H X , et al . 6D object pose estimation for low-quality rendering images [J]. Control and Decision , 2022 , 37 ( 1 ): 135 - 141 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Variable Horizon Multi-Directional Scanning Method for Time Series Anomaly Detection

Differentially Private with Sparse and Smooth Self-Distillation

Operator Fusion Method and Hardware Architecture Design Based on Non-Standard Operators

A Method for Enhancing the Quality of Decompressed Point Clouds Based on Attention-Fused Multi-Scale Features

Related Author

WANG Shen-long

YONG Yu

WU Chen-rui

HUANG Yu-zhe

GUAN Yong-yuan

WEI Song-jie

ZHAO Deng-feng

XUE Da-xuan

Related Institution

College of Mechanical Engineering， University of Shanghai for Science and Technology

School of Computer Science and Engineering, School of Cyber Science and Engineering, Nanjing University of Science and Technology

School of Information, Renmin University of China

College of Information Engineering, Capital Normal University

School of Mathematical Science, Capital Normal University

⁰