Uncalibrated Hand Eye Coordination Method for Robot Based on Pose Transformation Matrix in Variable Environment

JIN Zi-feng; PAN Si-cong; WEI Hui

doi:10.12263/DZXB.20201353

您当前的位置：

首页 >

文章列表页 >

Uncalibrated Hand Eye Coordination Method for Robot Based on Pose Transformation Matrix in Variable Environment

PAPERS | 更新时间：2025-12-08

- Uncalibrated Hand Eye Coordination Method for Robot Based on Pose Transformation Matrix in Variable Environment
- ACTA ELECTRONICA SINICA Vol. 50, Issue 10, Pages: 2318-2328(2022)
- 作者机构：
  
  复旦大学计算机科学技术学院认知算法模型实验室，上海 200438
- 作者简介：
- 基金信息：
- DOI：10.12263/DZXB.20201353
  CLC： TP391;
- Received：30 November 2020，
  
  Revised：2021-05-12，
  
  Published：25 October 2022
- 稿件说明：
移动端阅览
金紫凤,潘思聪,危辉.可变环境下基于位姿变换矩阵的机器人无标定手眼协调方法[J].电子学报,2022,50(10):2318-2328.

JIN Zi-feng,PAN Si-cong,WEI Hui.Uncalibrated Hand Eye Coordination Method for Robot Based on Pose Transformation Matrix in Variable Environment[J].ACTA ELECTRONICA SINICA,2022,50(10):2318-2328.
金紫凤,潘思聪,危辉.可变环境下基于位姿变换矩阵的机器人无标定手眼协调方法[J].电子学报,2022,50(10):2318-2328. DOI： 10.12263/DZXB.20201353.

JIN Zi-feng,PAN Si-cong,WEI Hui.Uncalibrated Hand Eye Coordination Method for Robot Based on Pose Transformation Matrix in Variable Environment[J].ACTA ELECTRONICA SINICA,2022,50(10):2318-2328. DOI： 10.12263/DZXB.20201353.

摘要

智能机器人不同于工业流水线上固定任务的机器人，它们往往需要面对背景、目标物体形状、位置、姿态、尺寸的多种不确定性，这就要求机器人能够像人一样具有极好的手眼协调能力，能基于动态反馈临机调整自己的动作.本文采用基于RGB-D相机的三维深度信息采集手段，构建了一个用于无标定可变场景机器人手眼协调方法，它能够利用几何关系理解三维场景，实时跟踪并准确分割机械臂末端执行器和待抓取的物体，在没有3D模型的条件下计算末端执行器的姿态，通过计算夹持器的姿态以及夹持器和目标对象在相机坐标系中的相对位移向量之后，将该相对位移矢量从相机坐标系转换到夹持器坐标系，估计夹持器位姿变换矩阵，并将齐次变换矩阵输入到机械手逆运动学程序中，求解出机械臂对应的各关节扭角，从而驱动夹持器指向或移动到物体.这一方法硬件资源需求量少、计算速度快、实时性好.

Abstract

Intelligent robots are different from the fixed task robots on the industrial assembly line. They often need to face a variety of uncertainties

such as the background

the shape

position

posture and size of the target object. This requires that the robot can have excellent hand eye coordination ability like us

and adjust its actions based on dynamic feedback. In this paper

a 3D depth information acquisition method based on RGB-D camera is used to construct a hand eye coordination method for uncalibrated variable scene robot. It can understand the 3D scene by using geometric relations

track and segment the end effector and the object to be grasped in real time

calculate the pose of the end effector without 3D model

and calculate the pose of the gripper. After the relative displacement vector of the gripper and the target object in the camera coordinate system

the relative displacement vector is transformed from the camera coordinate system to the gripper coordinate system

and the pose transformation matrix of the gripper is estimated

and the homogeneous transformation matrix is input into the inverse kinematics program of the manipulator to solve the corresponding joint torsion angles of the manipulator

so as to drive the gripper to point or move to the object. This method has the advantages of less hardware resource requirement

fast computing speed and good real-time performance.

关键词

Keywords

references

ZHANG Z . A flexible new technique for camera calibration [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2000 , 22 ( 11 ): 1330 - 1334 .

ZHANG Y , QIU Z , ZHANG X . A simultaneous optimization method of calibration and measurement for a typical hand-eye positioning system [J]. IEEE Transactions on Instrumentation and Measurement , 2020 , 70 : 1 - 11 .

WANG H , FAN X , LU X . Application of a hand-eye self-calibration technique in robot vision [C]// Proceedings of the 2013 25th Chinese Control and Decision Conference (CCDC) . Taiyuan : IEEE , 2013 : 3765 - 3769 .

HERVÉ J Y , SHARMA R , CUCKA P . Toward robust vision-based control: Hand/eye coordination without calibration [C]// Proceedings of the 1991 IEEE International Symposium on Intelligent Control . Arlington : IEEE , 1991 : 457 - 462 .

COOPERSTOCK J R , MILIOS E E . Self-supervised learning for docking and target reaching [J]. Robotics and Autonomous Systems , 1993 , 11 ( 3-4 ): 243 - 260 .

SU J B , PAN Q L , XI Y G . Dynamic coordination of uncalibrated hand/eye robotic system based on neural network [J]. Journal of Systems Engineering and Electronics , 2001 , 12 ( 3 ): 45 - 50 .

XIANG Z , SU J , MA Z . Uncalibrated hand-eye coordination based HRI on humanoid robot [C]// Proceedings of the 33rd Chinese Control Conference . Nanjing : IEEE , 2014 : 8438 - 8443 .

ZENG A , SONG S , YU K T , et al . Robotic pick-and-place of novel objects in clutter with multi-affordance grasping and cross-domain image matching [C]// Proceedings of the 2018 IEEE International Conference on Robotics and Automation(ICRA) . Brisbane : IEEE , 2018 : 3750 - 3757 .

ZENG A , YU K T , SONG S , et al . Multi-view self-supervised deep learning for 6d pose estimation in the amazon picking challenge [C]// Proceedings of the 2017 IEEE International Conference on Robotics and Automation(ICRA) . Singapore : IEEE , 2017 : 1386 - 1383 .

RUTKOWSKI S , ADAMCZYK M , PASTUŁA A , et al . Training using a commercial immersive virtual reality system on hand-eye coordination and reaction time in young musicians: A pilot study [J]. International Journal of Environmental Research and Public Health , 2021 , 18 ( 3 ): 1297 .

BOYD L C , POPOVIC V , SIEBERT J P . Deep reinforcement learning control of hand-eye coordination with a software retina [C]// Proceedings of the 2020 International Joint Conference on Neural Networks(IJCNN) . Shenzhen : IEEE , 2020 : 1 - 8 .

WEI A H , CHEN B Y . Robotic object recognition and grasping with a natural background [J]. International Journal of Advanced Robotic Systems , 2020 , 17 ( 2 ): 1729881420921102 .

HINTERSTOISSER S , HOLZER S , CAGNIART C , et al . Multimodal templates for real-time detection of texture-less objects in heavily cluttered scenes [C]// Proceedings of the 2011 IEEE International Conference on Computer Vision(ICCV) . Barcelona : IEEE , 2011 : 858 - 865 .

PRISACARIU V A , REID I D . PWP3D: Real-time segmentation and tracking of 3D objects [J]. International Journal of Computer Vision , 2012 , 98 ( 3 ): 335 - 354 .

KEHL W , MANHARDT F , TOMBARI F , et al . Ssd-6d: Making rgb-based 3d detection and 6d pose estimation great again [C]// Proceedings of the 2017 IEEE International Conference on Computer Vision . Venice : IEEE , 2017 : 1521 - 1529 .

HINTERSTOISSER S , CAGNIART C , ILIC S , et al . Gradient response maps for real-time detection of textureless objects [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2011 , 34 ( 5 ): 876 - 888 .

XIANG Y , SCHMIDT T , NARAYANAN V , et al . Posecnn: A Convolutional Neural Network for 6d Object Pose Estimation in Cluttered Scenes [EB/OL]. ( 2017-11-01 )[ 2020-11-30 ]. https://arxiv.org/abs/1711.00199 https://arxiv.org/abs/1711.00199 .

WANG C , XU D , ZHU Y , et al . Densefusion: 6d object pose estimation by iterative dense fusion [C]// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Long Beach : IEEE/CVF , 2019 : 3343 - 3352 .

LABBÉ Y , CARPENTIER J , AUBRY M , et al . CosyPose: Consistent multi-view multi-object 6D pose estimation [C]// Proceedings of the 2020 European Conference on Computer Vision . Glasgow : Springer , 2020 : 574 - 591 .

FU C Y , SHVETS M , BERG A C . RetinaMask: Learning to Predict Masks Improves State-of-the-Art Single-Shot Detection for Free [EB/OL]. ( 2019-01-10 )[ 2020-11-30 ]. https://arxiv.org/abs/1901.03353 https://arxiv.org/abs/1901.03353 .

KIFORENKO L , DROST B , TOMBARI F , et al . A performance evaluation of point pair features [J]. Computer Vision and Image Understanding , 2018 , 166 : 66 - 80 .

HINTERSTOISSER S , LEPETIT V , RAJKUMAR N , et al . Going further with point pair features [C]// Proceedings of the 2016 European Conference on Computer Vision . Amsterdam : Springer , 2016 : 834 - 848 .

HENRIQUES J F , CASEIRO R , MARTINS P , et al . High-speed tracking with kernelized correlation filters [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2014 , 37 ( 3 ): 583 - 596 .

LIVERANI A , AMATI G , CALIGIANA G . A CAD-augmented reality integrated environment for assembly sequence check and interactive validation [J]. Concurrent Engineering , 2004 , 12 ( 1 ): 67 - 77 .

FISCHLER M A , BOLLES R C . Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography [J]. Communications of the ACM , 1981 , 24 ( 6 ): 381 - 395 .

HUANG G S , TUNG C K , LIN H C , et al . Inverse kinematics analysis trajectory planning for a robot arm [C]// Proceedings of the 2011 8th Asian Control Conference (ASCC) . Kaohsiung : IEEE , 2011 : 965 - 970 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

An Improved Bionic Cognitive Neural Network for Robot

A Nonlinear Visual Classifier Based on Computational Geometry

A Filter Conflicts Resolving Algorithm Based on Cutting Mapping

A Protocol for Privacy-Preserving Intersect-Determination of Two Polygons

Research on Robot High-Level Planning Based on Fuzzy Cognitive Map

Related Author

SONG Ai-guo

QIAN Kui

HONG Wen-xue

ZHANG Tao

LU Xian-liang

LI Lin

LUO Yong-long

HUANG Liu-sheng

Related Institution

School of Instrument Science and Engineering, Southeast University

燕山大学电气工程学院河北秦皇岛

燕山大学信息科学与工程学院河北秦皇岛

燕山大学电气工程学院

燕山大学信息科学与工程学院

⁰