可变环境下基于位姿变换矩阵的机器人无标定手眼协调方法

金紫凤; 潘思聪; 危辉

doi:10.12263/DZXB.20201353

您当前的位置：

首页 >

文章列表页 >

可变环境下基于位姿变换矩阵的机器人无标定手眼协调方法

学术论文 | 更新时间：2025-12-08

- 可变环境下基于位姿变换矩阵的机器人无标定手眼协调方法
- Uncalibrated Hand Eye Coordination Method for Robot Based on Pose Transformation Matrix in Variable Environment
- 电子学报 2022年50卷第10期页码：2318-2328
- 作者机构：
  
  复旦大学计算机科学技术学院认知算法模型实验室，上海 200438
- 作者简介：
  
  [ "金紫凤女，1995年生，浙江嘉善人.现为复旦大学计算机科学技术学院硕士研究生.主要研究方向为智能机器人.E-mail: zfjin18@fudan.edu.cn" ]
  [ "潘思聪男，1996年生，上海人.现为复旦大学计算机科学技术学院硕士研究生.主要研究方向为智能机器人.E-mail: 18210240033@fudan.edu.cn" ]
  [ "危辉（通讯作者）男，1971年生，江西南昌人.现为复旦大学计算机科学技术学院教授、博士生导师.主要研究方向为人工智能理论与技术、认知科学." ]
- 基金信息：
  
  国家自然科学基金(61771146;61375122)
- DOI：10.12263/DZXB.20201353
  中图分类号： TP391;
- 收稿：2020-11-30，
  
  修回：2021-05-12，
  
  纸质出版：2022-10-25
- 稿件说明：
移动端阅览
金紫凤,潘思聪,危辉.可变环境下基于位姿变换矩阵的机器人无标定手眼协调方法[J].电子学报,2022,50(10):2318-2328.

JIN Zi-feng,PAN Si-cong,WEI Hui.Uncalibrated Hand Eye Coordination Method for Robot Based on Pose Transformation Matrix in Variable Environment[J].ACTA ELECTRONICA SINICA,2022,50(10):2318-2328.
金紫凤,潘思聪,危辉.可变环境下基于位姿变换矩阵的机器人无标定手眼协调方法[J].电子学报,2022,50(10):2318-2328. DOI： 10.12263/DZXB.20201353.

JIN Zi-feng,PAN Si-cong,WEI Hui.Uncalibrated Hand Eye Coordination Method for Robot Based on Pose Transformation Matrix in Variable Environment[J].ACTA ELECTRONICA SINICA,2022,50(10):2318-2328. DOI： 10.12263/DZXB.20201353.

摘要

智能机器人不同于工业流水线上固定任务的机器人，它们往往需要面对背景、目标物体形状、位置、姿态、尺寸的多种不确定性，这就要求机器人能够像人一样具有极好的手眼协调能力，能基于动态反馈临机调整自己的动作.本文采用基于RGB-D相机的三维深度信息采集手段，构建了一个用于无标定可变场景机器人手眼协调方法，它能够利用几何关系理解三维场景，实时跟踪并准确分割机械臂末端执行器和待抓取的物体，在没有3D模型的条件下计算末端执行器的姿态，通过计算夹持器的姿态以及夹持器和目标对象在相机坐标系中的相对位移向量之后，将该相对位移矢量从相机坐标系转换到夹持器坐标系，估计夹持器位姿变换矩阵，并将齐次变换矩阵输入到机械手逆运动学程序中，求解出机械臂对应的各关节扭角，从而驱动夹持器指向或移动到物体.这一方法硬件资源需求量少、计算速度快、实时性好.

Abstract

Intelligent robots are different from the fixed task robots on the industrial assembly line. They often need to face a variety of uncertainties

such as the background

the shape

position

posture and size of the target object. This requires that the robot can have excellent hand eye coordination ability like us

and adjust its actions based on dynamic feedback. In this paper

a 3D depth information acquisition method based on RGB-D camera is used to construct a hand eye coordination method for uncalibrated variable scene robot. It can understand the 3D scene by using geometric relations

track and segment the end effector and the object to be grasped in real time

calculate the pose of the end effector without 3D model

and calculate the pose of the gripper. After the relative displacement vector of the gripper and the target object in the camera coordinate system

the relative displacement vector is transformed from the camera coordinate system to the gripper coordinate system

and the pose transformation matrix of the gripper is estimated

and the homogeneous transformation matrix is input into the inverse kinematics program of the manipulator to solve the corresponding joint torsion angles of the manipulator

so as to drive the gripper to point or move to the object. This method has the advantages of less hardware resource requirement

fast computing speed and good real-time performance.

关键词

Keywords

references

ZHANG Z . A flexible new technique for camera calibration [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2000 , 22 ( 11 ): 1330 - 1334 .

ZHANG Y , QIU Z , ZHANG X . A simultaneous optimization method of calibration and measurement for a typical hand-eye positioning system [J]. IEEE Transactions on Instrumentation and Measurement , 2020 , 70 : 1 - 11 .

WANG H , FAN X , LU X . Application of a hand-eye self-calibration technique in robot vision [C]// Proceedings of the 2013 25th Chinese Control and Decision Conference (CCDC) . Taiyuan : IEEE , 2013 : 3765 - 3769 .

HERVÉ J Y , SHARMA R , CUCKA P . Toward robust vision-based control: Hand/eye coordination without calibration [C]// Proceedings of the 1991 IEEE International Symposium on Intelligent Control . Arlington : IEEE , 1991 : 457 - 462 .

COOPERSTOCK J R , MILIOS E E . Self-supervised learning for docking and target reaching [J]. Robotics and Autonomous Systems , 1993 , 11 ( 3-4 ): 243 - 260 .

SU J B , PAN Q L , XI Y G . Dynamic coordination of uncalibrated hand/eye robotic system based on neural network [J]. Journal of Systems Engineering and Electronics , 2001 , 12 ( 3 ): 45 - 50 .

XIANG Z , SU J , MA Z . Uncalibrated hand-eye coordination based HRI on humanoid robot [C]// Proceedings of the 33rd Chinese Control Conference . Nanjing : IEEE , 2014 : 8438 - 8443 .

ZENG A , SONG S , YU K T , et al . Robotic pick-and-place of novel objects in clutter with multi-affordance grasping and cross-domain image matching [C]// Proceedings of the 2018 IEEE International Conference on Robotics and Automation(ICRA) . Brisbane : IEEE , 2018 : 3750 - 3757 .

ZENG A , YU K T , SONG S , et al . Multi-view self-supervised deep learning for 6d pose estimation in the amazon picking challenge [C]// Proceedings of the 2017 IEEE International Conference on Robotics and Automation(ICRA) . Singapore : IEEE , 2017 : 1386 - 1383 .

RUTKOWSKI S , ADAMCZYK M , PASTUŁA A , et al . Training using a commercial immersive virtual reality system on hand-eye coordination and reaction time in young musicians: A pilot study [J]. International Journal of Environmental Research and Public Health , 2021 , 18 ( 3 ): 1297 .

BOYD L C , POPOVIC V , SIEBERT J P . Deep reinforcement learning control of hand-eye coordination with a software retina [C]// Proceedings of the 2020 International Joint Conference on Neural Networks(IJCNN) . Shenzhen : IEEE , 2020 : 1 - 8 .

WEI A H , CHEN B Y . Robotic object recognition and grasping with a natural background [J]. International Journal of Advanced Robotic Systems , 2020 , 17 ( 2 ): 1729881420921102 .

HINTERSTOISSER S , HOLZER S , CAGNIART C , et al . Multimodal templates for real-time detection of texture-less objects in heavily cluttered scenes [C]// Proceedings of the 2011 IEEE International Conference on Computer Vision(ICCV) . Barcelona : IEEE , 2011 : 858 - 865 .

PRISACARIU V A , REID I D . PWP3D: Real-time segmentation and tracking of 3D objects [J]. International Journal of Computer Vision , 2012 , 98 ( 3 ): 335 - 354 .

KEHL W , MANHARDT F , TOMBARI F , et al . Ssd-6d: Making rgb-based 3d detection and 6d pose estimation great again [C]// Proceedings of the 2017 IEEE International Conference on Computer Vision . Venice : IEEE , 2017 : 1521 - 1529 .

HINTERSTOISSER S , CAGNIART C , ILIC S , et al . Gradient response maps for real-time detection of textureless objects [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2011 , 34 ( 5 ): 876 - 888 .

XIANG Y , SCHMIDT T , NARAYANAN V , et al . Posecnn: A Convolutional Neural Network for 6d Object Pose Estimation in Cluttered Scenes [EB/OL]. ( 2017-11-01 )[ 2020-11-30 ]. https://arxiv.org/abs/1711.00199 https://arxiv.org/abs/1711.00199 .

WANG C , XU D , ZHU Y , et al . Densefusion: 6d object pose estimation by iterative dense fusion [C]// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Long Beach : IEEE/CVF , 2019 : 3343 - 3352 .

LABBÉ Y , CARPENTIER J , AUBRY M , et al . CosyPose: Consistent multi-view multi-object 6D pose estimation [C]// Proceedings of the 2020 European Conference on Computer Vision . Glasgow : Springer , 2020 : 574 - 591 .

FU C Y , SHVETS M , BERG A C . RetinaMask: Learning to Predict Masks Improves State-of-the-Art Single-Shot Detection for Free [EB/OL]. ( 2019-01-10 )[ 2020-11-30 ]. https://arxiv.org/abs/1901.03353 https://arxiv.org/abs/1901.03353 .

KIFORENKO L , DROST B , TOMBARI F , et al . A performance evaluation of point pair features [J]. Computer Vision and Image Understanding , 2018 , 166 : 66 - 80 .

HINTERSTOISSER S , LEPETIT V , RAJKUMAR N , et al . Going further with point pair features [C]// Proceedings of the 2016 European Conference on Computer Vision . Amsterdam : Springer , 2016 : 834 - 848 .

HENRIQUES J F , CASEIRO R , MARTINS P , et al . High-speed tracking with kernelized correlation filters [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2014 , 37 ( 3 ): 583 - 596 .

LIVERANI A , AMATI G , CALIGIANA G . A CAD-augmented reality integrated environment for assembly sequence check and interactive validation [J]. Concurrent Engineering , 2004 , 12 ( 1 ): 67 - 77 .

FISCHLER M A , BOLLES R C . Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography [J]. Communications of the ACM , 1981 , 24 ( 6 ): 381 - 395 .

HUANG G S , TUNG C K , LIN H C , et al . Inverse kinematics analysis trajectory planning for a robot arm [C]// Proceedings of the 2011 8th Asian Control Conference (ASCC) . Kaohsiung : IEEE , 2011 : 965 - 970 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

一种改进型机器人仿生认知神经网络

基于计算几何的非线性可视化分类器设计

一种基于切割映射的规则冲突消除算法

一个保护私有信息的多边形相交判定协议

基于模糊认知图的机器人高层规划研究