灵巧机器人智能可视化操作界面课件

Proceedings of the 2000 IEEEInternational Conference on Robotics&AutomationSan Francisco,CA April 2000灵巧机器人智能可视化操作界面灵巧机器人智能可视化操作界面 An Intelligent Vision-only Operator Interface for Dexterous RobotsThe goal（本方案的目标）vDevelop methods and algorithms enabling the natural and unencumbered generation of commands for anthropomorphic robots while minimizing specific operator training and dedicated equipment.v1、开发仿人机器人算法：在操作者接受尽量少的特别训练和专门仪器的情况下，使仿人机器人命令的生成正常而顺畅。The project aims at allowing the use of sophisticated,human-like robots in critical environments,such as the International Space Station and the proposed Human Settlement on Mars,where time,space and mass constraints prevent the use of more traditional telepresence systems,such as joysticks and force feedback devices.v2、致力于使复杂精密的仿人机器人在如：国际空间站或已提出的人类登陆火星恶劣环境下可以工作，时间、地点等大量约束都迫使我们不能再使用如操纵杆和力反馈装置等传统的遥控系统。This work is motivated by the parallel development,at NASA Johnson Space Center,of the anthropomorphic robotic system Robonaut,pursuing the design of a robot the size of a suited astronaut.This robot will be teleoperated by a non-intrusive telepresence interface,taking advantage of the natural physical correspondence between the robot and the human operator.v这项工作由NASA JOHNSON空间中心设计的跟宇航员大小相同的仿人机器人系统ROBONAUT的发展来推动。这个机器人将由无干扰远程监控界面进行遥操作，利用机器人和操作人员之间的物理通信。teleoperator interfaces遥操作界面 v1、The input system used here is a visual tracker of the operator arm motions.输入系统：对操作者手臂运动的视觉跟踪器（visual tracker）v2、Graphical displays have been a constant feature of teleoperation output systems.输出系统：图像显示（Graphical displays）是遥操作输出系统中常见的形式。v3、Interface intelligence is usually embedded in the manipulator control algorithm with specific robustness and stability features.智能系统：界面的智能性（Interface intelligence）通常包含在控制器中关于稳定性的控制算法内 visual track视觉跟踪 In the past,visual tracking has been addressed with two main approaches 过去视觉跟踪主要采用两种方式：vmarker-based systems 采用MARKER的系统v marker-less systems 不采用MARKER的系统Marker-based systems use reflective markers on the body of the subject,infrared lighting and multi-ple cameras.v采用MARKER的系统利用红外光和多台相机，将具有反射作用的MARKER放在物体表面。v此方法可以以非常高的精度计算MARKER的三维空间位置。The marker-less approach is based on modeling the appearance of body parts in the images generated by one or more cameras.v不采用MARKER的方法是基于由一架或多架相机生成的图像中身体各部分表面的模型。Graphical displays图像显示vComputer generated graphical simulations are used to give the operator status information and to compensate for transmission time delay.采用计算机生成的图像仿真将位置信息传递给操作者，并补偿传输时间延迟。vIntelligent interaction with the operator is addressed in this paper,where an interactive path planner for the teleoperation of dexterous manipulators is proposed.文中提出了遥操作交互的途径，使智能系统可以与操作者进行交互。vA program monitors task execution and gives the operator feedback about task status.程序监测任务的执行，将任务执行的情况反馈给操作者。Interface intelligence界面的智能性vInterface intelligence is usually embedded in the manipulator control algorithm with specific robustness and stability features.界面的智能性通常包含在控制器中具有精确和稳定特点的控制算法中。vFor example,identification of kinematic singularities is critical for the correct teleoperation of a dexterous arm,since singularity location is not self-evident.In the past,singularity detection has been addressed primarily from the point of view of avoiding any unexpected and dangerous motion of the robot.In particular,approaches such as 15 solve the problem by forcing the robot to pass at a distance from the nominal trajectory,whereas solutions such as 17 keep the robot velocity within acceptable limits.In a teleoperation system these changes can confuse the operator when they happen without warnings.例如，鉴别运动学奇异点对于正确操作灵巧臂是十分重要，但奇异点的位置并不直观。过去，奇异点的探测和定位，主要通过观察，以避免机器人产生任何的意外危险的动作。在一个遥操作系统内这些变化在没有预警的情况下会干扰操作者。the proposed vision-based telepresence interface可视化远程监控界面 vThe features of the proposed vision-based telepresence interface are demonstrated in an off-line implementation based on the graphical simulation of a Robotics Research Corp.(RRC)该特点被基于机器人研究中心（RRC）的图像仿真离线运行所论证 v数据和步骤流程：如图1所示v操作者移动手臂 视觉追踪提取关键点的笛卡儿坐标（肩、肘、腕）将这些坐标转换为关节角 分析关节角是否存在自身碰撞，计算与运动学奇异点的距离 以图像方式叠加在真实场景视频图像上，显示给操作者本界面的整体结构及其主要构成 v视觉跟踪系统视觉跟踪系统 The Visual Tracker v图像显示图像显示 The Graphical Displayv自身碰撞检测自身碰撞检测 Self-Collision Detectionv奇异空间的可视化奇异空间的可视化 Visualization of Singularity SpaceThe Visual Tracker视觉跟踪系统视觉跟踪系统 v对操作者运动起视觉跟踪作用的硬件、软件系统：前端设备 front-end 动作获取设备 acquisition hardware 离线标定软件 off-line calibration software 3维重构软件 3D reconstructing software 标注软件 labeling software front-end前端设备vThe front-end for measuring the operators arm mo-tion is a non-invasive system based on a set of 4 video-cameras and small markers placed on the main joints of the oper,ators body.由四个摄像头和一些位于操作者身体主要关节的小MARKER组成的无干扰系统（non-invasive system）。vThe current implementation simplifies the problem of detecting the markers in each camera,using small light bulbs as markers and requiring a low level of illumination in the acquisition area.现行系统通过用小灯泡作为MARKER并在捕捉区域内采用极低的照明将每个相机探测MARKER的问题简化。Acquisition动作获取We use 4 monochrome NTSC interlaced cameras with 6mm lenses located on a semicircle around the center of the calibrated volume at a distance of about 5 m from the center and about 3 m above the floor level.The automatic gain in each camera was disabled and the shutter time was set to 1/500 sec as a trade off between image contrast and motion blur.用4架镜头焦距为6毫米的单色NTSC相机，围绕标定区域中心5米远、3米高围成半圆。每架相机不能自动获取图像，快门时间为1/500秒以达到画面对比和运动模糊之间的平衡。Camera calibration相机的标定 v内部参数 internal parametersv相机的相互位置 mutual camera positionv世界参考坐标系之间的严格转换 rigid transformation between cameras and world reference frame.The 3D reconstruction software3维重构软件 The 3D reconstrucion software triangulates between different views to determine the 3 dimensional position of each markers.v在不同视角之间做三角测量以测定每个MARKER的3维位置。The visual tracking algorithm takes the set of reconstructed 3D marker coordinates at each frame and assigns to each marker the correct body part.The program requires t o manually label the 3D points in one of the frames in the sequence.v视觉追踪算法采用重构每祯MARKER的3维坐标，然后将每个MARKER赋予正确的身体部位。程序需要在序列中的一祯对这些3维的点进行人工标识。labeling software标注软件vThe visualization tool shown in Figure 3 was implemented in Matlab t o provide a graphical representation of the computed data(for example representing the reconstructed body with a stick figure),for inspection and editing.v图3所示为：可视化工具在MATLAB下执行，为计算数据提供图像显示，用于检验和编辑。The Graphical Display图像显示图像显示vThe operator interface described here interacts with the operator exclusively through a graphical interface incorporating live video of the robot workspace.本操作界面可以让操作者仅通过图形界面与机器人工作空间真实视频画面的结合进行人机交互。vA graphic model of the robot,with a viewpoint matching the video camera,is overlaid t o the video.机器人的图像模型以与相机匹配的视角叠加到真实画面上。Figure 4 shows the graphical simulation of the RRC arm as a transparent image superimposed to the live video of the remote site.图4所视是RRC手臂以透明图像叠加到遥控地点的真实画面的图形仿真Self-Collision Detection自身碰撞检测自身碰撞检测vSelf-collision is defined as the situation when a manipulator link is too close to one of its other links.自身碰撞被定义为一条操作杆与其他操作杆过于接近的情况。vApproach:to test for line-segment t o line-segment distances,taking into account the radii of the links.方法：检测线段间的距离并考虑连杆的半径 Visualization of Singularity Space奇异空间的可视化奇异空间的可视化 vThe approach used here t o generate the joint angles and resolve the kinematic redundancy,given the set of end effector positions generated by the visual tracker,is the Configuration Control method 这里采用生成关节角和解决运动学冗余的方法，由视觉追踪生成末端执行器位置。这就是配置控制方法。vAlthough configuration control prevents dangerous motions,it also alters the robot trajectory,possibly confusing the operator and creating problems in tight environments.尽管配置控制避免了危险动作，但他改变了机器人运动轨迹，可能会干扰操作者并在狭窄环境下产生问题。vTo solve this problem we generate graphical warnings t o localize nearby singularities by using the distance from a singularity expressed as det(JJT)为了解决这个问题，我们生成了图像警报，通过det()得到的到奇异区域的距离确定是否靠近奇异区。vwe visualize the singularity volume in the vicinity of the manipulator wrist based on the value of the determinant.我们根据行列式值使执行器附近的奇异区域可视化。结论和展望结论和展望vThe innovative features of this system include a visual tracker to teleoperate a remote manipulator by simply moving the arms,and an intelligent advisor,to analyze the commanded motions for kinematic correctness,enabling the operator to avoid self collision and singularities.v该系统的创新点包括：对远端控制者通过移动手臂进行遥操作进行视觉跟踪，可以避免自身碰撞，对奇异点的智能运动学分析vThis work is only the beginning of the development of a complete vision-based operator interface.v这一工作只是开发一套完整的可视操作界面的开始vVisual tracking technology needs to be enhanced in:real-time computation of the 3D upper-body pose of the operator.v视觉跟踪技术需要提高到可以对操作者上肢姿态进行三维实时计算的程度 专业术语vNASA National Aeronautics and Space Administration (美国)国家航空和宇航局vRRC Robotics Research Corp 机器人研究中心vZMP Zero moment point 零力矩点vMPEG Movement picture expert group 运动图象专家组,一种压缩比率较大的活动图象和声音的压缩标准vDOF Degree of freedom 自由度专业词汇vanthropomorphic robot 人形机器人vHumanoid robot 仿人机器人vMarker 标识vMotion analysis 运动分析vMotion capture 运动捕捉vKinematics 运动学vDynamics 动力学vImage intensity 图像色饱和度vOrthographic projections 正交投影vRobustness 鲁棒性vStability 稳定性vPixels 像素vConfiguration control method 配置控制法vThreshold 阈值vInteraction 交互vResolution 分辨率vCharacteristic 特征值v servo mechanism 伺服机构vReal time control 实时控制vOff-line programming 离线规划vSingularity 奇异点vTeleoperate 遥操作vTelepresence 远程呈现