3D视觉检测-王振宇.ppt

2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,1,2020/8/17,3D-Locate & 3D Sensor,刘少鹏 E-Mail: Shaopeng.liuximing- Tel: 15261520319,Continuously Exceeds customer Expectations,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,2,ID Readers,Cognex Core Products,What is Cognex 3D Vision?,Cognex 3D vision uses two dimensional images and tools from one or more cameras point of view to generate information about an object in 3D space. Using 2D images and tools allows Cognex 3D vision to accuratelypinpoint a 3D position by establishinga mathematical relationship betweenthe 2D coordinate systems in eachcameras image and the physicalworld.,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,2020/8/17,What is VisionPro 3D Add-In?,The VisionPro 3D Add-In is a .Net library of classes that enables VisionPro software to perform 3D calibration and object location. A supported software version of VisionPro is required to create a 3D vision application using the VisionPro 3D Add-In.,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,2020/8/17,What is VisionPro 3D Add-In?,An extension to VisionPro for 3D pose determination that can be used in different configurations:,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,2020/8/17,VisionPro 3D Add-In Getting Starter Kit,Tripod Mounting plate, screws, and Allen wrenches Dual-port GigE network adapter Cognex 10mm calibration plate Cognex 20mm calibration plate(optional) Two Basler Ace 1.3 MP cameras and PoE power supplies(update to 2MP in 7.2 Starter Kit) Two 6mm lenses 4 ENET cables Two 25mm x 50mm x 75mm spacer blocks,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,2020/8/17,VisionPro 3D Starter Kit Hardware,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,2020/8/17,Calibration Overview,3D calibration establishes a mathematical relationship between the 2D coordinate system associated with the pixels in an acquired image,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,2020/8/17,3D Calibration,Calibration procedure depends on application setup: 3D Camera calibration (fixed cameras, manually move calibration plate to different positions) 3D Robot Hand-Eye calibration (moving camera mounted at robot hand, fixed calibration plate) 3D Robot Hand-Eye calibration (stationary camera, moving calibration plate mounted at robot hand),2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,2020/8/17,3D Calibration,Calibration plate,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,2020/8/17,Raw2D Concepts,Raw2D space refers to a left-handed 2D coordinate system established by and acquired image. Uses pixels as units. Origin at upper left-hand corner. A Raw2D point in Raw2D space is simply an (x, y) coordinate identified within a Raw2D coordinate system.,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,2020/8/17,Phys3D Concepts,Phys3D refers to a right-handed 3D physical coordinate system established by the 3D calibration process. Uses physical units (initially defined by the calibration plate and spacing between plate views). Origin defined by fiducial mark on calibration plate. X and Y axes are aligned to calibration grid, Z axis normal to plate extending away from camera.,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,2020/8/17,3D Calibration Basics,3D calibration establishes an accurate correspondence between 2D locations in and image using a 3D calibrated camera and a 3D location in physical in front of the camera. The correspondence is created by the mathematical linking of the different transformations (Coordinate spaces).,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,2020/8/17,3D Camera Calibration,A Camera Calibration includes two sets of parameters: Intrinsic parameters Properties of the acquisition system that are internal to the camera/lens system (lens distortion) Extrinsic parameters Properties of the acquisition system that are external to the camera/lens system (position and orientation of camera in 3D space),2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,2020/8/17,3D Camera Calibration,A Camera Calibration is needed for all 3D operations (triangulation, pose estimation,) A CameraCalibration can be created from a HandEye calibration result and a robot pose.,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,2020/8/17,3D Camera Calibration,Multiple stationary cameras Requires one World Origin Viewset Requires Tilted Viewsets Elevated Viewsets recommended(Optional) CalibrationWizard application can be used for 3D Camera Calibration.,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,2020/8/17,3D Camera Calibration,World Origin ViewsetThe Calibration Plate coordinate system defines the 3D World Coordinate System (Phys3D space),2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,2020/8/17,3D Camera Calibration,Tilted ViewsetsThe Calibration Plate must be tilted and rotated,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,2020/8/17,3D Camera Calibration,Elevated Viewsets The Calibration plate must be elevated by multiple steps across the operating volume height. It must remain parallel to the World Origin position! Elevated views improve accuracy of Intrinsic params.,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,2020/8/17,Assessing the Calibrations Quality,Using the Cog3DCameraCalibrationResult class: Use the OverallResidualsRaw2D() and OverallResidualsPhys3D() methods to obtain statistics for all plate poses and cameras. These global residual error measurements reflect the overall accuracy of the calibration Use the ResidualsRaw2D(j, i) and ResidualsPhys3D(j, i) for statistics on a combination of a single camera (i) and a single plate pose (j). These plate and camera level statistics help to identify optical or mechanical issue with a particular camera of plate pose.,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,2020/8/17,Spaces associated with a 3D calibration using two cameras,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,2020/8/17,3D Robot Hand-Eye Calibration (moving camera(s),Fixed calibration plate must be acquired from different distances and view angles. Robot (hand) positions must be recorded.,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,2020/8/17,Robot Hand-Eye Calibration (moving camera(s),Coordinate spaces(Moving Camera(s),2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,2020/8/17,Robot Hand-Eye Calibration( Stationary Camera(s),Coordinate spaces(Moving calibration plate),2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,2020/8/17,Robot Pose and HandEye Calibration result,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,2020/8/17,What is the Robot Pose?,Hand3DFromRobotBase3D is the pose of Hand3D space (TCP/End effector) in RobotBase3D space. Robot: ABB/Kuka/Mitsubishi/Denso/Keba / Cog3DTransformRigid Translation-Cog3DVect3(X,Y,Z) Rotation-Cog3DTransformRotation EulerXYZ, EulerZYX, EulerZYZ Cog3DQuaternion Cog3DMatrix3x3,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,2020/8/17,3D Coordinate Mapping,Map 3D coordinates to 2D Image space,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,2020/8/17,3D Coordinate Mapping,Map 2D coordinates to 3D space,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,2020/8/17,3D Point Triangulation,Triangulation is the process of determining a 3D location of a physical point using two or more cameras calibrated to the same 3D physical space and their 2D projections of the same physical point.,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,2020/8/17,3D Point Triangulation,You must provide accurate projections of the same 3D point to each of the cameras.,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,2020/8/17,3D Model Pose Estimation Overview,3D model pose estimation is the process of using corresponding sets of 2D image points from 3D calibrated cameras to determine the pose of a 3D shape. The 2D points can be obtained using common vision tools like PMAlign, PointPointIntersect, FindEllipse, FindCircle ,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,2020/8/17,3D Model Pose Estimation Concepts,3D pose estimation depends on 3 factors: You must be able to construct (or communicate to the application) an accurate 3D model using a 3D model space. The points you obtain must be accurate projections of 3D points of the model. The 2D points you obtain from the calibrated images must be correctly corresponded to the 3D model points.,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,2020/8/17,Obtaining Accurate 2D Model Points,The most effective technique for obtaining 2D model points for an accurate 3D alignment is to use vision tools. Use CogFindLineTool to locate edges whose lines extend to known coordinate points of intersection. Use CogIntersectLineLine to generate accurate 2D point information.,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,Recommended Parts and Features,In general, rounded features are not the best candidates for accurate 2D points. Rounded edge features in 2D do not consistently correspond to 3D edges as parts rotate.,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,Recommended Parts and Features,Sharp corners and creased edges in 2D correspond more consistently to 3D features as parts rotate in the FOV. Using line intersect tools is recommended for these types of features.,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,Locating Features on Flat Surfaces,Even when locating features on a planar surface, you can employ linear tools to get the most accurate correspondence between 2D and 3D features.,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,Recommended Parts and Features,Circular features on a planar surface offer a distinct advantage due to their geometric properties. A circle can single-handedly be used to represent a 3D plane. As the circle rotates it can communicate 2D and 3D information based on its appearance to each camera.,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,2D Pattern Alignment Tool Considerations,Planar features may sometimes be used to find corresponding 2D features in 3D space. However, 2D pattern alignment tools like PMAlign and SearchMax have limited tolerance for pattern deformation so use with caution as accuracy can be compromised.,Object Pose Estimation,3D Accuracy,Accuracy can be influenced by many factors: Calibration accuracy (robot accuracy, accuracy of calibration plate and ist positions). Camera baseline width. Number of cameras. Accuracy of 2D tools. Accuracy of reference points (e.g. sharp or rounded corners) Image resolution,Summary,VisionPro-3D computes 3D information from 2D and vice versa. can determine the 3D pose of an object. can be used in many robotic applications. can be used with various camera configurations. provides powerful and yet easy to use calibration procedure. consumes input data from familiar 2D vision tools.,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,42,VisionPro3D APPlication,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,43,VisionPro3D APPlication,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,44,VisionPro3D APPlication,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,45,VisionPro3D APPlication,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,46,VisionPro3D APPlication,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,47,VisionPro3D APPlication,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,48,VisionPro3D APPlication,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,49,3D Displacement Sensor DS1100,3D Displacement Sensor - DS1100,Sensor designed to inspect and measure in 3D Height, Volume, Tilt, shape, OCR Calibrated 3D inspection Factory calibrated Real world measurements in mm Contrast independent Inspection Dark objects on dark background Independent of color,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,DS1100 Package,DS1100 hardware and VisionPro software VisionProMax with DS1100 3D Tools Required Accessories IP65 Ethernet Cable IP65 Power & IO Cable Optional Accessories Right angle mounting bracket IP69K Stainless steel enclosure,DS1100,VisionPro,VisionPro,+,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,What is the DS 1100 ?,Sensor designed to inspect and measure in 3D Calibrated 3D inspection Factory calibrated Real world measurements in mm Contrast independent Inspection Low contrast inspection Independent of color,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,How does it work?,Imaging Lens,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,What does the DS 1100 do?,Generates a 16bit 3D range image,0,65,535,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,Range images,The grayscale intensity represents the mm height value,0 0 0 0 0 0 3 5 7 9 11 13 15 15 15 15 15 13 11 9 7 5 3 0 0 0 0 0 0,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,Intensity slices,With each encoder tick as the camera or part moves past the DS 1100 an intensity slice is taken and transformed into the range image Each acquired intensity image of a laser stripe is the basis of a single row in the range image.,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,DS 1100 Hardware,24VDC Power IO + Encoder,Camera,658nm Laser Class 2M,GigE Connector,Status LEDs,IP65 Enclosure,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,Resolution,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,Practical impact of a calibrated system,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat D, Floor 27, No.58, Xin Jin Qiao Road, Pudong, Shanghai ShangHai Ximing-Vision Tech,Accuracy and Profile Rate,2020/8/17,TEL：021-33821033 / 33821031 / 33820483 / 50302319 FAX：021-33821032 Add： Plat