Issue with Point Cloud Alignment After Manual Handeye Calibration - NANO Ultra GL Linux API 2.5.4

Q&A
1

Hi I am using the Mecheye Nano ultra-gl connected to a linux computer running ubuntu 20.04. I have installed your SDK v2.5.4.

I am using captureStereo2D(…) with the argument bool isRectified=true (I have also tried false) and using “right” Frame2D as my primary camera image. I am using the result from getCameraIntrinsics(…) to extract the intrinsic parameters of this image. I am then using a set of these images and the intrinsic parameters to run handeye calibration using a separate trusted algorithm. Note that the MechEye camera is mounted on a robot, the robot calibration is also trusted.

The problem is that when I then scan an object from varying orientations using capture3DWithNormal(), the point clouds do not seem to align.
I have been assuming that the Frame3D returned from capture3DWithNormal() is in the primary camera frame and that the primary camera is the “right” Frame2D overwritten in captureStereo2D(…). Is one of those not the case? And if so, what is the convention?

2 Likes
2

The coordinate system of the Frame3D point cloud = the left camera optical center coordinate system.

The “right” Frame2D from captureStereo2D(…, isRectified, …) ≠ the point cloud coordinate system.

Please use the 2D image and intrinsic parameters of the left camera for hand-eye calibration to ensure proper point cloud alignment.

If you need to further confirm the specific numbering of the left/right cameras or the method for extracting intrinsic parameters, please refer to the SDK documentation or contact Mech-Mind technical support.