The camera R&D team has decided to cancel the factory extrinsic parameter calibration for all cameras, considering the following:
Even after the improvement in intrinsic parameter calibration, the offset calibration for cameras (mainly PRO M 2000 V4, LSR L 3000 V4) that still undergo extrinsic parameter (offset) calibration does not have significantly enhanced absolute accuracy.
Furthermore, the use of extrinsic parameters (offset) tied to the robotic arm cannot address the issues faced by the robotic arm on-site, rendering the factory-calibrated parameters ineffective.
FAQ：
-
The original intention behind offset:
- In the early days, the company utilized third-party cameras with relatively poor accuracy (manifested as dimensional errors when measuring a standard object with the camera, such as a 1-meter pole being measured as 1.1 meters). The hand-eye system, composed of the camera and the robot, allowed the robot’s end effector position to serve as ground truth to compensate for the camera’s point cloud deviation. This compensation matrix became known as “offset.”
- The following diagram illustrates this concept:
- The right side is a horizontal cross-section of the left side.
- 1: the camera
- 2: robot space
- 3: point cloud space (with distortion)
- 4: Grid the space, and each grid corresponds to a transformation matrix between camera space and robot space. A series of these transformation matrices is what the offset in the extrinsic parameters represents now.
-
Where is the offset recorded?
- In the extrinsic parameter file (extri_param.json): the depthToCloud_offset field.
- In the extrinsic parameter file (extri_param.json): the depthToCloud_offset field.
-
Where does the offset function?
- It functions during the process of converting the depth image to a point cloud, transforming the point cloud generated directly from the camera into a grid. In simple terms, it deforms the camera’s point cloud to fit the robot’s spatial requirements.
- In some cases, noticeable deformation of the point cloud inside the Mech-Vision project occurs. This happens when the Offset file is contaminated. A typical operation leading to contamination is: when there is a significant error in extrinsic parameter calibration, forcefully recalculating compensation parameters, reducing the apparent extrinsic parameter error.
-
What can Offset be used for?
- Its original purpose is to compensate for the deviations occurring when the camera measures objects. We utilize robot spatial coordinates as ground truth (refer to the diagram below). When there are deformations in the robot’s spatial coordinates, it conveniently compensates for the errors in the hand-eye system composed of the camera and the robot. This is a key reason why it has continued to exist.
- In the image:
Purple: point cloud space
Red: theoretical value
Green: robot
- Taking a cross-section, indicating the error: When the robot precision is perfect, the offset compensates for error E1. When the robot precision is not good, the offset compensates for error E2, which is related to the robot’s joint posture at the time of sampling.
-
Please note: The original purpose of offsetting is to compensate for the camera; its compensation for robot errors is limited in degrees of freedom. The compensation matrix of offsetting for robot errors is local and depends on the robot’s pose at the time of compensation calculation. If the robot is commanded to pick with joint positions significantly different from those during calibration, errors in the results are normal.
-
After thorough analysis: In real-world projects, robot absolute accuracy often poses challenges. Offset can compensate well for ETH (eye to hand) scenarios, but its compensation for EIH (eye in hand) scenarios is not optimal.
In EIH (eye in hand) scenarios, robot errors can cause deviations in the image-capturing positions, which cannot be solved by offset. Phenomenon: Different sampling paths result in different EIH extrinsic parameters. -
What is factory offset?
- In the early days, the company’s cameras had low accuracy, but there was a desire from the business side to have higher accuracy when the cameras left the factory. Thus, the idea of “performing offset calibration when the camera leaves the factory” was proposed. As a result, each camera’s intrinsic parameters came with an accompanying factory offset parameter file.
- However, due to the lack of high-precision measurement equipment, the factory utilized a robotic arm as the calibration device. Hence, there has been the statement: “The use of extrinsic parameters (offset) tied to the robotic arm cannot address the issues faced by the robotic arm on-site.”
-
Why is factory offset no longer necessary?
- Reasons:
- The accuracy of V4 cameras has significantly improved, making it inappropriate to use a regular robotic arm as the calibration reference.
- Through experimental verification, there are cases where factory offset can lead to reverse compensation.
- For high-accyracy projects, the “teaching by pick points” method is still needed to ensure accuracy.
- In the context of EIH scenarios, Offset fundamentally cannot effectively compensate for system errors. Moreover, the camera’s working distance is relatively short, and camera accuracy is not the main factor contributing to system errors.
- In ETH scenarios, if the robot’s accuracy on-site is not satisfactory, factory Offset also holds no value. Recalibration of offset on-site is still necessary.
- Reasons:
-
What is the impact of removing factory offset on the site?
- There is no impact; it means that the cameras shipped subsequently will no longer contain built-in compensation parameters for extrinsic parameter use.
- For older cameras, the existing mechanism still persists (further instructions will be provided once this mechanism is removed in future versions).
-
Is it necessary to retain offset?
- The current conclusion is: distortion compensation still has practical use cases. In future versions, it might exist in other forms.
- Only in ETH scenarios, when the extrinsic parameter calibration error exceeds the tolerance error for picking (for example, actual picking accuracy is 4 mm, but the extrinsic parameter calibration result shows
100% < 6 mm), it is recommended to increase the number of image capturing positions to cover the entire picking area and [recalculate compensation parameters].
- For other scenarios, it is not recommended to ecalculate compensation parameters.
- Only in ETH scenarios, when the extrinsic parameter calibration error exceeds the tolerance error for picking (for example, actual picking accuracy is 4 mm, but the extrinsic parameter calibration result shows
- The current conclusion is: distortion compensation still has practical use cases. In future versions, it might exist in other forms.
-
What to do if the extrinsic parameter error is consistently large in scenarios of picking scattered workpieces?
-
- Check the camera’s intrinsic parameters.
-
- Ensure that the camera and robot are properly warmed up before calibration.
-
- Verify the robot’s zero point.
-
- Improve the absolute positional accuracy of the robotic arm (calibrate or replace the robotic arm).
-
Other
-
**Note：Regularly inspect and record changes in camera precision.
-
How to restore factory offset after on-site offset is damaged?
- In the future, there will be only two scenarios:
- a. Not using offset.
- b. Using the offset file saved in the Mech-Vision project (for running projects or cameras where offset was recalculated on-site).
- There will be no need to restore factory offset for scenario a. In this case, it is necessary to inspect the camera’s intrinsic parameters to ensure they meet the factory standards ([On-site] Guidebook and data record form for checking camera’s intrinsic parameters). If they do not meet the standards, recalibration of intrinsic parameters is required.
- In the future, there will be only two scenarios:
-
How to reset factory offset to identity matrix or empty?
- Mech-Vision 1.8.0 will default to not reading the camera’s built-in offset file, regardless of whether the camera has factory calibrated offset. There are two cases where compensation is applied:
- a. Offset file already exists in the Mech-Vision project (for ongoing projects).
- b. On-site recalculation of offset (for new projects in the future).
- There is generally no need for special treatment for current on-site cameras. Current issues can be resolved by replacing with the identity matrix.
- Mech-Vision 1.8.0 will default to not reading the camera’s built-in offset file, regardless of whether the camera has factory calibrated offset. There are two cases where compensation is applied:
-
Note:
- “Offset” ≠ “Intrinsic parameters”;
- “Factory Offset” is not entirely equivalent to “Offset”;
- Offset is planned to be renamed as: hand-eye system compensation parameter.