FAQ: Using Mech-Eye API

[Mech-Mind_Dshuai]

Errors

• Dependency error from Mech-Eye SDK’s Python sample

• Cannot find or load dynamic link library (DLL file)

  1. Check if VC runtime is installed on your computer or try using the DirectX repair tool.
  2. Confirm that the file paths in the environment variables are correct, and restart the application to apply the new environment variable settings.
  3. Alternatively, place the required DLL files or debug versions of DLL files into the folder where the compiled application (exe) is located.

• Errors due to calling the wrong dynamic link library (DLL)

  1. If errors occur due to calling an outdated version of the DLL file, prevent this by uninstalling the old version of Mech-Eye SDK before installing the new one.
  2. Errors may result from mixing release and debug versions of DLL files; ensure proper solution configuration choices during generation.

• Camera with RGB image output encounters an error during 2D image acquisition using Halcon

  • If you encounter this issue, first check for error messages in Mech-Eye Viewer. If you find an error message like “Load 2D Camera Init Configs Failure: The configs file /home/ubuntu/Server/2DCameraInitConfig.json may be invalid as an empty JSON object. Please edit the configs file!” please contact technical support for assistance in resolving the issue.

• Data packet transmission too small (MTU=1500) when using GigE or GenICam

  • Set the camera’s MTU value to 9000, enable jumbo frames on the computer, and ensure that if using a switch or router, it supports jumbo frames. For detailed instructions, see here.

• Common issues with CMake

  1. Missing files: When downloading Mech-Eye API sample code from GitHub, ensure compatibility with the installed Mech-Eye SDK version.
  2. Low versions of OpenCV or PCL: Upgrade to higher versions and rerun CMake (recommended versions are OpenCV 3.4.5 and PCL 1.8.1 or above).
  3. Missing compiler: When installing Visual Studio, ensure that “Desktop development with C++” and “Universal Windows Platform development” components are selected. After installing these components, restart the computer.

Calibration

• FAQ for hand-eye calibration in Mech-Eye API & Halcon (Mech-Eye Viewer 2.1.0)

Data aquisition

• Image data format in Mech-Eye API

• Depth map to point cloud conversion sample (C++) (with diagrams and formulas)

• Mech-Eye API Dependencies (Third Party Libraries) and Versions

• Mapping color image pixels to depth map point cloud coordinates with a camera featuring separate color lenses (such as LSR L)

• How to Convert ColorMap to C# Bitmap Format and Access Each Value Column-wise with ColorMap.Data(), Avoiding ColorMap.At()

  • You can use the following method to convert ColorMap to Bitmap format and save it (please add the camera connection and disconnection code as needed). The method utilizes the IntPtr Data() function. In Mech-Eye API, not only does it provide IntPtr Data() but also Resize(uint width, uint height) methods, allowing for convenient access based on row and column indices.
  • For details, see How to Convert ColorMap to C# Bitmap

• Mapping RGB images to depth maps for external cameras like LSR and DEEP for feature processing and deep learning

  • For cameras of this type, the Mech-Eye API provides rigid transformation data for the depth camera matrix, texture camera matrix, and depth-to-texture conversion. By establishing the corresponding relationships, you can convert the color images generated by the texture camera to the coordinate system of the depth camera.
    For detailed implementation, see Mapping color image pixels to depth map point cloud coordinates with a camera featuring separate color lenses (such as LSR L)

• How to save point clouds in millimeters instead of meters in Mech-Eye API

  • The data in Mech-Eye API is actually in millimeters by default. It’s just that during output and saving, it’s converted to meters using PCL. Therefore, you can modify it in PCL to save the point cloud in millimeters.

• How to obtain a point cloud with normals in Halcon

  • In Halcon, the surface_normals_object_model_3d operator can be used to compute the normals of a point cloud. For detailed instructions, see Calculating normal vectors using Halcon.

• Obtaining the XY corresponding to Z in Halcon using the obtain_depth_map example

  • In the example, ‘ChunkScan3dCoordinateSelector’ selects ‘CoordinateC’ corresponding to Z, while selecting ‘CoordinateA’ and ‘CoordinateB’ corresponds to XY. By modifying the selection, you can obtain the corresponding data.

• Retrieving camera intrinsic parameters and converting from depth map to point cloud

  1. Retrieve intrinsic parameters: Obtain intrinsic parameters through the GetCameraIntri example.
  2. Depth map to point cloud conversion is demonstrated in the CaptureCloudFromDepth sample. For detailed instructions, see Depth map to point cloud conversion sample (C++) (with diagrams and formulas).

• Are point clouds saved by Mech-Eye SDK ordered?

  1. Point clouds saved through Mech-Eye API are ordered point clouds.
  2. Point clouds saved through Mech-Eye Viewer are unordered for faster rendering and smaller memory footprint.

Communication & Connection

• How to connect to a camera via its IP address using Mech-Eye API

• Possible Solutions When Encountering Connection Issues with the ‘connect’ Interface

• Mech-Eye API and Mech-Eye Viewer Cannot Find or Connect to the Camera

  • If the camera is not detected using Mech-Eye API, you can first try searching with Mech-Eye Viewer. If it still cannot be found, you can follow the steps here.

• Mech-Eye API and Mech-Eye Viewer Cannot Find or Connect the Camera

  1. If the camera cannot be found using Mech-Eye API, you can first try searching with Mech-Eye Viewer. If it still cannot be found, you can follow the steps in Troubleshooting guidance for camera connection problems.

• Only Mech-Eye API Cannot Find or Connect the Camera

  1. Get the camera’s IP address and connect directly to the camera IP. For details, see How to connect to a camera via its IP address using Mech-Eye API.
  2. Mech-Eye API cannot find the camera: Disable other network interfaces or directly connect the industrial computer to the camera. Try upgrading the network card driver, check and close other network utilities such as virtual machines or VPN if possible, or change the network connection.
  3. VMWARE cannot find the camera: Run the SDK in a virtual machine, select virtual machine settings in the virtual machine interface, change the virtual machine network connection to bridge mode, copy the physical network connection status, and run again.
  4. Linux system cannot find the camera: If the industrial computer and the camera have different IP segments, ensure that they are in the same segment. This is because Linux devices may have difficulty receiving broadcast packets across segments: rp_filter, which rejects packets with IP addresses that do not match those on a certain network card to prevent IP spoofing. Filtering camera broadcasts causes issues, and disabling this configuration can resolve the problem.

• Inconsistent IP display in Mech-Eye API and Mech-Eye Viewer

  1. If the IP displays are inconsistent and cannot connect, you can modify the camera IP in Mech-Eye Viewer to match the IP displayed in Mech-Eye API.
  2. If you want to change it to a specific IP, modify the IP in Mech-Eye Viewer, restart the camera, and then use Mech-Eye API to search again.

• How to change the camera IP address using Mech-Eye API

  • Mech-Eye API offers an IP configuration tool, supporting the modification of camera IP configurations in Ubuntu systems. For specific instructions, please refer to the IP Configuration Tool user manual.

Image Capture Time

1. The image capture time is influenced by camera parameter settings, so it is essential to ensure reasonable parameter configurations during testing.
2. Ensure that the camera is operating in a Gigabit Ethernet or higher-speed network environment; otherwise, transmission time may be affected. You can check the network speed in the top-left corner of Mech-Eye Viewer, and stable speed should be above 700 Mbps (For Nano and ProXS, the speed should be above 300 Mbps).
3. Taking Mech-Eye API in C++ as an example, the time for capturing the depth map is calculated using “device.captureDepthMap(depth);”, the time for capturing 2D images is calculated using “device.captureColorMap(color);”, and the time for capturing point cloud images is calculated using “device.capturePointXYZMap(pointXYZMap);”. It is recommended to capture continuously fifteen times and take the average of the last ten as the stable image capture time (with a suggested interval of 3 seconds).
   For more details, see Calculate imaging time in different applications.