Assistance Needed: Integration of Mech-Vision with Robotic Arm for Pick-and-Place Application

Q&A
1

Hello everyone… :innocent:

I am currently working on a project where I am trying to integrate Mech-Vision with a robotic arm to automate a pick-and-place task. The setup involves Mech-Eye 3D cameras, Mech-Vision for object detection and pose estimation, and Mech-Viz for motion planning. My end goal is to reliably pick up small objects (about 5-10 cm in size) from a conveyor belt and place them into designated bins based on color and shape.

Here’s where I am encountering challenges:

  1. Pose Accuracy Issues: While the camera detects the objects, the robotic arm sometimes misses the exact grasping position. The pose data in Mech-Vision seems accurate, but something may be going wrong during the handover to Mech-Viz. Could it be a calibration mismatch?
  2. Dynamic Objects on Conveyor: Objects move at a moderate speed on the conveyor belt. I’ve set up a trigger to capture images at specific intervals, but it doesn’t always align with the robot’s action. Is there a way to improve synchronization between the vision system and the conveyor?
  3. Error Handling: Occasionally, the robot fails to pick an object due to overlap or misdetection. I want to add a retry mechanism or a fallback workflow in Mech-Viz, but I’m not sure where to start.

I also check this: https://community.mech-mind.com/t/topicalteryx But I have not found any solution.

Thank you in advance for your help!

Respected community member!

2

image

I understand your system are roughly as shown in the diagram. It’s truly impressive that you’ve managed to use a structured-light camera for dynamic object picking on a conveyor belt.

Mech-Eye and Mech-Vision does not have hardware synchronization. If you want to improve the success rate of picking on the conveyor belt, here are my personal suggestions:

  1. Ensure the conveyor belt speed is stable and as slow as possible to optimize the point cloud quality.
  2. Considering the differences in computation time for each object，Your software should implement more precise clock synchronization. For instance, record the time difference between sending the capture trigger command and receiving the result. Use this dynamic offset to adjust for the robot, rather than relying on a fixed offset.
  3. The robot itself theoretically needs to have conveyor belt tracking functionality, but if its movements are fast enough, this may not be necessary.
  4. For objects that are failed picked, it is recommended to enable the conveyor belt itself with a recycling capability.
3

One more thing ：Mech-Viz does not have these capabilities, and it is not recommended to use Mech-Viz to directly control the robot.