Could you give some examples?
When the point cloud of the workpiece is of poor quality in a project, it is generally necessary to use a combination of deep learning and 3D point cloud processing for workpiece recognition, localization, and grasping. Here are a few examples:
In cases where crucial features of the workpiece are absent from the point cloud data
The red boxes in the image represent the front view of workpieces A and B, while the blue box shows the reverse side of the A and B frames. The arrows in the image point to the main feature regions that distinguish the two types of workpieces.
As can be observed in the screenshot of the point cloud below, the features that differentiate between the types of workpieces cannot be obtained from the point cloud.
The project benefits from high-quality 2D images, making it possible to employ deep learning directly for segmentation and differentiation of workpiece types.
The type of deep learning utilized in this project is instance segmentation. By assigning appropriate classification labels to various types of workpieces, the instance segmentation process extracts individual workpiece masks and simultaneously provides the corresponding category label for each identified workpiece.
The workpiece’s point cloud data exhibits significant gaps and missing portions. During 3D point cloud matching, mismatches can occur, leading to substantial deviations in grasping pose.
The reflective workpieces in the image are closely packed and numerous. However, the edges and shape features of the workpieces’ can be clearly discerned.
Due to the reflective nature of the workpieces, point cloud data may exhibit gaps, with the primary gaps occurring along the axial direction of the workpieces.
In this project, an instance segmentation model is employed to recognize the workpieces, generating corresponding masks. Subsequently, the point cloud corresponding to each mask is extracted.
In projects of this type, conventional methods for recognition and grasping are ineffective. Instead, it’s necessary to employ deep learning to generate masks and subsequently compute poses for point cloud generation. Note: High-quality 2D images are essential.
The waveform sleeves often consist of reflective components and are tightly packed within the material container.
Upon examining the point cloud image, it is evident that the point cloud for this particular workpiece is highly unstable. There is a significant possibility of encountering instances where the workpiece has absolutely no points in the point cloud.
Despite the workpieces being reflective in nature, the edge information of the workpieces is clear in the 2D images. Therefore, utilizing deep learning for workpiece recognition can achieve stable and reliable results in this project.