Deep learning can't effectively recognize sacks

Software versions
Mech-Vision 1.7.2
Mech-Viz: 1.7.2
Mech-Center: 1.7.2
Mech-Eye: 2.1.0

Robot Model
YASKAWA GP180
Camera working distance: 1.8m to 4.1m
On-site environment image:

Open environment without direct sunlight
Camera model:
Deep V4
QAA30235A4030074

ROI:
image

(1) False recognition

image

(2) Recognition omittance

The highest-layer sacks were not recognized.

Low scores:

Explanations on the problem:

In many cases, even after taking multiple photos at that time, recognition was not possible. However, when tested with images, the sacks could be recognized normally, making it difficult to identify abnormal images.

Measures taken:

False recognition: Increasing exposure or reducing exposure did not solve the problem of false recognition.

Recognition omittance: Adjusting the region of interest (ROI) in deep learning sometimes results in successful recognition, while other times it does not. Extracting the highest-layer point cloud, projecting it into a 2D image, and then applying deep learning still leads to cases of missed recognition.

What I ask for:

  1. How to prevent false recognition from occurring.
  2. How to set the deep learning ROI for stable recognition.

How to prevent false recognition:

  1. Try to keep the on-site lighting conditions stable, avoiding overexposure, underexposure, and shadows. These conditions can cause unclear edges of the sacks in the images, affecting the stability of the model’s recognition.
    • Additionally, you can increase the brightness of the images, which can make the edges of the sacks clearer and be beneficial for the deep learning model’s recognition.
  2. Adjust the camera’s white balance.
  3. Do not place unnecessary sacks next to the stack location, as this may lead to misrecognizing the nearby sacks first.
  4. Adjust the ROI with the appropriate stack location.

ROI setting for deep learning:

We suggest setting the ROI for this project based on the on-site conditions with fully stacked sacks, specifically considering the shape of the top layer.

  1. The ROI setting for a general supermodel is based on the largest stack of that type. You can set an ROI slightly larger than the full-layer stack.
  2. If after the above operation, the stability of the model’s recognition is still insufficient, then it is necessary to fine-tune the model for this type of sack. Fine-tuning the model using data from this sack can help improve recognition stability.

See the docs for details on model finetuning.

Note: Remember to use the most up-to-date deep learning model from the download center