Mechanisms of 3D cameras
Visual sensors relate to machines in almost the same way that eyes relate to us humans: they obtain visual information for machines or us humans.
By imitating humans’ eyes, machine vision is progressing from 2D to 3D, surpassing human vision in certain aspects.
Nowadays, state-of-the-art technologies of 2D optical measurement and computer vision provide integrated intelligent sensing systems for various applications. With one more dimension, advanced 3D optical measurement and vision technology now have a profound impact on science and industry, including artificial robots.
Non-contact optical 3D measurement
Optical 3D measurement is non-contact, making it suitable for remote sensing and other non-contact scenes. Based on mechanisms, there are 3 main types of 3D measurement: time-of-flight measurements, structured light measurements, and passive binocular measurement. Their classification is as followed:
- Time of flight (ToF)
- Pulsed modulation ToF
- Continuous wave modulation ToF
- Triangulation
- Passive 3D measurement: Passive binocular measurement
- Active 3D measurement
- Line structured light
- Laser triangulation
- Surface structured light
- Spatial encoding
- Sequential encoding
- Line structured light
Click the link above to see details of the corresponding technology.
Time of flight (ToF)
Left: ToF image sensor.
Right: VCSEL.
Stereovision (passive binocular stereovision)
Left: CMOS image sensor.
Structured light (Mech-Mind)
Left: CMOS image sensor.
Right: DLP projector.
A comparison of different technologies
| Camera types | iTOF area array | Passive binocular | Speckle binocular | Structured light (sequential encoding) |
|---|---|---|---|---|
| Measuring approach | Active | Passive | Active | Active |
| Mechanisms | Distance = light velocity * time | Match feature points and calculate by triangulation indirectly. | Project known encoding patterns and consequential distortions are related to distances and directions. | Projecting known encoding patterns makes better matching features. |
| Measuring accuracy | Centimeters | Millimeters | Measurement within a short distance: 0.01 mm–1 mm | Measurement within a short distance: 0.01 mm–1 mm |
| Measurement range | Within 10 m | Within 5 m | Within 5 m | Within 5 m |
| Resolution | 0.3 M (640 × 480) | 0.3 M–12 M | 0.3 M–12 M | 0.3 M–12 M |
| Frame rate | High Up to about 100 fps |
Medium 30 fps @ 0.3 Mp |
Medium 30 fps @ 0.3 Mp |
Flexible 0.1–10 fps @ 0.3–12 Mp |
| Software complexity | Low | High (with complex matching algorithms) | High (recognizing image patterns) | High (recognizing image patterns) |
| Hardware complexity | Medium (with integrated modules) | Low | Medium | High (requiring dynamic projection) |
| Power dissipation | Medium (requiring repeated sampling) | Low (projection is needless) | Medium | Medium |
| Influencing factors | Immune to light conditions and workpieces’ textures. Susceptible to the multipath effect. |
Susceptible to light conditions and workpieces’ textures. Cannot be used at night. |
Susceptible to light conditions, workpieces’ textures, and reflections. | Immune to light conditions and workpieces’ textures. |
| Running outdoors | Depending on the power source | OK | With preconditions | Depending on projection patterns |
| Advantage(s) | Long working distance. High accuracy at long working distances. |
Simple hardware system. Cost-effective. Indoors and outdoors. |
Real-time processing. Dynamic modeling. |
High accuracy. |
| Disadvantage(s) | Low accuracy at short working distances. Low lateral resolution. |
Susceptible to ambient light. Unsuitable for texture-less scenes. Long matching time. |
Lower accuracy than sequential encoding. Long matching time. |
Require multiple encoding patterns for projection. |
| Applicable scenarios | Self-driving cars Car range finders |
Vehicles’ crude navigation in controlled or complicated environments | Face and gesture recognition | Industry and robot SLAM (simultaneous localization and mapping) |
Structured light—the pros and cons of monocular and binocular measurement when applying sequential encoding
- Monocular measurement
- Pros
- Strong model assumptions improve measurement accuracy.
- Fewer blind spots, as the camera’s viewing pyramid aligns with that of the projector.
- Cons
- Distortion occurs if the data is inconsistent with the model. For instance, the presence of multipath effects can result in visible interference, even when the obtrusive light is only 1% as strong as the signal light.
- Pros
- Binocular measurement
- Pros
- Binocular cameras perform better when dealing with interference (such as noise or systematic errors like multipath effects) and coding errors.
- Cons
- Binocular cameras present significant blind spot issues, as they require three viewing pyramids to overlap (two cameras and the laser projection).
- The matching process can be time-consuming, although regular coding can expedite this process. However, the more the coding relies on regularity, the less resistant it becomes to interference.
- Pros
Advantages of Mech-Eye Industrial 3D Cameras
- Exclusive projection coding technology: Cameras work well with reflections, multipath interference, and flickers
- Reliability and certification
- Unique optical design and calibration algorithms: High absolute accuracy and repeatability
- Cameras work well in ambient light
Exclusive projection coding technology
Cameras work well with reflections
The point cloud is clear and complete even for finished highly reflective workpieces:
Grayscale image:
Point cloud:
Cameras work well with multipath interference
Exclusive projection coding technology minimizes the multipath effect from metal workpieces, making point clouds accurate and complete:
Metal workpieces:
In this depth map generated by the traditional approach, inside red rectangles are defective parts:
In this depth map generated by a Mech-Mind’s exclusive approach, the screenshot of point clouds and the related data are both accurate:
A comparison of point cloud data (black parts are reflective)—below are metal workpieces and their depth maps:
| Metal workpieces | Mech-Eye Pro | Zivid Two | Photoneo | Keyence RB-1200 |
|---|
Metal curved surfaces captured by Mech-Eye Industrial 3D Cameras are real and complete. In the picture below, from left to right are:
- Metal workpieces: Steel bars.
- Mech-Eye PRO:
- Point clouds are complete (up).
- Steel bars’ curves are accurate (down).
- Zivid Two:
- Defective point clouds (up).
- Defective steel bars’ curves (down).
- Photoneo:
- Point clouds between steel bars are defective (up).
- Steel bars’ curves are roughly rebuilt yet tops are flat and flying spots in the flanks are too many (down).
- Keyence RB-1200: Point clouds are highly defective and details are not acceptable.
Cameras work well with flickers
For sequentially encoded structured light 3D cameras, ambient light interferes with capturing. In certain frequencies, flickers from ambient light interfere with structured light stripes, causing periodic interference of the point cloud.
With the Anti-Flicker Mode, we can optimize screenshots of point clouds interfered with by flickers in frequencies like 50 Hz or 60 Hz.
Images below show a Mech-Eye Pro’s capturing with Anti-Flicker Mode off (left) and on (right):
Reliability and certification
- Fully enclosed aluminum alloy shell. IP Rating: IP65.
- Standards are affirmed by CE, FCC, VCCI, and RoHS.
- Some series have a MTBF (mean time between failures) of 40,000 hours.
Unique optical design and calibration algorithms
High accuracy
Absolute accuracy: Measured by a ceramic stick on standards of VDI VDE 2634 Part-II and ISO 10360-13-2021.
| Margin of error’s median | Max margin of error (±2σ, <= 95%) | |
|---|---|---|
| Mech-Eye PRO | 0.15% | 0.25% |
| Zivid Two | 0.18% | > 0.4% |
| Photoneo | 0.15% | > 0.3% |
Margin of error = measurement error / the distance between the camera and the stick
σ: Measurement errors’ standard deviation
Repeatability:
| Point repeatability | |
|---|---|
| Mech-Eye PRO | < 0.05 mm @ 1 m |
| Zivid Two | 0.2 mm @ 1 m |
| Photoneo | 0.1–0.2 mm @ 1 m |
Cameras work well in ambient light
When the distance between the camera and a common workpiece is 1 m, with ambient light of more than 80,000 lux, PRO S V4 cameras can bring high-quality point cloud screenshots. Compared with cameras of other brands, PRO S V4 series are more capable of capturing clear images in ambient light.
Left: Captured in strong ambient light (> 80,000 lux).
Middle: The grayscale image captured by Mech-Eye PRO S V4.
Right: The depth map captured by Mech-Eye PRO S V4.
