Fusion of deepmap and 2D image

Hello,
I have a mecheye deep camera. My project is to find boxes on a pallet at different heights. Using the API and OpenCV I’m trying to overlay the deepmap and the 2D image, but I’m having some problems.

My first doubt is if I have to calibrate the cameras to be able to join both images. I suppose that the deepmap will not be necessary, but I have doubts with the 2D camera.

I have also tried to use the function ‘CameraIntrinsics()’ that returns the translation and rotation between both images, but I have not got any optimal result.
Could you help me with my problem?
Best regards and thanks

Hello, is the camera model you’re using DEEP V4? You can refer to this document: https://community.mech-mind.com/t/topic/1061

Hi, thanks for your reply. I have tried with that example but I want to get a 2D image at a certain distance.
Something like that but with OpenCV
https://docs.mech-mind.net/en/suite-tutorial/1.7.4/single-case-cartons/single-case-cartons-vision.html

Does this mask-related example help you: Mapping2DImageToDepthMap

Hello,
That’s almost what I want to do, but I don’t want to get a point cloud, I want to keep only the parts of the 2D image at a certain distance from the camera.

Perhaps you could try using the CaptureStereo2DImages example directly to obtain the 2D images needed for generating depth maps. This way, they can directly correspond to the depth maps.

Hello,

Thanks for your help

Using the Mapping2DImageToDepthMap example I have managed to project the point cloud to a 2D image that I can use to analyse the scene.

The problem I have now is that I don’t know how to change my program so that I don’t have to save the point cloud in a .ply file.

I am trying to go from the point cloud that the camera gives me with the following format:

points_xyz_bgr = TexturedPointCloud()

To a point cloud in open3d:

point_cloud = o3d.geometry.PointCloud

Right now my code works by saving the point cloud and then reading it with the following code, but it is very slow.

point_cloud = o3d.io.read_point_cloud(‘TexturedPointCloud.ply’)

To transfer point cloud data directly from the TexturedPointCloud format to an Open3D point cloud without saving a .ply file.
This direct data transfer can greatly improve efficiency by removing the need to save and reload the point cloud data from disk.

I have tried to perform this data transfer but I am unable to do so. Can you help me?

You need to obtain data from the TexturedPointCloud class according to your requirements. You can refer to the documentation for this class.

image993×615 65.9 KB