Image data format in Mech-Eye API

[Mech-Mind_Dshuai]

The image data format mainly includes 2D images, depth maps, and point cloud data. 2D images and depth maps are generated and converted into various formats using OpenCV, while point cloud data is generated and formatted using PCL. For point clouds stored in ASCII text formats like .CSV or .XYZ, manual writing is available.

OpenCV

For RGB 2D images:

For depth maps:

1. “Mat” properties

“Mat” is a common way images are represented in OpenCV. Its basic properties include:

1.1 Height

Height is the number of rows, representing the height of the image in terms of pixels.

1.2 Width

Width is the number of columns, representing the width of the image in terms of pixels.

1.3 Channel and type

They specify the number of channels and the data type of the image. For example, 8UC1 indicates an 8-bit single-channel image, 8UC2 indicates an 8-bit, 2-channel image, and so on.

1.4 Data

This attribute holds the address of the actual image data, where the pixel values are stored.

2. Data type

In the example program, the data type represented by “CV_8UC3” is a matrix of 8-bit unsigned (U) integers with 3 channels (C).

Here is the breakdown of the format CV_<bit_depth>(S|U|F)C<number_of_channels>:

2.1 bit_depth

This part indicates the number of bits used to store each value, which defines the bit depth of the image. For most images, the typical depth is 8 bits, representing an 8-bit color depth.

2.2 S|U|F

‘S’ stands for signed int (signed integer).
‘U’ stands for unsigned int (unsigned integer).
‘F’ stands for float (single-precision floating-point).

2.3 C<number_of_channels>

This part specifies the number of channels in an image.
A grayscale image (grayImg) is a single-channel image.
An RGB color image is a three-channel image.
An RGB image with an Alpha channel is a four-channel image.

3. Type conversion

3.1

When converting from a small data range to a larger data range, it is necessary to change the data type first. When converting types, you do not need to consider channels. Below is an example of converting from CV_8U to CV_16U:

mat.convertTo(mat,2); //The type becomes CV_16U
mat = mat * 257; //Multiplying an 8-bit data by 257 converts it to 16-bit

3.2

When converting from a larger data range to a smaller data range, you should first perform the conversion and then change the data type. Let’s take the example of converting from CV_16U to CV_8U:

out = mat / 257;
out.convertTo(out, CV_8U);

3.3

When converting from an integer type to a floating-point type, you should first convert the type and then adjust the data values. Consider the following example:

Mat img = imread("before.png",IMREAD_UNCHANGED);
cout<<img.type()<<endl;
img.convertTo(img, 5); // Convert to 32-bit floating-point
img = img / 65535; // Adjust data to the range [0.0, 1.0]

4. Common formats

Common formats in OpenCV include:

• Windows bitmap file (.bmp, .dib)
• Portable image formats (.pbm, .pgm, .ppm)
• Sun rasters (.sr, .ras)
• JPEG files (.jpeg, .jpg, .jpe)
• JPEG 2000 files (.jp2)
• Portable Network Graphics (.png)
• TIFF files (.tiff, .tif)
• WebP (.webp)
• OpenEXR HDR images (.EXR)

You can modify the file format by changing the file extension, as shown in the screenshot below:

PCL

1. The PointXYZ data type is the most common point data type in PCL. It only contains the three-dimensional XYZ coordinate information. This data type allows for the conversion of point cloud distances from millimeters to meters, as shown in the diagram below:
   

2. The PointXYZRGB data type includes the color information corresponding to each point in the point cloud. This data type also allows for the conversion of point cloud distances from millimeters to meters, as shown in the diagram below:
   

3. If you need to save a point cloud in PCD format, you can follow the structure of the savePLY function. For specific implementation details, refer to the diagram below:
   
   

   
   
   
   
   If you wish to use the savePCD function in your example, you can refer to the diagram below for guidance:
   

4. In PCL, common point cloud storage file formats include:

   • *.ply: This is a widely used point cloud format.
   • *.pcd: PCL’s official specified format for point cloud data.

   Commonly used formats that require conversion include:

   • *.stl: A model file format mainly used in CAD and CAM fields.
   • *.obj: A standard 3D model file format that is well-suited for 3D software model interchange.
   • *.x3d: A file format based on XML and compliant with ISO standards, used to represent 3D computer graphics data.
   • *.las: The industrial standard format for LiDAR data, which is a binary file format.

Point clouds stored in ASCII text formats

These formats can be written manually. Below are sample programs provided for reference:

1. Saved in .csv format:

// Include necessary header files
#include <iostream>
#include <fstream>
#include <pcl/io/pcd_io.h>

// Save the data as .csv format
std::ofstream csvFile("output_cloud.csv");
for (const auto& point : cloud->points) {
    csvFile << point.x << "," << point.y << "," << point.z << "\n";
}
csvFile.close();

2. Saved in .xyz format:

// Include necessary header files
#include <iostream>
#include <fstream>
#include <pcl/io/pcd_io.h>

// Save the data as .csv format
std::ofstream xyzFile("output_cloud.xyz");
for (const auto& point : cloud->points) {
    xyzFile << point.x << " " << point.y << " " << point.z << "\n";
}
xyzFile.close();