2022 RSDP Mid-term project Disparity patch for Dense stereo matching

I introduce a module called DisPatch designed to enable Dense 3D reconstruction with only two stereo images. It simply operates with the following two functions.

• Keypoint 3D reconstruction: It is a function that matches the keypoints of stereo images found by SIFT and converts them into 3D coordinate systems.

• Local Patch 3D reconstruction: This function extracts very small patches around the Keypoint of the two stereo images and then performs stereo matching partially on these patches. Using this, dense stereo matching with improved accuracy could be performed.

Ubuntu 20.04
Python 3.8.10
OpenCV 4.5.5
Open3d 0.13.0 

If you download the file (midterm_3Dreconstruction), you can see that there are three folders inside.

cd reconstruction
python feature_match.py

1. Camera calibration
2. Stereo image undistortion
3. Keypoint 3d reconstruction
4. Local Patch 3D reconstruction

After executing in order, you can even check the process of floating a point cloud on a 3D space with Open3D.

• calib: Folder containing 15 checkerboard calibration images
• stereo: Folders containing stereo images
• reconstruction: Folder where calibration.py and feature_matching.py exist

Get the checkerboard image in ./calib and perform calibration. The method used at this time is Zhengyou Zhang's method.

OpenCV's undistort function was used.

After searching only key points with SIFT and KnnMatcher, 3D reconstruction is performed.

Apply the Dispatch algorithm designed for Dense Matching.