Speckle-Scanner/05_disparity/zncc/Readme.md

Stereo Disparity Map and Point Cloud Generation

This project implements a dense stereo disparity map and point cloud generation pipeline using spatio-temporal Zero-Normalized Cross-Correlation (ZNCC) with subpixel interpolation methods and other advanced features.

Features

• Dense Disparity Map Calculation: Generates Disparity and vertical shift maps.
• Subpixel Interpolation: Supports parabolic, Gaussian, and equiangular interpolation methods for subpixel accuracy.
• Correlation Map: Computes ZNCC correlation for each pixel.
• Point Cloud Generation: Reprojects disparity maps into 3D space using a Q matrix.
• Color Mapping: Normalizes depth values and applies a colormap for visualization.
• Optional Median Filtering: Smoothens the disparity map.
• Visualization: Saves disparity maps, vertical shift maps, and correlation maps with colorbars.

Requirements

• Python 3.9 or later

# This step only
pip install -r ~/Speckle-Scanner/05_disparity/zncc/requirements.txt

# Or install everything for the full pipeline
pip install -r ~/Speckle-Scanner/requirements.txt

# Optional GPU acceleration (pick one matching your CUDA version)
pip install cupy-cuda12x   # CUDA 12.x
pip install cupy-cuda11x   # CUDA 11.2–11.8

Packages: numpy, opencv-python, matplotlib, scipy, numba, open3d. Without cupy, the pipeline falls back to CPU (numba).

Input Requirements

• Stereo Rectified Images:
  • Images must be stored in .png or .bmp format.
  • Pipeline pairing (in 02_rect_images/): lc_ts1634840093_ck….png is matched with rc_ts1634840093_ck….png on the shared ts token; the ck… suffix is ignored.
  • Pairs are sorted by timestamp; --num_images defaults to all pairs; if limited, the last N (highest timestamps) are used.
  • Standalone runs may still use separate left/ and right/ folders with any naming; pipeline mode uses one folder with lc_ / rc_ prefixes.
  • PNG is preferred when both PNG and BMP pairs exist.
• Q Matrix File:
  • A .cvstore file containing the Q matrix for 3D reprojection.

Outputs

• Disparity Map: Saved as Disparity.npy.
• Vertical Shift Map: Saved as vertical_shift_map.npy.
• Correlation Map: Saved as correlation_map.npy.
• Disparity and Shift Maps with Colorbars:
  • Disparity_map_colorbar.png
  • vertical_shift_map_colorbar.png
  • correlation_map_colorbar.png
• Point Cloud:
  • Saved as Point_cloud.ply in PLY format with depth-based color information.

Usage

1. Clone the repository and navigate to the project directory.
2. Prepare your stereo images and paste them in data folder Example directory structure:

project_directory/ 

  .data/
    ├── left/
    │   ├── 0.png
    │   ├── 1.png
    │   └── ...
    └── right/
        ├── 0.png
        ├── 1.png
        └── ...

Or

  .data/
    ├── left/
    │   ├── lc00000.bmp
    │   └── lc00001.bmp
    └── right/
        ├── rc00000.bmp
        └── rc00001.bmp

3. The Q matrix must be provided in a YAML file (e.g., Q.cvstore). Format example:

  %YAML:1.0 
  --- 
  Q: !!opencv-matrix 
     rows: 4 
     cols: 4 
     dt: f 
     data: [ 1.0, 0.0, 0.0, -5.5863211059570312e+02, 
             0.0, 1.0, 0.0, -7.3320780181884766e+02, 
             0.0, 0.0, 0.0, 3.3665836719173171e+03, 
             0.0, 0.0, 1.4262549294100269e+00, 1.0682440279589944e+03 ] 

This matrix is essential for converting disparity maps into a 3D point cloud. 3. Run the script with desired parameters (see below).

Basic Command

# stereo_disparity_main.py auto-selects GPU (CUDA) or CPU.

cd ~/Speckle-Scanner/05_disparity/zncc

# Default parameters (saves disparity map only)
python stereo_disparity_main.py \
  --left_dir   <path_to_rect_images> \
  --right_dir  <path_to_rect_images> \
  --left_prefix  lc_ \
  --right_prefix rc_ \
  --q_file     <path_to_lc-rc_Q.cvstore> \
  --disp_output_dir <path_to_04_zncc_disp_map> \
  --pcl_output_dir  <path_to_06_zncc_pcl>

# Troubleshooting mode — saves all outputs (vertical shift, correlation, point cloud)
python stereo_disparity_main.py \
  --left_dir   <path_to_rect_images> \
  --right_dir  <path_to_rect_images> \
  --left_prefix  lc_ \
  --right_prefix rc_ \
  --q_file     <path_to_lc-rc_Q.cvstore> \
  --disp_output_dir <path_to_04_zncc_disp_map> \
  --pcl_output_dir  <path_to_06_zncc_pcl> \
  --troubleshooting

This runs the program with default parameters. Same commands for stereo_disparity_cpu.py and stereo_disparity_gpu.py

Custom Parameters

You can override any parameter through command-line arguments. Below is an example with some customized parameters:

python stereo_disparity_main.py \
  --window_size 5 \
  --H_neg_range -165 \
  --H_pos_range 65 \
  --v_neg_range -4 \
  --v_pos_range 0 \
  --zncc_threshold 0.4 \
  --num_images 10 \   # optional: last 10 pairs by timestamp; omit for all pairs
  --noise_filter "open3d" \
  --interpolation True \
  --noise_remove False \
  --method "gaussian" \
  --q_file     "./data/Q_matrix.cvstore" \
  --left_dir   "./data/left" \
  --right_dir  "./data/right" \
  --disp_output_dir "./output/disp" \
  --pcl_output_dir  "./output/pcl" \
  --troubleshooting

Available Parameters

Parameter	Default Value	Description
--window_size	5	Size of the window for block matching.
--H_neg_range	0	Horizontal negative disparity range.
--H_pos_range	65	Horizontal positive disparity range.
--v_neg_range	-4	Vertical negative disparity range.
--v_pos_range	2	Vertical positive disparity range.
--zncc_threshold	0.4	ZNCC threshold for valid matches.
--num_images	all pairs	Stereo pairs to use per scan. Default: all lc_ts*/rc_ts* pairs matched on ts. If fewer than available, the last N pairs (highest timestamps) are used; ck* suffix is ignored.
--noise_filter	open3d	Post-processing noise filter method (median[Disparity Map] or open3d[Pointcloud]).
--interpolation	True	Enable subpixel interpolation (True or False).
--noise_remove	True	noise removal using bidirectional disparity estimation (True or False).
--method	parabolic	Subpixel interpolation method (parabolic, gaussian, or equiangular).
--q_file	./data1/Q.cvstore	Path to the Q matrix file.
--left_dir	./data1/left	Path to the directory containing left images.
--right_dir	./data1/right	Path to the directory containing right images.
--left_prefix	""	Filename prefix to filter left images (e.g. lc_) when both cameras share a folder.
--right_prefix	""	Filename prefix to filter right images (e.g. rc_) when both cameras share a folder.
--output_dir	./results	Fallback output directory (used when --disp_output_dir/--pcl_output_dir are not set).
--disp_output_dir	same as --output_dir	Directory for disparity map outputs (npy + png).
--pcl_output_dir	same as --output_dir	Directory for point cloud outputs (PLY + TXT).
--troubleshooting	False	When set, saves all outputs (vertical shift map, correlation map, point cloud); otherwise only disparity map is saved.

Details of Post-processing noise filter method

• open3d: used for denoising point clouds.
• median: used to reduce noise in the disparity map.

Details of Subpixel Interpolation Methods

• Parabolic: Fits a parabola to the ZNCC values and finds the peak.
• Gaussian: Uses the logarithm of ZNCC values to model a Gaussian distribution and find the peak.
• Equiangular: Uses angular interpolation for smoother and more robust subpixel shifts.

Notes

• Ensure stereo images are rectified before processing.
• Adjust the disparity ranges (H_neg_range, H_pos_range, etc.) based on your dataset.
• If disparity maps appear noisy, enable median filtering or refine the ZNCC threshold.

Execution Time

• Execution time depends on the number of images, window size, and disparity ranges.
• Subpixel interpolation adds computational overhead but improves accuracy.

Point Cloud File

• The generated .ply file can be visualized using point cloud tools like MeshLab or CloudCompare.
• Depth values are normalized and color-coded using the jet colormap.

Troubleshooting

• Ensure the Q.cvstore file exists in the specified path.
• Check file permissions.
• Ensure the left and right images are rectified and have the same resolution.
• Reduce the number of images or use a smaller window size.
• Run on a machine with higher processing power.

---

Pipeline Usage (Automated Path Resolution)

Use run_zncc_pipeline.py instead of calling stereo_disparity_main.py directly. It resolves all paths automatically from the project folder structure and processes every scan in a session (or a single scan you name).

Folder structure assumed

~/Speckle-Scanner_Processing_data/
└── <project>/
    └── <date>/
        └── <session>/
            ├── params_link/
            │   └── lc-rc_Q.cvstore          ← Q matrix (input)
            └── <ScanXXXXXX>/
                ├── 02_rect_images/           ← lc_ts*.png + rc_ts*.png (input)
                ├── 04_zncc_disp_map/         ← disparity .npy + colorbar .png (created)
                └── 06_zncc_pcl/              ← Point_cloud.ply + .txt (created)

Commands

cd ~/Speckle-Scanner/05_disparity/zncc

# Process ALL scans in a session (all matched lc/rc pairs per scan)
python run_zncc_pipeline.py \
  --project Olsen_wings \
  --date    2026-05-12 \
  --session session1

# Process ALL sessions on a date (omit --session)
python run_zncc_pipeline.py \
  --project Olsen_wings \
  --date    2026-05-12

# Process a SINGLE scan — only the last 3 pairs (highest timestamps)
python run_zncc_pipeline.py \
  --project Olsen_wings \
  --date    2026-05-12 \
  --session session1 \
  --scan    Scan000001 \
  --num_images 3

# Full outputs including point cloud and debug maps (troubleshooting mode)
python run_zncc_pipeline.py \
  --project        Olsen_wings \
  --date           2026-05-12 \
  --session        session1 \
  --scan           Scan000001 \
  --troubleshooting

# Custom ZNCC parameters
python run_zncc_pipeline.py \
  --project       Olsen_wings \
  --date          2026-05-12 \
  --session       session1 \
  --window_size   7 \
  --H_neg_range   0 \
  --H_pos_range   80 \
  --v_neg_range   -4 \
  --v_pos_range   2 \
  --zncc_threshold 0.5 \
  --noise_filter  open3d \
  --method        gaussian

Pipeline parameters

Parameter	Default	Description
--project	—	Project name (e.g. Olsen_wings)
--date	—	Date string (e.g. 2026-05-12)
--session	all	Session name (e.g. session1); omit to process all sessions on that date
--scan	all	Single scan to process (e.g. Scan000001); omit to process all scans in session
--troubleshooting	off	When set, saves vertical shift map, correlation map, and point cloud PLY/TXT; by default only disparity.npy and Disparity_map_colorbar.png are saved
--num_images	all	Image pairs per scan (matched on ts; last N if limited)
All other ZNCC params	see above table	Forwarded directly to stereo_disparity_main.py

What gets saved

File	Default	--troubleshooting
04_zncc_disp_map/disparity.npy	✓	✓
04_zncc_disp_map/Disparity_map_colorbar.png	✓	✓
04_zncc_disp_map/vertical_shift_map.npy		✓
04_zncc_disp_map/vertical_shift_map_colorbar.png		✓
04_zncc_disp_map/correlation_map.npy		✓
04_zncc_disp_map/correlation_map_colorbar.png		✓
06_zncc_pcl/Point_cloud.ply		✓
06_zncc_pcl/Point_cloud.txt		✓

---

Contributions and improvements are welcome! Feel free to open an issue or submit a pull request on GitHub.