This project implements a dense stereo disparity map and point cloud generation pipeline using spatio-temporal Zero-Normalized Cross-Correlation (ZNCC) with subpixel interpolation methods and other advanced features.

Features

Dense Disparity Map Calculation: Generates Disparity and vertical shift maps.
Subpixel Interpolation: Supports parabolic, Gaussian, and equiangular interpolation methods for subpixel accuracy.
Correlation Map: Computes ZNCC correlation for each pixel.
Point Cloud Generation: Reprojects disparity maps into 3D space using a Q matrix.
Color Mapping: Normalizes depth values and applies a colormap for visualization.
Optional Median Filtering: Smoothens the disparity map.
Visualization: Saves disparity maps, vertical shift maps, and correlation maps with colorbars.

Requirements

Python 3.9 or later

# This step only
pip install -r ~/Speckle-Scanner/05_disparity/zncc/requirements.txt

# Or install everything for the full pipeline
pip install -r ~/Speckle-Scanner/requirements.txt

# Optional GPU acceleration (pick one matching your CUDA version)
pip install cupy-cuda12x   # CUDA 12.x
pip install cupy-cuda11x   # CUDA 11.2–11.8

Packages: numpy, opencv-python, matplotlib, scipy, numba, open3d. Without cupy, the pipeline falls back to CPU (numba).

Input Requirements

Stereo Rectified Images:
- Images must be stored in .png or .bmp format.
- Pipeline pairing (in 02_rect_images/): lc_ts1634840093_ck….png is matched with rc_ts1634840093_ck….png on the shared ts token; the ck… suffix is ignored.
- Pairs are sorted by timestamp; --num_images defaults to all pairs; if limited, the last N (highest timestamps) are used.
- Standalone runs may still use separate left/ and right/ folders with any naming; pipeline mode uses one folder with lc_ / rc_ prefixes.
- PNG is preferred when both PNG and BMP pairs exist.
Q Matrix File:
- A .cvstore file containing the Q matrix for 3D reprojection.

Outputs

Disparity Map: Saved as Disparity.npy.
Vertical Shift Map: Saved as vertical_shift_map.npy.
Correlation Map: Saved as correlation_map.npy.
Disparity and Shift Maps with Colorbars:
- Disparity_map_colorbar.png
- vertical_shift_map_colorbar.png
- correlation_map_colorbar.png
Point Cloud:
- Saved as Point_cloud.ply in PLY format with depth-based color information.

Usage

Clone the repository and navigate to the project directory.
Prepare your stereo images and paste them in data folder Example directory structure:


project_directory/ 

  .data/
    ├── left/
    │   ├── 0.png
    │   ├── 1.png
    │   └── ...
    └── right/
        ├── 0.png
        ├── 1.png
        └── ...

Or

  .data/
    ├── left/
    │   ├── lc00000.bmp
    │   └── lc00001.bmp
    └── right/
        ├── rc00000.bmp
        └── rc00001.bmp

The Q matrix must be provided in a YAML file (e.g., Q.cvstore). Format example:


  %YAML:1.0 
  --- 
  Q: !!opencv-matrix 
     rows: 4 
     cols: 4 
     dt: f 
     data: [ 1.0, 0.0, 0.0, -5.5863211059570312e+02, 
             0.0, 1.0, 0.0, -7.3320780181884766e+02, 
             0.0, 0.0, 0.0, 3.3665836719173171e+03, 
             0.0, 0.0, 1.4262549294100269e+00, 1.0682440279589944e+03 ]

This matrix is essential for converting disparity maps into a 3D point cloud. 3. Run the script with desired parameters (see below).

Basic Command

# stereo_disparity_main.py auto-selects GPU (CUDA) or CPU.

cd ~/Speckle-Scanner/05_disparity/zncc

# Default parameters (saves disparity map only)
python stereo_disparity_main.py \
  --left_dir   <path_to_rect_images> \
  --right_dir  <path_to_rect_images> \
  --left_prefix  lc_ \
  --right_prefix rc_ \
  --q_file     <path_to_lc-rc_Q.cvstore> \
  --disp_output_dir <path_to_04_zncc_disp_map> \
  --pcl_output_dir  <path_to_06_zncc_pcl>

# Troubleshooting mode — saves all outputs (vertical shift, correlation, point cloud)
python stereo_disparity_main.py \
  --left_dir   <path_to_rect_images> \
  --right_dir  <path_to_rect_images> \
  --left_prefix  lc_ \
  --right_prefix rc_ \
  --q_file     <path_to_lc-rc_Q.cvstore> \
  --disp_output_dir <path_to_04_zncc_disp_map> \
  --pcl_output_dir  <path_to_06_zncc_pcl> \
  --troubleshooting

This runs the program with default parameters. Same commands for stereo_disparity_cpu.py and stereo_disparity_gpu.py

Custom Parameters

You can override any parameter through command-line arguments. Below is an example with some customized parameters:

python stereo_disparity_main.py \
  --window_size 5 \
  --H_neg_range -165 \
  --H_pos_range 65 \
  --v_neg_range -4 \
  --v_pos_range 0 \
  --zncc_threshold 0.4 \
  --num_images 10 \   # optional: last 10 pairs by timestamp; omit for all pairs
  --noise_filter "open3d" \
  --interpolation True \
  --noise_remove False \
  --method "gaussian" \
  --q_file     "./data/Q_matrix.cvstore" \
  --left_dir   "./data/left" \
  --right_dir  "./data/right" \
  --disp_output_dir "./output/disp" \
  --pcl_output_dir  "./output/pcl" \
  --troubleshooting

Available Parameters

Parameter	Default Value	Description
`--window_size`	`5`	Size of the window for block matching.
`--H_neg_range`	`0`	Horizontal negative disparity range.
`--H_pos_range`	`65`	Horizontal positive disparity range.
`--v_neg_range`	`-4`	Vertical negative disparity range.
`--v_pos_range`	`2`	Vertical positive disparity range.
`--zncc_threshold`	`0.4`	ZNCC threshold for valid matches.
`--num_images`	all pairs	Stereo pairs to use per scan. Default: all `lc_ts`/`rc_ts` pairs matched on `ts`. If fewer than available, the last N pairs (highest timestamps) are used; `ck*` suffix is ignored.
`--noise_filter`	`open3d`	Post-processing noise filter method (`median`[Disparity Map] or `open3d`[Pointcloud]).
`--interpolation`	`True`	Enable subpixel interpolation (`True` or `False`).
`--noise_remove`	`True`	noise removal using bidirectional disparity estimation (`True` or `False`).
`--method`	`parabolic`	Subpixel interpolation method (`parabolic`, `gaussian`, or `equiangular`).
`--q_file`	`./data1/Q.cvstore`	Path to the Q matrix file.
`--left_dir`	`./data1/left`	Path to the directory containing left images.
`--right_dir`	`./data1/right`	Path to the directory containing right images.
`--left_prefix`	`""`	Filename prefix to filter left images (e.g. `lc_`) when both cameras share a folder.
`--right_prefix`	`""`	Filename prefix to filter right images (e.g. `rc_`) when both cameras share a folder.
`--output_dir`	`./results`	Fallback output directory (used when `--disp_output_dir`/`--pcl_output_dir` are not set).
`--disp_output_dir`	same as `--output_dir`	Directory for disparity map outputs (npy + png).
`--pcl_output_dir`	same as `--output_dir`	Directory for point cloud outputs (PLY + TXT).
`--troubleshooting`	`False`	When set, saves all outputs (vertical shift map, correlation map, point cloud); otherwise only disparity map is saved.

Details of Post-processing noise filter method

open3d: used for denoising point clouds.
median: used to reduce noise in the disparity map.

Details of Subpixel Interpolation Methods

Parabolic: Fits a parabola to the ZNCC values and finds the peak.
Gaussian: Uses the logarithm of ZNCC values to model a Gaussian distribution and find the peak.
Equiangular: Uses angular interpolation for smoother and more robust subpixel shifts.

Notes

Ensure stereo images are rectified before processing.
Adjust the disparity ranges (H_neg_range, H_pos_range, etc.) based on your dataset.
If disparity maps appear noisy, enable median filtering or refine the ZNCC threshold.

Execution Time

Execution time depends on the number of images, window size, and disparity ranges.
Subpixel interpolation adds computational overhead but improves accuracy.

Point Cloud File

The generated .ply file can be visualized using point cloud tools like MeshLab or CloudCompare.
Depth values are normalized and color-coded using the jet colormap.

Troubleshooting

Ensure the Q.cvstore file exists in the specified path.
Check file permissions.
Ensure the left and right images are rectified and have the same resolution.
Reduce the number of images or use a smaller window size.
Run on a machine with higher processing power.

Pipeline Usage (Automated Path Resolution)

Use run_zncc_pipeline.py instead of calling stereo_disparity_main.py directly. It resolves all paths automatically from the project folder structure and processes every scan in a session (or a single scan you name).

Folder structure assumed

~/Speckle-Scanner_Processing_data/
└── <project>/
    └── <date>/
        └── <session>/
            ├── params_link/
            │   └── lc-rc_Q.cvstore          ← Q matrix (input)
            └── <ScanXXXXXX>/
                ├── 02_rect_images/           ← lc_ts*.png + rc_ts*.png (input)
                ├── 04_zncc_disp_map/         ← disparity .npy + colorbar .png (created)
                └── 06_zncc_pcl/              ← Point_cloud.ply + .txt (created)

Commands

cd ~/Speckle-Scanner/05_disparity/zncc

# Process ALL scans in a session (all matched lc/rc pairs per scan)
python run_zncc_pipeline.py \
  --project Olsen_wings \
  --date    2026-05-12 \
  --session session1

# Process ALL sessions on a date (omit --session)
python run_zncc_pipeline.py \
  --project Olsen_wings \
  --date    2026-05-12

# Process a SINGLE scan — only the last 3 pairs (highest timestamps)
python run_zncc_pipeline.py \
  --project Olsen_wings \
  --date    2026-05-12 \
  --session session1 \
  --scan    Scan000001 \
  --num_images 3

# Full outputs including point cloud and debug maps (troubleshooting mode)
python run_zncc_pipeline.py \
  --project        Olsen_wings \
  --date           2026-05-12 \
  --session        session1 \
  --scan           Scan000001 \
  --troubleshooting

# Custom ZNCC parameters
python run_zncc_pipeline.py \
  --project       Olsen_wings \
  --date          2026-05-12 \
  --session       session1 \
  --window_size   7 \
  --H_neg_range   0 \
  --H_pos_range   80 \
  --v_neg_range   -4 \
  --v_pos_range   2 \
  --zncc_threshold 0.5 \
  --noise_filter  open3d \
  --method        gaussian

Pipeline parameters

Parameter	Default	Description
`--project`	—	Project name (e.g. `Olsen_wings`)
`--date`	—	Date string (e.g. `2026-05-12`)
`--session`	all	Session name (e.g. `session1`); omit to process all sessions on that date
`--scan`	all	Single scan to process (e.g. `Scan000001`); omit to process all scans in session
`--troubleshooting`	off	When set, saves vertical shift map, correlation map, and point cloud PLY/TXT; by default only `disparity.npy` and `Disparity_map_colorbar.png` are saved
`--num_images`	all	Image pairs per scan (matched on `ts`; last N if limited)
All other ZNCC params	see above table	Forwarded directly to `stereo_disparity_main.py`

What gets saved

File	Default	`--troubleshooting`
`04_zncc_disp_map/disparity.npy`	✓	✓
`04_zncc_disp_map/Disparity_map_colorbar.png`	✓	✓
`04_zncc_disp_map/vertical_shift_map.npy`		✓
`04_zncc_disp_map/vertical_shift_map_colorbar.png`		✓
`04_zncc_disp_map/correlation_map.npy`		✓
`04_zncc_disp_map/correlation_map_colorbar.png`		✓
`06_zncc_pcl/Point_cloud.ply`		✓
`06_zncc_pcl/Point_cloud.txt`		✓

Contributions and improvements are welcome! Feel free to open an issue or submit a pull request on GitHub.

Readme.md Unescape Escape

Stereo Disparity Map and Point Cloud Generation