# **Stereo Disparity Map and Point Cloud Generation**

This project implements a dense stereo disparity map and point cloud generation pipeline using spatio-temporal Zero-Normalized Cross-Correlation (ZNCC) with subpixel interpolation methods and other advanced features.

## **Features**

- **Dense Disparity Map Calculation**: Generates Disparity and vertical shift maps.
- **Subpixel Interpolation**: Supports parabolic, Gaussian, and equiangular interpolation methods for subpixel accuracy.
- **Correlation Map**: Computes ZNCC correlation for each pixel.
- **Point Cloud Generation**: Reprojects disparity maps into 3D space using a Q matrix.
- **Color Mapping**: Normalizes depth values and applies a colormap for visualization.
- **Optional Median Filtering**: Smoothens the disparity map.
- **Visualization**: Saves disparity maps, vertical shift maps, and correlation maps with colorbars.

## **Requirements**

- Python 3.9 or later

```bash
# This step only
pip install -r ~/Speckle-Scanner/05_disparity/zncc/requirements.txt

# Or install everything for the full pipeline
pip install -r ~/Speckle-Scanner/requirements.txt

# Optional GPU acceleration (pick one matching your CUDA version)
pip install cupy-cuda12x   # CUDA 12.x
pip install cupy-cuda11x   # CUDA 11.2–11.8
```

Packages: `numpy`, `opencv-python`, `matplotlib`, `scipy`, `numba`, `open3d`. Without `cupy`, the pipeline falls back to CPU (`numba`).

## **Input Requirements**

- **Stereo Rectified Images**:
  - Images must be stored in `.png` or `.bmp` format.
  - **Pipeline pairing** (in `02_rect_images/`): `lc_ts1634840093_ck….png` is matched with `rc_ts1634840093_ck….png` on the shared `ts` token; the `ck…` suffix is ignored.
  - Pairs are sorted by timestamp; `--num_images` defaults to **all** pairs; if limited, the **last N** (highest timestamps) are used.
  - Standalone runs may still use separate `left/` and `right/` folders with any naming; pipeline mode uses one folder with `lc_` / `rc_` prefixes.
  - PNG is preferred when both PNG and BMP pairs exist.
- **Q Matrix File**:
  - A `.cvstore` file containing the Q matrix for 3D reprojection.

## **Outputs**

- **Disparity Map**: Saved as `Disparity.npy`.
- **Vertical Shift Map**: Saved as `vertical_shift_map.npy`.
- **Correlation Map**: Saved as `correlation_map.npy`.
- **Disparity and Shift Maps with Colorbars**:
  - `Disparity_map_colorbar.png`
  - `vertical_shift_map_colorbar.png`
  - `correlation_map_colorbar.png`
- **Point Cloud**:
  - Saved as `Point_cloud.ply` in PLY format with depth-based color information.

## **Usage**

1. Clone the repository and navigate to the project directory.
2. Prepare your stereo images and paste them in data folder
Example directory structure: 
``` 

project_directory/ 

  .data/
    ├── left/
    │   ├── 0.png
    │   ├── 1.png
    │   └── ...
    └── right/
        ├── 0.png
        ├── 1.png
        └── ...

Or

  .data/
    ├── left/
    │   ├── lc00000.bmp
    │   └── lc00001.bmp
    └── right/
        ├── rc00000.bmp
        └── rc00001.bmp

```
3. The Q matrix must be provided in a YAML file (e.g., `Q.cvstore`). 
Format example: 
``` 

  %YAML:1.0 
  --- 
  Q: !!opencv-matrix 
     rows: 4 
     cols: 4 
     dt: f 
     data: [ 1.0, 0.0, 0.0, -5.5863211059570312e+02, 
             0.0, 1.0, 0.0, -7.3320780181884766e+02, 
             0.0, 0.0, 0.0, 3.3665836719173171e+03, 
             0.0, 0.0, 1.4262549294100269e+00, 1.0682440279589944e+03 ] 

```
This matrix is essential for converting disparity maps into a 3D point cloud. 
3. Run the script with desired parameters (see below).

### **Basic Command**

```bash
# stereo_disparity_main.py auto-selects GPU (CUDA) or CPU.

cd ~/Speckle-Scanner/05_disparity/zncc

# Default parameters (saves disparity map only)
python stereo_disparity_main.py \
  --left_dir   <path_to_rect_images> \
  --right_dir  <path_to_rect_images> \
  --left_prefix  lc_ \
  --right_prefix rc_ \
  --q_file     <path_to_lc-rc_Q.cvstore> \
  --disp_output_dir <path_to_04_zncc_disp_map> \
  --pcl_output_dir  <path_to_06_zncc_pcl>

# Troubleshooting mode — saves all outputs (vertical shift, correlation, point cloud)
python stereo_disparity_main.py \
  --left_dir   <path_to_rect_images> \
  --right_dir  <path_to_rect_images> \
  --left_prefix  lc_ \
  --right_prefix rc_ \
  --q_file     <path_to_lc-rc_Q.cvstore> \
  --disp_output_dir <path_to_04_zncc_disp_map> \
  --pcl_output_dir  <path_to_06_zncc_pcl> \
  --troubleshooting
```
This runs the program with default parameters. 
Same commands for stereo_disparity_cpu.py and stereo_disparity_gpu.py

### **Custom Parameters** 


You can override any parameter through command-line arguments. Below is an example with some customized parameters:

```bash
python stereo_disparity_main.py \
  --window_size 5 \
  --H_neg_range -165 \
  --H_pos_range 65 \
  --v_neg_range -4 \
  --v_pos_range 0 \
  --zncc_threshold 0.4 \
  --num_images 10 \   # optional: last 10 pairs by timestamp; omit for all pairs
  --noise_filter "open3d" \
  --interpolation True \
  --noise_remove False \
  --method "gaussian" \
  --q_file     "./data/Q_matrix.cvstore" \
  --left_dir   "./data/left" \
  --right_dir  "./data/right" \
  --disp_output_dir "./output/disp" \
  --pcl_output_dir  "./output/pcl" \
  --troubleshooting
```

 
## **Available Parameters**

| Parameter          | Default Value       | Description                                                                           |
| ------------------ | ------------------- | --------------------------------------------------------------------------------------|
| `--window_size`    | `5`                 | Size of the window for block matching.                                                |
| `--H_neg_range`    | `0`                 | Horizontal negative disparity range.                                                  |
| `--H_pos_range`    | `65`                | Horizontal positive disparity range.                                                  |
| `--v_neg_range`    | `-4`                | Vertical negative disparity range.                                                    |
| `--v_pos_range`    | `2`                 | Vertical positive disparity range.                                                    |
| `--zncc_threshold` | `0.4`               | ZNCC threshold for valid matches.                                                     |
| `--num_images`     | all pairs           | Stereo pairs to use per scan. Default: all `lc_ts*`/`rc_ts*` pairs matched on `ts`. If fewer than available, the **last N** pairs (highest timestamps) are used; `ck*` suffix is ignored. |
| `--noise_filter`   | `open3d`            | Post-processing noise filter method (`median`[Disparity Map] or `open3d`[Pointcloud]).|
| `--interpolation`  | `True`              | Enable subpixel interpolation (`True` or `False`).                                    |
| `--noise_remove`   | `True`              | noise removal using bidirectional disparity estimation (`True` or `False`).           |
| `--method`         | `parabolic`         | Subpixel interpolation method (`parabolic`, `gaussian`, or `equiangular`).            |
| `--q_file`            | `./data1/Q.cvstore` | Path to the Q matrix file.                                                         |
| `--left_dir`          | `./data1/left`      | Path to the directory containing left images.                                      |
| `--right_dir`         | `./data1/right`     | Path to the directory containing right images.                                     |
| `--left_prefix`       | `""`                | Filename prefix to filter left images (e.g. `lc_`) when both cameras share a folder. |
| `--right_prefix`      | `""`                | Filename prefix to filter right images (e.g. `rc_`) when both cameras share a folder. |
| `--output_dir`        | `./results`         | Fallback output directory (used when `--disp_output_dir`/`--pcl_output_dir` are not set). |
| `--disp_output_dir`   | same as `--output_dir` | Directory for disparity map outputs (npy + png).                                |
| `--pcl_output_dir`    | same as `--output_dir` | Directory for point cloud outputs (PLY + TXT).                                  |
| `--troubleshooting`   | `False`             | When set, saves all outputs (vertical shift map, correlation map, point cloud); otherwise only disparity map is saved. |


## **Details of Post-processing noise filter method**

- **open3d**: used for denoising point clouds.
- **median**: used to reduce noise in the disparity map.

## **Details of Subpixel Interpolation Methods**

- **Parabolic**: Fits a parabola to the ZNCC values and finds the peak.
- **Gaussian**: Uses the logarithm of ZNCC values to model a Gaussian distribution and find the peak.
- **Equiangular**: Uses angular interpolation for smoother and more robust subpixel shifts.

## **Notes**

- Ensure stereo images are rectified before processing.
- Adjust the disparity ranges (`H_neg_range`, `H_pos_range`, etc.) based on your dataset.
- If disparity maps appear noisy, enable median filtering or refine the ZNCC threshold.

## **Execution Time**

- Execution time depends on the number of images, window size, and disparity ranges.
- Subpixel interpolation adds computational overhead but improves accuracy.

## **Point Cloud File**

- The generated `.ply` file can be visualized using point cloud tools like MeshLab or CloudCompare.
- Depth values are normalized and color-coded using the `jet` colormap.

## **Troubleshooting** 

- Ensure the `Q.cvstore` file exists in the specified path. 
- Check file permissions. 
- Ensure the left and right images are rectified and have the same resolution. 
- Reduce the number of images or use a smaller window size. 
- Run on a machine with higher processing power. 

---

## **Pipeline Usage (Automated Path Resolution)**

Use `run_zncc_pipeline.py` instead of calling `stereo_disparity_main.py` directly.
It resolves all paths automatically from the project folder structure and processes
every scan in a session (or a single scan you name).

### **Folder structure assumed**

```
~/Speckle-Scanner_Processing_data/
└── <project>/
    └── <date>/
        └── <session>/
            ├── params_link/
            │   └── lc-rc_Q.cvstore          ← Q matrix (input)
            └── <ScanXXXXXX>/
                ├── 02_rect_images/           ← lc_ts*.png + rc_ts*.png (input)
                ├── 04_zncc_disp_map/         ← disparity .npy + colorbar .png (created)
                └── 06_zncc_pcl/              ← Point_cloud.ply + .txt (created)
```

### **Commands**

```bash
cd ~/Speckle-Scanner/05_disparity/zncc

# Process ALL scans in a session (all matched lc/rc pairs per scan)
python run_zncc_pipeline.py \
  --project Olsen_wings \
  --date    2026-05-12 \
  --session session1

# Process ALL sessions on a date (omit --session)
python run_zncc_pipeline.py \
  --project Olsen_wings \
  --date    2026-05-12

# Process a SINGLE scan — only the last 3 pairs (highest timestamps)
python run_zncc_pipeline.py \
  --project Olsen_wings \
  --date    2026-05-12 \
  --session session1 \
  --scan    Scan000001 \
  --num_images 3

# Full outputs including point cloud and debug maps (troubleshooting mode)
python run_zncc_pipeline.py \
  --project        Olsen_wings \
  --date           2026-05-12 \
  --session        session1 \
  --scan           Scan000001 \
  --troubleshooting

# Custom ZNCC parameters
python run_zncc_pipeline.py \
  --project       Olsen_wings \
  --date          2026-05-12 \
  --session       session1 \
  --window_size   7 \
  --H_neg_range   0 \
  --H_pos_range   80 \
  --v_neg_range   -4 \
  --v_pos_range   2 \
  --zncc_threshold 0.5 \
  --noise_filter  open3d \
  --method        gaussian
```

### **Pipeline parameters**

| Parameter           | Default | Description                                                                        |
|---------------------|---------|------------------------------------------------------------------------------------|
| `--project`         | —       | Project name (e.g. `Olsen_wings`)                                                  |
| `--date`            | —       | Date string (e.g. `2026-05-12`)                                                    |
| `--session`         | all     | Session name (e.g. `session1`); omit to process **all sessions** on that date      |
| `--scan`            | all     | Single scan to process (e.g. `Scan000001`); omit to process all scans in session   |
| `--troubleshooting` | off     | When set, saves vertical shift map, correlation map, and point cloud PLY/TXT; by default only `disparity.npy` and `Disparity_map_colorbar.png` are saved |
| `--num_images`      | all     | Image pairs per scan (matched on `ts`; last N if limited)                           |
| All other ZNCC params | see above table | Forwarded directly to `stereo_disparity_main.py`                     |

### **What gets saved**

| File | Default | `--troubleshooting` |
|------|:-------:|:-------------------:|
| `04_zncc_disp_map/disparity.npy` | ✓ | ✓ |
| `04_zncc_disp_map/Disparity_map_colorbar.png` | ✓ | ✓ |
| `04_zncc_disp_map/vertical_shift_map.npy` | | ✓ |
| `04_zncc_disp_map/vertical_shift_map_colorbar.png` | | ✓ |
| `04_zncc_disp_map/correlation_map.npy` | | ✓ |
| `04_zncc_disp_map/correlation_map_colorbar.png` | | ✓ |
| `06_zncc_pcl/Point_cloud.ply` | | ✓ |
| `06_zncc_pcl/Point_cloud.txt` | | ✓ |

---

Contributions and improvements are welcome! Feel free to open an issue or submit a pull request on GitHub.


