AsadUllah/Speckle-Scanner
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit: Speckle-Scanner 3D pipeline with setup README

Browse Source
This commit is contained in:
AsadUllah
2026-06-10 03:09:05 +05:00
commit 1765934846
375 changed files with 123081 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
05_disparity/zncc/Readme.md
+323
View File
@@ -0,0 +1,323 @@
# **Stereo Disparity Map and Point Cloud Generation**
This project implements a dense stereo disparity map and point cloud generation pipeline using spatio-temporal Zero-Normalized Cross-Correlation (ZNCC) with subpixel interpolation methods and other advanced features.
## **Features**
- **Dense Disparity Map Calculation**: Generates Disparity and vertical shift maps.
- **Subpixel Interpolation**: Supports parabolic, Gaussian, and equiangular interpolation methods for subpixel accuracy.
- **Correlation Map**: Computes ZNCC correlation for each pixel.
- **Point Cloud Generation**: Reprojects disparity maps into 3D space using a Q matrix.
- **Color Mapping**: Normalizes depth values and applies a colormap for visualization.
- **Optional Median Filtering**: Smoothens the disparity map.
- **Visualization**: Saves disparity maps, vertical shift maps, and correlation maps with colorbars.
## **Requirements**
- Python 3.9 or later
```bash
# This step only
pip install -r ~/Speckle-Scanner/05_disparity/zncc/requirements.txt
# Or install everything for the full pipeline
pip install -r ~/Speckle-Scanner/requirements.txt
# Optional GPU acceleration (pick one matching your CUDA version)
pip install cupy-cuda12x # CUDA 12.x
pip install cupy-cuda11x # CUDA 11.2–11.8
```
Packages: `numpy`, `opencv-python`, `matplotlib`, `scipy`, `numba`, `open3d`. Without `cupy`, the pipeline falls back to CPU (`numba`).
## **Input Requirements**
- **Stereo Rectified Images**:
- Images must be stored in `.png` or `.bmp` format.
- **Pipeline pairing** (in `02_rect_images/`): `lc_ts1634840093_ck….png` is matched with `rc_ts1634840093_ck….png` on the shared `ts` token; the `ck…` suffix is ignored.
- Pairs are sorted by timestamp; `--num_images` defaults to **all** pairs; if limited, the **last N** (highest timestamps) are used.
- Standalone runs may still use separate `left/` and `right/` folders with any naming; pipeline mode uses one folder with `lc_` / `rc_` prefixes.
- PNG is preferred when both PNG and BMP pairs exist.
- **Q Matrix File**:
- A `.cvstore` file containing the Q matrix for 3D reprojection.
## **Outputs**
- **Disparity Map**: Saved as `Disparity.npy`.
- **Vertical Shift Map**: Saved as `vertical_shift_map.npy`.
- **Correlation Map**: Saved as `correlation_map.npy`.
- **Disparity and Shift Maps with Colorbars**:
- `Disparity_map_colorbar.png`
- `vertical_shift_map_colorbar.png`
- `correlation_map_colorbar.png`
- **Point Cloud**:
- Saved as `Point_cloud.ply` in PLY format with depth-based color information.
## **Usage**
1. Clone the repository and navigate to the project directory.
2. Prepare your stereo images and paste them in data folder
Example directory structure:
```
project_directory/
.data/
├── left/
│ ├── 0.png
│ ├── 1.png
│ └── ...
└── right/
├── 0.png
├── 1.png
└── ...
Or
.data/
├── left/
│ ├── lc00000.bmp
│ └── lc00001.bmp
└── right/
├── rc00000.bmp
└── rc00001.bmp
```
3. The Q matrix must be provided in a YAML file (e.g., `Q.cvstore`).
Format example:
```
%YAML:1.0
---
Q: !!opencv-matrix
rows: 4
cols: 4
dt: f
data: [ 1.0, 0.0, 0.0, -5.5863211059570312e+02,
0.0, 1.0, 0.0, -7.3320780181884766e+02,
0.0, 0.0, 0.0, 3.3665836719173171e+03,
0.0, 0.0, 1.4262549294100269e+00, 1.0682440279589944e+03 ]
```
This matrix is essential for converting disparity maps into a 3D point cloud.
3. Run the script with desired parameters (see below).
### **Basic Command**
```bash
# stereo_disparity_main.py auto-selects GPU (CUDA) or CPU.
cd ~/Speckle-Scanner/05_disparity/zncc
# Default parameters (saves disparity map only)
python stereo_disparity_main.py \
--left_dir <path_to_rect_images> \
--right_dir <path_to_rect_images> \
--left_prefix lc_ \
--right_prefix rc_ \
--q_file <path_to_lc-rc_Q.cvstore> \
--disp_output_dir <path_to_04_zncc_disp_map> \
--pcl_output_dir <path_to_06_zncc_pcl>
# Troubleshooting mode — saves all outputs (vertical shift, correlation, point cloud)
python stereo_disparity_main.py \
--left_dir <path_to_rect_images> \
--right_dir <path_to_rect_images> \
--left_prefix lc_ \
--right_prefix rc_ \
--q_file <path_to_lc-rc_Q.cvstore> \
--disp_output_dir <path_to_04_zncc_disp_map> \
--pcl_output_dir <path_to_06_zncc_pcl> \
--troubleshooting
```
This runs the program with default parameters.
Same commands for stereo_disparity_cpu.py and stereo_disparity_gpu.py
### **Custom Parameters**
You can override any parameter through command-line arguments. Below is an example with some customized parameters:
```bash
python stereo_disparity_main.py \
--window_size 5 \
--H_neg_range -165 \
--H_pos_range 65 \
--v_neg_range -4 \
--v_pos_range 0 \
--zncc_threshold 0.4 \
--num_images 10 \ # optional: last 10 pairs by timestamp; omit for all pairs
--noise_filter "open3d" \
--interpolation True \
--noise_remove False \
--method "gaussian" \
--q_file "./data/Q_matrix.cvstore" \
--left_dir "./data/left" \
--right_dir "./data/right" \
--disp_output_dir "./output/disp" \
--pcl_output_dir "./output/pcl" \
--troubleshooting
```
## **Available Parameters**
| Parameter | Default Value | Description |
| ------------------ | ------------------- | --------------------------------------------------------------------------------------|
| `--window_size` | `5` | Size of the window for block matching. |
| `--H_neg_range` | `0` | Horizontal negative disparity range. |
| `--H_pos_range` | `65` | Horizontal positive disparity range. |
| `--v_neg_range` | `-4` | Vertical negative disparity range. |
| `--v_pos_range` | `2` | Vertical positive disparity range. |
| `--zncc_threshold` | `0.4` | ZNCC threshold for valid matches. |
| `--num_images` | all pairs | Stereo pairs to use per scan. Default: all `lc_ts*`/`rc_ts*` pairs matched on `ts`. If fewer than available, the **last N** pairs (highest timestamps) are used; `ck*` suffix is ignored. |
| `--noise_filter` | `open3d` | Post-processing noise filter method (`median`[Disparity Map] or `open3d`[Pointcloud]).|
| `--interpolation` | `True` | Enable subpixel interpolation (`True` or `False`). |
| `--noise_remove` | `True` | noise removal using bidirectional disparity estimation (`True` or `False`). |
| `--method` | `parabolic` | Subpixel interpolation method (`parabolic`, `gaussian`, or `equiangular`). |
| `--q_file` | `./data1/Q.cvstore` | Path to the Q matrix file. |
| `--left_dir` | `./data1/left` | Path to the directory containing left images. |
| `--right_dir` | `./data1/right` | Path to the directory containing right images. |
| `--left_prefix` | `""` | Filename prefix to filter left images (e.g. `lc_`) when both cameras share a folder. |
| `--right_prefix` | `""` | Filename prefix to filter right images (e.g. `rc_`) when both cameras share a folder. |
| `--output_dir` | `./results` | Fallback output directory (used when `--disp_output_dir`/`--pcl_output_dir` are not set). |
| `--disp_output_dir` | same as `--output_dir` | Directory for disparity map outputs (npy + png). |
| `--pcl_output_dir` | same as `--output_dir` | Directory for point cloud outputs (PLY + TXT). |
| `--troubleshooting` | `False` | When set, saves all outputs (vertical shift map, correlation map, point cloud); otherwise only disparity map is saved. |
## **Details of Post-processing noise filter method**
- **open3d**: used for denoising point clouds.
- **median**: used to reduce noise in the disparity map.
## **Details of Subpixel Interpolation Methods**
- **Parabolic**: Fits a parabola to the ZNCC values and finds the peak.
- **Gaussian**: Uses the logarithm of ZNCC values to model a Gaussian distribution and find the peak.
- **Equiangular**: Uses angular interpolation for smoother and more robust subpixel shifts.
## **Notes**
- Ensure stereo images are rectified before processing.
- Adjust the disparity ranges (`H_neg_range`, `H_pos_range`, etc.) based on your dataset.
- If disparity maps appear noisy, enable median filtering or refine the ZNCC threshold.
## **Execution Time**
- Execution time depends on the number of images, window size, and disparity ranges.
- Subpixel interpolation adds computational overhead but improves accuracy.
## **Point Cloud File**
- The generated `.ply` file can be visualized using point cloud tools like MeshLab or CloudCompare.
- Depth values are normalized and color-coded using the `jet` colormap.
## **Troubleshooting**
- Ensure the `Q.cvstore` file exists in the specified path.
- Check file permissions.
- Ensure the left and right images are rectified and have the same resolution.
- Reduce the number of images or use a smaller window size.
- Run on a machine with higher processing power.
---
## **Pipeline Usage (Automated Path Resolution)**
Use `run_zncc_pipeline.py` instead of calling `stereo_disparity_main.py` directly.
It resolves all paths automatically from the project folder structure and processes
every scan in a session (or a single scan you name).
### **Folder structure assumed**
```
~/Speckle-Scanner_Processing_data/
└── <project>/
└── <date>/
└── <session>/
├── params_link/
│ └── lc-rc_Q.cvstore ← Q matrix (input)
└── <ScanXXXXXX>/
├── 02_rect_images/ ← lc_ts*.png + rc_ts*.png (input)
├── 04_zncc_disp_map/ ← disparity .npy + colorbar .png (created)
└── 06_zncc_pcl/ ← Point_cloud.ply + .txt (created)
```
### **Commands**
```bash
cd ~/Speckle-Scanner/05_disparity/zncc
# Process ALL scans in a session (all matched lc/rc pairs per scan)
python run_zncc_pipeline.py \
--project Olsen_wings \
--date 2026-05-12 \
--session session1
# Process ALL sessions on a date (omit --session)
python run_zncc_pipeline.py \
--project Olsen_wings \
--date 2026-05-12
# Process a SINGLE scan — only the last 3 pairs (highest timestamps)
python run_zncc_pipeline.py \
--project Olsen_wings \
--date 2026-05-12 \
--session session1 \
--scan Scan000001 \
--num_images 3
# Full outputs including point cloud and debug maps (troubleshooting mode)
python run_zncc_pipeline.py \
--project Olsen_wings \
--date 2026-05-12 \
--session session1 \
--scan Scan000001 \
--troubleshooting
# Custom ZNCC parameters
python run_zncc_pipeline.py \
--project Olsen_wings \
--date 2026-05-12 \
--session session1 \
--window_size 7 \
--H_neg_range 0 \
--H_pos_range 80 \
--v_neg_range -4 \
--v_pos_range 2 \
--zncc_threshold 0.5 \
--noise_filter open3d \
--method gaussian
```
### **Pipeline parameters**
| Parameter | Default | Description |
|---------------------|---------|------------------------------------------------------------------------------------|
| `--project` | — | Project name (e.g. `Olsen_wings`) |
| `--date` | — | Date string (e.g. `2026-05-12`) |
| `--session` | all | Session name (e.g. `session1`); omit to process **all sessions** on that date |
| `--scan` | all | Single scan to process (e.g. `Scan000001`); omit to process all scans in session |
| `--troubleshooting` | off | When set, saves vertical shift map, correlation map, and point cloud PLY/TXT; by default only `disparity.npy` and `Disparity_map_colorbar.png` are saved |
| `--num_images` | all | Image pairs per scan (matched on `ts`; last N if limited) |
| All other ZNCC params | see above table | Forwarded directly to `stereo_disparity_main.py` |
### **What gets saved**
| File | Default | `--troubleshooting` |
|------|:-------:|:-------------------:|
| `04_zncc_disp_map/disparity.npy` | ✓ | ✓ |
| `04_zncc_disp_map/Disparity_map_colorbar.png` | ✓ | ✓ |
| `04_zncc_disp_map/vertical_shift_map.npy` | | ✓ |
| `04_zncc_disp_map/vertical_shift_map_colorbar.png` | | ✓ |
| `04_zncc_disp_map/correlation_map.npy` | | ✓ |
| `04_zncc_disp_map/correlation_map_colorbar.png` | | ✓ |
| `06_zncc_pcl/Point_cloud.ply` | | ✓ |
| `06_zncc_pcl/Point_cloud.txt` | | ✓ |
---
Contributions and improvements are welcome! Feel free to open an issue or submit a pull request on GitHub.