Initial commit: Speckle-Scanner 3D pipeline with setup README

This commit is contained in:
2026-06-10 03:09:05 +05:00
commit 1765934846
375 changed files with 123081 additions and 0 deletions
+174
View File
@@ -0,0 +1,174 @@
# Multi-Format Disparity to Point Cloud Converter
This script converts disparity maps in various formats into 3D point clouds and saves them in the `.ply` format for easy visualization.
---
## 📌 Features
- Supports disparity map files in `.xml`, `.png`, and `.npy` formats.
- Automatically detects file type and processes accordingly.
- Uses OpenCV's `cv2.reprojectImageTo3D()` with a Q matrix for 3D reconstruction.
- Applies a color map (`jet`) based on depth (Z-value).
- Saves the generated point cloud as `.ply` files (`.txt` only with `--troubleshooting`).
---
## Troubleshooting flag
| `--troubleshooting` | Output |
|---------------------|--------|
| **False** (default) | `Point_cloud.ply` only |
| **True** | `Point_cloud.ply` + `Point_cloud.txt` (ASCII XYZ-RGB) |
---
## ▶️ How to Run
1. Place your disparity files inside the `./data/` folder.
2. Output files will be saved in the `./output/` folder.
3. Install dependencies:
```bash
pip install -r ~/Speckle-Scanner/06_Pointcloud/requirements.txt
```
4. Run the standalone script or the pipeline runner (see **Pipeline runner** below):
```bash
python pointcloud_genration.py --disparity <path> --q <path>
```
## 📁 Supported Input Types
You can place any of the following file types inside the ./data folder:
- .xml OpenCV format with disparity stored under the key "disparity".
- .png 16-bit grayscale or float disparity maps.
- .npy NumPy arrays containing disparity values.
## 📄 Output Naming Convention
The point cloud file name is generated based on the disparity filename, with the following rules:
| Input Disparity Filename | Output Point Cloud Filename |
|------------------------------|---------------------------------|
| disparity_0001.xml | Pointcloud_0001.ply |
| horizontal_disparity.png | Pointcloud_disparity.ply |
| lr_sgm01.npy | Pointcloud_sgm01.ply |
---
## Pipeline Usage (Automated Path Resolution)
Use `run_pcl_pipeline.py` to generate point clouds across the project folder structure.
It reads disparity maps from `03_sgm_disp_map/` and/or `04_zncc_disp_map/`, takes the Q matrix
from each session's own `params_link/lc-rc_Q.cvstore`, and saves results to `05_sgm_pcl/`
and/or `06_zncc_pcl/`.
### Folder structure
```
<project>/<date>/<session>/
params_link/
lc-rc_Q.cvstore ← Q matrix (per session, used automatically)
<ScanXXXXXX>/
03_sgm_disp_map/
disparity.xml ← SGM input
04_zncc_disp_map/
disparity.npy ← ZNCC input
05_sgm_pcl/
Point_cloud.ply ← SGM output (always)
Point_cloud.txt ← only with --troubleshooting
06_zncc_pcl/
Point_cloud.ply ← ZNCC output (always)
Point_cloud.txt ← only with --troubleshooting
```
### Commands
```bash
cd ~/Speckle-Scanner/06_Pointcloud
# Both SGM + ZNCC — all scans in a session
python run_pcl_pipeline.py \
--project Olsen_wings \
--date 2026-05-12 \
--session session1
# Both SGM + ZNCC — all sessions on a date (omit --session)
python run_pcl_pipeline.py \
--project Olsen_wings \
--date 2026-05-12
# Single scan, both modes
python run_pcl_pipeline.py \
--project Olsen_wings \
--date 2026-05-12 \
--session session1 \
--scan Scan000001
# Single scan, SGM only
python run_pcl_pipeline.py \
--project Olsen_wings \
--date 2026-05-12 \
--session session1 \
--scan Scan000001 \
--mode sgm
# Single scan, ZNCC only
python run_pcl_pipeline.py \
--project Olsen_wings \
--date 2026-05-12 \
--session session1 \
--scan Scan000001 \
--mode zncc
# Also save ASCII .txt point clouds
python run_pcl_pipeline.py \
--project Olsen_wings \
--date 2026-05-12 \
--session session1 \
--scan Scan000001 \
--troubleshooting
```
### Parameters
| Parameter | Default | Description |
|-------------|---------|------------------------------------------------------------------------------------|
| `--project` | — | Project name (e.g. `Olsen_wings`) |
| `--date` | — | Date string (e.g. `2026-05-12`) |
| `--session` | all | Session name (e.g. `session1`); omit to process **all sessions** on that date |
| `--scan` | all | Single scan (e.g. `Scan000001`); omit to process all scans in the session |
| `--mode` | `both` | Which disparity source to convert: `sgm`, `zncc`, or `both` |
| `--troubleshooting` | off | When set, also saves `Point_cloud.txt` ASCII export; default writes `.ply` only |
### What gets saved
| File | Default | `--troubleshooting` |
|------|---------|---------------------|
| `05_sgm_pcl/Point_cloud.ply` | yes | yes |
| `05_sgm_pcl/Point_cloud.txt` | no | yes |
| `06_zncc_pcl/Point_cloud.ply` | yes | yes |
| `06_zncc_pcl/Point_cloud.txt` | no | yes |
---
## Dependencies
```bash
# This step only
pip install -r ~/Speckle-Scanner/06_Pointcloud/requirements.txt
# Or install everything for the full pipeline
pip install -r ~/Speckle-Scanner/requirements.txt
```
Packages: `numpy`, `opencv-python`, `matplotlib`. `open3d` is optional (only for `plot_ASCII.py`).
+22
View File
@@ -0,0 +1,22 @@
import open3d as o3d
import numpy as np
def load_txt_point_cloud(filename):
# Load the data from the ASCII file
data = np.loadtxt(filename)
points = data[:, 0:3] # X, Y, Z
colors = data[:, 3:6] / 255.0 # R, G, B (normalize to [0,1])
# Create Open3D point cloud
pcd = o3d.geometry.PointCloud()
pcd.points = o3d.utility.Vector3dVector(points)
pcd.colors = o3d.utility.Vector3dVector(colors)
return pcd
# Path to your .txt file
txt_file_path = "/home/yoga/3D_Scanner/Jost/SubseaScanner_Convert_Disp_2_Pcl/output/Pointcloud_0000.txt"
# Load and visualize
pcd = load_txt_point_cloud(txt_file_path)
o3d.visualization.draw_geometries([pcd])
+173
View File
@@ -0,0 +1,173 @@
import argparse
import os
import numpy as np
import cv2
import matplotlib.pyplot as plt
_SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
# Default paths: repo layout has input/ and rectified/ next to this package folder.
_DEFAULT_DISPARITY = os.path.normpath(
os.path.join(_SCRIPT_DIR, "..", "input", "disparity.xml")
)
_DEFAULT_Q = os.path.normpath(os.path.join(_SCRIPT_DIR, "..", "rectified", "Q.cvstore"))
def load_disparity_map(disparity_file):
"""Load disparity from .xml (OpenCV FileStorage), .png, or .npy."""
disparity_file = os.fspath(disparity_file)
if disparity_file.endswith(".xml"):
fs = cv2.FileStorage(disparity_file, cv2.FILE_STORAGE_READ)
disp_map = fs.getNode("disparity").mat().astype(np.float32)
fs.release()
elif disparity_file.endswith(".png"):
disp_map = cv2.imread(disparity_file, cv2.IMREAD_UNCHANGED).astype(np.float32)
elif disparity_file.endswith(".npy"):
disp_map = np.load(disparity_file).astype(np.float32)
else:
raise ValueError(
f"Unsupported disparity format: {disparity_file}. Use .xml, .png, or .npy."
)
return disp_map
def load_q_matrix(Q):
"""
Accept a 4x4 ndarray or a file path:
- .npy: array saved with numpy
- OpenCV FileStorage (.cvstore, .xml, .yaml, .yml): matrix node named 'Q'
"""
if isinstance(Q, (str, os.PathLike)):
path = os.fspath(Q)
lower = path.lower()
if lower.endswith(".npy"):
Q = np.load(path)
else:
fs = cv2.FileStorage(path, cv2.FILE_STORAGE_READ)
if not fs.isOpened():
raise ValueError(f"Could not open Q file: {path}")
q_node = fs.getNode("Q")
if q_node.empty():
fs.release()
raise ValueError(
f"No matrix 'Q' found in {path}. Expected OpenCV FileStorage node 'Q'."
)
Q = q_node.mat()
fs.release()
Q = np.asarray(Q, dtype=np.float64)
if Q.shape != (4, 4):
raise ValueError(f"Q must be 4x4, got shape {Q.shape}.")
return Q
def disparity_to_pointcloud(disparity_file, Q, valid_mask=None):
"""
Build a 3D point cloud from a disparity map and stereo rectification Q matrix.
Parameters
----------
disparity_file : str or os.PathLike
Path to disparity (.xml with 'disparity' node, .png, or .npy).
Q : ndarray (4, 4) or str path
4x4 reprojection matrix; .npy or OpenCV store (.cvstore / .xml / .yaml) with node 'Q'.
valid_mask : ndarray of bool, optional
If None, uses pixels where disparity > min(disparity) (same as original script).
Returns
-------
points_xyz : ndarray, shape (N, 3)
3D points in rectified camera coordinates.
"""
disp_map = load_disparity_map(disparity_file)
Qm = load_q_matrix(Q)
point_cloud = cv2.reprojectImageTo3D(disp_map, Qm)
if valid_mask is None:
valid_mask = disp_map > disp_map.min()
else:
valid_mask = np.asarray(valid_mask, dtype=bool)
points_xyz = point_cloud[valid_mask].reshape(-1, 3)
return points_xyz
def pointcloud_with_depth_colors(points_xyz):
"""Attach jet colormap RGB in [0,1] from Z depth (same visualization as before)."""
depth = points_xyz[:, 2]
dmin, dmax = depth.min(), depth.max()
if dmax > dmin:
depth_norm = (depth - dmin) / (dmax - dmin)
else:
depth_norm = np.zeros_like(depth)
colors = plt.get_cmap("jet")(depth_norm)[:, :3]
return np.hstack((points_xyz, colors))
def save_ply(filename, points):
with open(filename, "w") as f:
f.write("ply\nformat ascii 1.0\n")
f.write(f"element vertex {points.shape[0]}\n")
f.write("property float x\nproperty float y\nproperty float z\n")
f.write("property uchar red\nproperty uchar green\nproperty uchar blue\nend_header\n")
for point in points:
x, y, z, r, g, b = point
f.write(f"{x} {y} {z} {int(r * 255)} {int(g * 255)} {int(b * 255)}\n")
def save_ascii_point_cloud(filename, points):
with open(filename, "w") as f:
for point in points:
x, y, z, r, g, b = point
f.write(f"{x} {y} {z} {int(r * 255)} {int(g * 255)} {int(b * 255)}\n")
def main():
parser = argparse.ArgumentParser(
description="Convert disparity map + Q matrix to a point cloud (PLY/TXT)."
)
parser.add_argument(
"--disparity",
"-d",
default=_DEFAULT_DISPARITY,
help=f"Disparity file (.xml, .png, .npy). Default: {_DEFAULT_DISPARITY}",
)
parser.add_argument(
"--q",
"-q",
default=_DEFAULT_Q,
help=f"Q matrix: .npy or OpenCV FileStorage (.cvstore, .xml, .yaml) with node 'Q'. Default: {_DEFAULT_Q}",
)
parser.add_argument(
"--out-dir",
default="./output",
help="Directory for PLY and TXT exports (default: ./output).",
)
parser.add_argument(
"--no-save",
action="store_true",
help="Only print point count; do not write PLY/TXT.",
)
parser.add_argument(
"--troubleshooting",
action="store_true",
help="Also save ASCII .txt point cloud (default: PLY only)",
)
args = parser.parse_args()
points_xyz = disparity_to_pointcloud(args.disparity, args.q)
print(f"Point cloud shape: {points_xyz.shape} (N x 3)")
if args.no_save:
return
os.makedirs(args.out_dir, exist_ok=True)
base = os.path.splitext(os.path.basename(os.fspath(args.disparity)))[0]
point_cloud_color = pointcloud_with_depth_colors(points_xyz)
out_ply = os.path.join(args.out_dir, f"Pointcloud_{base}.ply")
save_ply(out_ply, point_cloud_color)
print(f"Saved: {out_ply}")
if args.troubleshooting:
out_txt = os.path.join(args.out_dir, f"Pointcloud_{base}.txt")
save_ascii_point_cloud(out_txt, point_cloud_color)
print(f"Saved: {out_txt}")
if __name__ == "__main__":
main()
+10
View File
@@ -0,0 +1,10 @@
# 06_Pointcloud — Python dependencies
# Install: pip install -r requirements.txt
# Full pipeline (all steps): pip install -r ~/Speckle-Scanner/requirements.txt
#
# open3d is only needed for plot_ASCII.py (optional viewer script).
numpy>=1.21
opencv-python>=4.8
matplotlib>=3.5
open3d>=0.16
+155
View File
@@ -0,0 +1,155 @@
"""
Pipeline runner: disparity map → point cloud.
Reads disparity maps from the project folder structure and saves
PLY point clouds using pointcloud_genration.py functions (.txt with --troubleshooting).
Supported modes:
sgm → 03_sgm_disp_map/disparity.xml → 05_sgm_pcl/
zncc → 04_zncc_disp_map/disparity.npy → 06_zncc_pcl/
both → runs both above (default)
Q matrix is taken from each session's own params_link folder:
<processing_dir>/<project>/<date>/<session>/params_link/lc-rc_Q.cvstore
"""
import sys
import argparse
from pathlib import Path
# Resolve config.py from ~/Speckle-Scanner regardless of CWD
sys.path.insert(0, str(Path.home() / "Speckle-Scanner"))
# Resolve pointcloud_genration.py from the same directory as this script
sys.path.insert(0, str(Path(__file__).parent))
import config
from pointcloud_genration import (
disparity_to_pointcloud,
pointcloud_with_depth_colors,
save_ply,
save_ascii_point_cloud,
)
MODE_SGM = "sgm"
MODE_ZNCC = "zncc"
MODE_BOTH = "both"
SGM_DISP_FOLDER = "03_sgm_disp_map"
ZNCC_DISP_FOLDER = "04_zncc_disp_map"
SGM_PCL_FOLDER = "05_sgm_pcl"
ZNCC_PCL_FOLDER = "06_zncc_pcl"
def run_one(disp_path, q_path, out_dir, label, troubleshooting=False):
"""Convert one disparity file to a point cloud and save PLY (+ TXT if troubleshooting)."""
if not disp_path.exists():
print(f" [{label}] SKIP — disparity not found: {disp_path.name}")
return False
if not q_path.exists():
print(f" [{label}] SKIP — Q matrix not found: {q_path}")
return False
print(f" [{label}] {disp_path.name}{out_dir.name}/")
points_xyz = disparity_to_pointcloud(disp_path, q_path)
point_cloud = pointcloud_with_depth_colors(points_xyz)
out_dir.mkdir(parents=True, exist_ok=True)
save_ply(str(out_dir / "Point_cloud.ply"), point_cloud)
if troubleshooting:
save_ascii_point_cloud(str(out_dir / "Point_cloud.txt"), point_cloud)
print(f" [{label}] {points_xyz.shape[0]} points → {out_dir.name}/Point_cloud.ply")
if troubleshooting:
print(f" [{label}] ASCII copy → {out_dir.name}/Point_cloud.txt")
return True
def run_scan(project, date, session, scan, modes, troubleshooting=False):
base = config.PROCESSING_DIR / project / date / session / scan
q_file = config.get_params_link_dir(project, date, session) / "lc-rc_Q.cvstore"
print(f"\n{'='*60}")
print(f"[SCAN] {session}/{scan}")
print(f"{'='*60}")
results = []
if MODE_SGM in modes:
disp = base / SGM_DISP_FOLDER / "disparity.xml"
out = config.get_processing_step_dir(project, date, session, scan, SGM_PCL_FOLDER)
results.append(run_one(disp, q_file, out, "SGM", troubleshooting))
if MODE_ZNCC in modes:
disp = base / ZNCC_DISP_FOLDER / "disparity.npy"
out = config.get_processing_step_dir(project, date, session, scan, ZNCC_PCL_FOLDER)
results.append(run_one(disp, q_file, out, "ZNCC", troubleshooting))
success = all(results)
print(f"[{'DONE' if success else 'PARTIAL'}] {session}/{scan}")
return success
def run_session(project, date, session, scan_arg, modes, troubleshooting=False):
if scan_arg:
scans = [scan_arg]
else:
scans = config.list_scan_dirs(project, date, session)
if not scans:
print(f"[WARN] No scan folders found in {project}/{date}/{session}")
return [], []
print(f"\n Session {session}: {len(scans)} scan(s) found")
failed = []
for scan in scans:
ok = run_scan(project, date, session, scan, modes, troubleshooting)
if not ok:
failed.append(f"{session}/{scan}")
return scans, failed
def main():
parser = argparse.ArgumentParser(
description="Point cloud pipeline — converts disparity maps to PLY using project folder structure"
)
parser.add_argument("--project", required=True, help="Project name (e.g. Olsen_wings)")
parser.add_argument("--date", required=True, help="Date string (e.g. 2026-05-12)")
parser.add_argument("--session", default=None, help="Session name (e.g. session1); omit to process ALL sessions on that date")
parser.add_argument("--scan", default=None, help="Single scan (e.g. Scan000001); omit to process all scans in the session")
parser.add_argument("--mode", default=MODE_BOTH, choices=[MODE_SGM, MODE_ZNCC, MODE_BOTH],
help="Which disparity source to convert: sgm, zncc, or both (default: both)")
parser.add_argument(
"--troubleshooting",
action="store_true",
help="Also save ASCII Point_cloud.txt (default: PLY only)",
)
args = parser.parse_args()
modes = {MODE_SGM, MODE_ZNCC} if args.mode == MODE_BOTH else {args.mode}
# Determine sessions to process
if args.session:
sessions = [args.session]
else:
sessions = config.list_session_dirs(args.project, args.date)
if not sessions:
print(f"No session folders found under {args.project}/{args.date}")
raise SystemExit(1)
print(f"Found {len(sessions)} session(s): {sessions}")
total_scans = 0
all_failed = []
for session in sessions:
scans, failed = run_session(
args.project, args.date, session, args.scan, modes, args.troubleshooting
)
total_scans += len(scans)
all_failed.extend(failed)
print(f"\n{'='*60}")
print(f"Finished: {total_scans - len(all_failed)}/{total_scans} scans succeeded.")
if all_failed:
print(f"Partial/failed: {all_failed}")
raise SystemExit(1)
if __name__ == "__main__":
main()