AsadUllah/Speckle-Scanner
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit: Speckle-Scanner 3D pipeline with setup README

Browse Source
This commit is contained in:
AsadUllah
2026-06-10 03:09:05 +05:00
commit 1765934846
375 changed files with 123081 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
09_coloring/README.md
+145
View File
@@ -0,0 +1,145 @@
# Point Cloud Colouring Pipeline
Colours 3D point clouds by projecting each point onto the calibrated RGB and IR
cameras and sampling pixel colours. Each output PLY carries **dual texture**:
RGB colours (from `rg_*.bmp`) and IR colours (from `ir_*.png`), stored as separate
per-vertex properties so both can be visualised in CloudCompare or any PLY viewer.
---
## Core module: `colour_ply.py`
| Function | Purpose |
|----------|---------|
| `load_ascii_ply(ply_path)` | Read ASCII PLY, return Nx3 float32 xyz array |
| `colorize_combined_pointcloud(pts, rgb_img, ir_img, lc_rgb_yaml, lc_ir_yaml, lc_rc_yaml, out_ply)` | Un-rotate points using R1 from stereo rectification, project to RGB + IR cameras, save dual-texture binary PLY |
| `save_master_ply(filename, points, rgb_colors, ir_colors)` | Write binary_little_endian PLY with x,y,z,red,green,blue,ir_red,ir_green,ir_blue |
---
## Folder structure assumed
```
~/Speckle-Scanner_Processing_data/
└── <project>/
└── <date>/
└── <session>/
├── params_link/
│ ├── lc-rc_stereo_cam_model.yaml ← stereoRectify R1 (un-rotate 3D points)
│ ├── lc-rg_stereo_cam_model.yaml ← project to RGB camera
│ └── lc-ir_stereo_cam_model.yaml ← project to IR camera
└── <ScanXXXXXX>/
├── 02_rect_images/
│ ├── ir_*.png ← one IR image (input)
│ └── rg_*.bmp ← one RGB image (input)
├── 05_sgm_pcl/
│ └── Point_cloud.ply ← SGM point cloud (input)
├── 06_zncc_pcl/
│ └── Point_cloud.ply ← ZNCC point cloud (input)
├── 07_sgm_pcl_col/
│ └── Point_cloud_colored.ply ← SGM coloured output (created)
└── 08_zncc_pcl_col/
└── Point_cloud_colored.ply ← ZNCC coloured output (created)
```
---
## Pipeline commands
```bash
cd ~/Speckle-Scanner/09_coloring
# Both SGM + ZNCC — all scans in a session
python run_coloring_pipeline.py \
--project Olsen_wings \
--date 2026-05-12 \
--session session1
# Both SGM + ZNCC — all sessions on a date (omit --session)
python run_coloring_pipeline.py \
--project Olsen_wings \
--date 2026-05-12
# Single scan, both modes
python run_coloring_pipeline.py \
--project Olsen_wings \
--date 2026-05-12 \
--session session1 \
--scan Scan000001
# Single scan, SGM only
python run_coloring_pipeline.py \
--project Olsen_wings \
--date 2026-05-12 \
--session session1 \
--scan Scan000001 \
--mode sgm
# Single scan, ZNCC only
python run_coloring_pipeline.py \
--project Olsen_wings \
--date 2026-05-12 \
--session session1 \
--scan Scan000001 \
--mode zncc
```
---
## Parameters
| Parameter | Default | Description |
|-------------|---------|-------------|
| `--project` | — | Project name (e.g. `Olsen_wings`) |
| `--date` | — | Date string (e.g. `2026-05-12`) |
| `--session` | all | Session name (e.g. `session1`); omit to process **all sessions** on that date |
| `--scan` | all | Single scan (e.g. `Scan000001`); omit to process all scans in the session |
| `--mode` | `both` | Which point cloud to colour: `sgm`, `zncc`, or `both` |
---
## What gets saved
| File | Description |
|------|-------------|
| `07_sgm_pcl_col/Point_cloud_colored.ply` | SGM cloud with RGB + IR dual texture (binary PLY) |
| `08_zncc_pcl_col/Point_cloud_colored.ply` | ZNCC cloud with RGB + IR dual texture (binary PLY) |
The output PLY has the following vertex properties:
```
x, y, z — 32-bit float (metres)
red, green, blue — 8-bit uchar (from rg_*.bmp camera)
ir_red, ir_green, ir_blue — 8-bit uchar (from ir_*.png camera)
```
To visualise the IR texture in CloudCompare, select the `ir_red/ir_green/ir_blue`
scalar fields or load the file and choose the IR colour attribute.
---
## Skip conditions
The runner prints `[SKIP]` and moves to the next scan if any required file is absent:
- `02_rect_images/ir_*.png` — no IR image found
- `02_rect_images/rg_*.bmp` — no RGB image found
- `params_link/lc-rc_stereo_cam_model.yaml` — missing RC YAML
- `params_link/lc-rg_stereo_cam_model.yaml` — missing RG YAML
- `params_link/lc-ir_stereo_cam_model.yaml` — missing IR YAML
- `05_sgm_pcl/Point_cloud.ply` — SGM cloud not generated yet (run `run_pcl_pipeline.py --mode sgm` first)
- `06_zncc_pcl/Point_cloud.ply` — ZNCC cloud not generated yet (run `run_pcl_pipeline.py --mode zncc` first)
---
## Dependencies
```bash
# This step only
pip install -r ~/Speckle-Scanner/09_coloring/requirements.txt
# Or install everything for the full pipeline
pip install -r ~/Speckle-Scanner/requirements.txt
```
Packages: `numpy`, `opencv-python`.
09_coloring/colour_ply.py
+137
View File
@@ -0,0 +1,137 @@
import cv2
import numpy as np
def load_ascii_ply(ply_path):
print(f"Loading point cloud from: {ply_path}")
with open(ply_path, 'r') as f:
lines = f.readlines()
header_end = 0
for i, line in enumerate(lines):
if line.strip() == "end_header":
header_end = i + 1
break
data = np.loadtxt(lines[header_end:])
points_3d = data[:, :3].astype(np.float32)
return points_3d
def save_master_ply(filename, points, rgb_colors, ir_colors):
print(f"Writing Master Binary PLY file to: {filename}...")
# Create a structured numpy array for fast binary writing
vertex_type = [
('x', 'f4'), ('y', 'f4'), ('z', 'f4'),
('red', 'u1'), ('green', 'u1'), ('blue', 'u1'),
('ir_red', 'u1'), ('ir_green', 'u1'), ('ir_blue', 'u1')
]
vertices_struct = np.empty(len(points), dtype=vertex_type)
vertices_struct['x'] = points[:, 0]
vertices_struct['y'] = points[:, 1]
vertices_struct['z'] = points[:, 2]
vertices_struct['red'] = rgb_colors[:, 0]
vertices_struct['green'] = rgb_colors[:, 1]
vertices_struct['blue'] = rgb_colors[:, 2]
vertices_struct['ir_red'] = ir_colors[:, 0]
vertices_struct['ir_green'] = ir_colors[:, 1]
vertices_struct['ir_blue'] = ir_colors[:, 2]
with open(filename, 'wb') as f:
header = (
"ply\n"
"format binary_little_endian 1.0\n"
f"element vertex {len(points)}\n"
"property float x\n"
"property float y\n"
"property float z\n"
"property uchar red\n"
"property uchar green\n"
"property uchar blue\n"
"property uchar ir_red\n"
"property uchar ir_green\n"
"property uchar ir_blue\n"
"end_header\n"
)
f.write(header.encode('ascii'))
f.write(vertices_struct.tobytes())
print("Done! Master PLY is ready for the client & CloudCompare.")
def colorize_combined_pointcloud(points_3d, rgb_img_path, ir_img_path, lc_rgb_yaml, lc_ir_yaml, lc_rc_yaml, output_ply_path):
print(f"\nLoading RGB scan image...")
rgb_img = cv2.imread(rgb_img_path)
if rgb_img.shape[:2] != (2044, 2040):
rgb_img = cv2.resize(rgb_img, (2040, 2044), interpolation=cv2.INTER_LINEAR)
print(f"Loading IR scan image...")
ir_img = cv2.imread(ir_img_path)
if ir_img.shape[:2] != (480, 640):
ir_img = cv2.resize(ir_img, (640, 480), interpolation=cv2.INTER_LINEAR)
# Rectify & Un-rotate 3D points
fs_rc = cv2.FileStorage(lc_rc_yaml, cv2.FILE_STORAGE_READ)
R1, _, _, _, _, _, _ = cv2.stereoRectify(
fs_rc.getNode("L_Intrinsic").mat(), fs_rc.getNode("L_Distortion").mat(),
fs_rc.getNode("R_Intrinsic").mat(), fs_rc.getNode("R_Distortion").mat(),
(1920, 1464), fs_rc.getNode("Rotation").mat(), fs_rc.getNode("Translation").mat(),
flags=cv2.CALIB_ZERO_DISPARITY, alpha=1
)
fs_rc.release()
points_3d_raw = np.dot(R1.T, points_3d.T).T
# Project to RGB camera
fs_rgb = cv2.FileStorage(lc_rgb_yaml, cv2.FILE_STORAGE_READ)
rvec_rgb, _ = cv2.Rodrigues(fs_rgb.getNode("Rotation").mat())
pixels_rgb, _ = cv2.projectPoints(points_3d_raw, rvec_rgb, fs_rgb.getNode("Translation").mat(), fs_rgb.getNode("R_Intrinsic").mat(), fs_rgb.getNode("R_Distortion").mat())
pixels_rgb = pixels_rgb.reshape(-1, 2)
fs_rgb.release()
# Project to IR camera
fs_ir = cv2.FileStorage(lc_ir_yaml, cv2.FILE_STORAGE_READ)
rvec_ir, _ = cv2.Rodrigues(fs_ir.getNode("Rotation").mat())
pixels_ir, _ = cv2.projectPoints(points_3d_raw, rvec_ir, fs_ir.getNode("Translation").mat(), fs_ir.getNode("R_Intrinsic").mat(), fs_ir.getNode("R_Distortion").mat())
pixels_ir = pixels_ir.reshape(-1, 2)
fs_ir.release()
points_list, rgb_list, ir_list = [], [], []
# Apply your tuned offsets
rgb_x, rgb_y = 30, 5
ir_x, ir_y = -22, 9
print("Extracting combined colors...")
for i in range(len(points_3d)):
x_r, y_r = int(np.round(pixels_rgb[i][0])) + rgb_x, int(np.round(pixels_rgb[i][1])) + rgb_y
x_i, y_i = int(np.round(pixels_ir[i][0])) + ir_x, int(np.round(pixels_ir[i][1])) + ir_y
# Keep only points visible to BOTH cameras
if (0 <= x_r < 2040 and 0 <= y_r < 2044) and (0 <= x_i < 640 and 0 <= y_i < 480):
b, g, r = rgb_img[y_r, x_r]
ir_b, ir_g, ir_r = ir_img[y_i, x_i]
points_list.append(points_3d[i])
rgb_list.append([r, g, b])
# Store true IR RGB values (0-255) instead of converting to normals
ir_list.append([ir_r, ir_g, ir_b])
print(f"Successfully encoded dual-textures to {len(points_list)} points!")
save_master_ply(output_ply_path, np.array(points_list), np.array(rgb_list), np.array(ir_list))
if __name__ == "__main__":
# Your exact file paths
input_cloud = r"/mnt/e/jost/color-maping(Asad)/input/Point_cloud.ply"
raw_rgb = r"/mnt/e/jost/color-maping(Asad)/input/rgb_1.bmp"
raw_ir = r"/mnt/e/jost/color-maping(Asad)/input/IR_scan_000001.png"
lc_rgb_yaml = r"/mnt/e/jost/color-maping(Asad)/calibration-results lc-rgb/stereo_cam_model.yaml"
lc_ir_yaml = r"/mnt/e/jost/color-maping(Asad)/calibration-results lc-ir/stereo_cam_model.yaml"
lc_rc_yaml = r"/mnt/e/jost/color-maping(Asad)/calibration-results lc-rc/stereo_cam_model.yaml"
# Save the Client Master PLY
out_master = r"/mnt/e/jost/color-maping(Asad)/output/CLIENT_MASTER_COMBINED.ply"
pts = load_ascii_ply(input_cloud)
colorize_combined_pointcloud(pts, raw_rgb, raw_ir, lc_rgb_yaml, lc_ir_yaml, lc_rc_yaml, out_master)
09_coloring/requirements.txt
+6
View File
@@ -0,0 +1,6 @@
# 09_coloring — Python dependencies
# Install: pip install -r requirements.txt
# Full pipeline (all steps): pip install -r ~/Speckle-Scanner/requirements.txt
numpy>=1.21
opencv-python>=4.8
09_coloring/run_coloring_pipeline.py
+188
View File
@@ -0,0 +1,188 @@
"""
Pipeline runner: point cloud → coloured point cloud.
Reads PLY point clouds from the project folder structure, colours them using
the session's rectified IR and RGB images plus stereo calibration YAML files,
and saves dual-texture binary PLY files to the output folders.
Supported modes:
sgm → 05_sgm_pcl/Point_cloud.ply → 07_sgm_pcl_col/
zncc → 06_zncc_pcl/Point_cloud.ply → 08_zncc_pcl_col/
both → runs both above (default)
YAML files are taken from each session's own params_link folder:
<processing_dir>/<project>/<date>/<session>/params_link/
lc-rc_stereo_cam_model.yaml ← stereoRectify R1 for un-rotating 3D points
lc-rg_stereo_cam_model.yaml ← project to RGB (rg_*) camera
lc-ir_stereo_cam_model.yaml ← project to IR (ir_*) camera
Images are taken from each scan's 02_rect_images/ folder:
ir_*.png ← one IR image (first match used)
rg_*.bmp ← one RGB image (first match used)
"""
import sys
import argparse
from pathlib import Path
# Resolve config.py from ~/Speckle-Scanner regardless of CWD
sys.path.insert(0, str(Path.home() / "Speckle-Scanner"))
# Resolve colour_ply.py from the same directory as this script
sys.path.insert(0, str(Path(__file__).parent))
import config
from colour_ply import load_ascii_ply, colorize_combined_pointcloud
MODE_SGM = "sgm"
MODE_ZNCC = "zncc"
MODE_BOTH = "both"
SGM_PCL_FOLDER = "05_sgm_pcl"
ZNCC_PCL_FOLDER = "06_zncc_pcl"
SGM_COL_FOLDER = "07_sgm_pcl_col"
ZNCC_COL_FOLDER = "08_zncc_pcl_col"
YAML_RC = "lc-rc_stereo_cam_model.yaml"
YAML_RG = "lc-rg_stereo_cam_model.yaml"
YAML_IR = "lc-ir_stereo_cam_model.yaml"
def find_image(directory, prefix, ext):
"""Return first file matching <prefix>*<ext> in directory, or None."""
matches = sorted(Path(directory).glob(f"{prefix}*{ext}"))
return matches[0] if matches else None
def run_one(ply_in, rgb_img, ir_img, lc_rg_yaml, lc_ir_yaml, lc_rc_yaml, out_dir, label):
"""Colour one PLY and save to out_dir/Point_cloud_colored.ply."""
if not ply_in.exists():
print(f" [{label}] SKIP — PLY not found: {ply_in}")
return False
out_dir.mkdir(parents=True, exist_ok=True)
out_ply = out_dir / "Point_cloud_colored.ply"
print(f" [{label}] {ply_in.parent.name}/Point_cloud.ply → {out_dir.name}/")
pts = load_ascii_ply(str(ply_in))
colorize_combined_pointcloud(
pts,
str(rgb_img),
str(ir_img),
str(lc_rg_yaml),
str(lc_ir_yaml),
str(lc_rc_yaml),
str(out_ply),
)
print(f" [{label}] Saved → {out_ply}")
return True
def run_scan(project, date, session, scan, modes):
base = config.PROCESSING_DIR / project / date / session / scan
rect_dir = base / "02_rect_images"
params = config.get_params_link_dir(project, date, session)
print(f"\n{'='*60}")
print(f"[SCAN] {session}/{scan}")
print(f"{'='*60}")
# -- Images (shared for both modes) --
ir_img = find_image(rect_dir, "ir_", ".png")
rgb_img = find_image(rect_dir, "rg_", ".bmp")
if ir_img is None:
print(f" [SKIP] No ir_*.png found in {rect_dir}")
return False
if rgb_img is None:
print(f" [SKIP] No rg_*.bmp found in {rect_dir}")
return False
# -- YAML files (shared for both modes) --
lc_rc_yaml = params / YAML_RC
lc_rg_yaml = params / YAML_RG
lc_ir_yaml = params / YAML_IR
for yaml_path in (lc_rc_yaml, lc_rg_yaml, lc_ir_yaml):
if not yaml_path.exists():
print(f" [SKIP] YAML not found: {yaml_path}")
return False
print(f" IR image : {ir_img.name}")
print(f" RGB image : {rgb_img.name}")
results = []
if MODE_SGM in modes:
ply_in = base / SGM_PCL_FOLDER / "Point_cloud.ply"
out_dir = config.get_processing_step_dir(project, date, session, scan, SGM_COL_FOLDER)
results.append(run_one(ply_in, rgb_img, ir_img, lc_rg_yaml, lc_ir_yaml, lc_rc_yaml, out_dir, "SGM"))
if MODE_ZNCC in modes:
ply_in = base / ZNCC_PCL_FOLDER / "Point_cloud.ply"
out_dir = config.get_processing_step_dir(project, date, session, scan, ZNCC_COL_FOLDER)
results.append(run_one(ply_in, rgb_img, ir_img, lc_rg_yaml, lc_ir_yaml, lc_rc_yaml, out_dir, "ZNCC"))
success = all(results)
print(f"[{'DONE' if success else 'PARTIAL'}] {session}/{scan}")
return success
def run_session(project, date, session, scan_arg, modes):
if scan_arg:
scans = [scan_arg]
else:
scans = config.list_scan_dirs(project, date, session)
if not scans:
print(f"[WARN] No scan folders found in {project}/{date}/{session}")
return [], []
print(f"\n Session {session}: {len(scans)} scan(s) found")
failed = []
for scan in scans:
ok = run_scan(project, date, session, scan, modes)
if not ok:
failed.append(f"{session}/{scan}")
return scans, failed
def main():
parser = argparse.ArgumentParser(
description="Point cloud colouring pipeline — colours PLY files with IR + RGB images"
)
parser.add_argument("--project", required=True, help="Project name (e.g. Olsen_wings)")
parser.add_argument("--date", required=True, help="Date string (e.g. 2026-05-12)")
parser.add_argument("--session", default=None, help="Session name (e.g. session1); omit to process ALL sessions on that date")
parser.add_argument("--scan", default=None, help="Single scan (e.g. Scan000001); omit to process all scans in the session")
parser.add_argument("--mode", default=MODE_BOTH, choices=[MODE_SGM, MODE_ZNCC, MODE_BOTH],
help="Which point cloud source to colour: sgm, zncc, or both (default: both)")
args = parser.parse_args()
modes = {MODE_SGM, MODE_ZNCC} if args.mode == MODE_BOTH else {args.mode}
if args.session:
sessions = [args.session]
else:
sessions = config.list_session_dirs(args.project, args.date)
if not sessions:
print(f"No session folders found under {args.project}/{args.date}")
raise SystemExit(1)
print(f"Found {len(sessions)} session(s): {sessions}")
total_scans = 0
all_failed = []
for session in sessions:
scans, failed = run_session(
args.project, args.date, session, args.scan, modes
)
total_scans += len(scans)
all_failed.extend(failed)
print(f"\n{'='*60}")
print(f"Finished: {total_scans - len(all_failed)}/{total_scans} scans succeeded.")
if all_failed:
print(f"Partial/failed: {all_failed}")
raise SystemExit(1)
if __name__ == "__main__":
main()