395 lines
14 KiB
Markdown
395 lines
14 KiB
Markdown
# Speckle-Scanner — 3D Scanning Pipeline
|
||
|
||
A modular pipeline for stereo 3D scanning: camera calibration → image rectification → disparity computation → point cloud generation → point cloud colouring.
|
||
|
||
---
|
||
|
||
## Quick start — clone and prepare your machine
|
||
|
||
### 1. Clone into home
|
||
|
||
The pipeline expects this repository at **`~/Speckle-Scanner`**. All paths in `config.py` are anchored to your home directory.
|
||
|
||
```bash
|
||
cd ~
|
||
git clone https://gitea.subseascanning.com/AsadUllah/Speckle-Scanner.git
|
||
```
|
||
|
||
This creates `~/Speckle-Scanner/` with the code only — **no scan or calibration data** is included in the repo.
|
||
|
||
### 2. Create the required data folders (siblings of the repo)
|
||
|
||
Three folders must exist **next to** `Speckle-Scanner` under `~/`. They are **not** part of the git repository; you create or copy data into them locally.
|
||
|
||
```bash
|
||
mkdir -p ~/3D-Scans ~/Calib-data
|
||
# ~/Speckle-Scanner_Processing_data/ is created automatically by rectification (step 2)
|
||
```
|
||
|
||
| Folder | Required? | Purpose | You provide |
|
||
|--------|-----------|---------|-------------|
|
||
| `~/Speckle-Scanner/` | yes (clone) | Pipeline source code, scripts, READMEs | `git clone` |
|
||
| `~/3D-Scans/` | yes | Raw stereo scan images | Copy/mount your project data |
|
||
| `~/Calib-data/` | yes | Calibration chessboard images | Copy/mount calibration sessions |
|
||
| `~/Speckle-Scanner_Processing_data/` | auto | Rectified images, disparity, point clouds | Created by step 2 (rectification) |
|
||
|
||
**Layout after clone:**
|
||
|
||
```
|
||
~/
|
||
├── Speckle-Scanner/ ← git clone (this repo)
|
||
├── 3D-Scans/ ← you create + add raw scans
|
||
│ └── <project>/<date>/<session>/<scan>/01_raw_images/
|
||
├── Calib-data/ ← you create + add calibration images
|
||
│ └── <project>/<date>/<calib_name>/{lc,rc,rg,ir}/ + params/
|
||
└── Speckle-Scanner_Processing_data/ ← auto-created on first rectification run
|
||
└── <project>/<date>/<session>/...
|
||
```
|
||
|
||
> **Do not move the repo out of `~/`.** Scripts resolve paths via `config.py` using `Path.home() / "Speckle-Scanner"`. Cloning elsewhere will break path resolution unless you change `config.py`.
|
||
|
||
### 3. Python environment and dependencies
|
||
|
||
```bash
|
||
conda create -n speckle python=3.9 -y
|
||
conda activate speckle
|
||
|
||
pip install -r ~/Speckle-Scanner/requirements.txt
|
||
```
|
||
|
||
Per-step installs are also available (see [Per-step requirements](#per-step-requirementstxt) below).
|
||
|
||
### 4. Build libSGM (GPU disparity — one-time per machine)
|
||
|
||
Requires NVIDIA CUDA + CMake ≥ 3.18. The binary is **not** committed to git; rebuild on each machine.
|
||
|
||
```bash
|
||
cd ~/Speckle-Scanner/05_disparity/libsgm
|
||
mkdir -p build && cd build
|
||
cmake .. -DENABLE_SAMPLES=on
|
||
make stereosgm_new -j4
|
||
```
|
||
|
||
Verify:
|
||
|
||
```bash
|
||
ls ~/Speckle-Scanner/05_disparity/libsgm/build/sample/stereosgm_new
|
||
```
|
||
|
||
### 5. Copy your data and run the pipeline
|
||
|
||
**Calibration data** → `~/Calib-data/<project>/<date>/<calib_name>/`
|
||
Folders: `lc/`, `rc/`, `rg/` (or `rgb/`), `ir/` with chessboard images.
|
||
|
||
**Raw scans** → `~/3D-Scans/<project>/<date>/<session>/<scan>/01_raw_images/`
|
||
Files: `lc_ts*.png`, `rc_ts*.png`, `ir_*.png`, `rg_*.png`, etc.
|
||
|
||
Then run the pipeline (replace names with yours):
|
||
|
||
```bash
|
||
# Step 1 — calibrate
|
||
cd ~/Speckle-Scanner/02_Calibration
|
||
python main.py --project Olsen_wings --date 2026-05-12 --calib_name calib1
|
||
|
||
# Step 2 — rectify (creates ~/Speckle-Scanner_Processing_data/...)
|
||
cd ~/Speckle-Scanner/04_Rectification
|
||
python main.py --project Olsen_wings --date 2026-05-12 --calib_name calib1
|
||
|
||
# Steps 3–5 — disparity, point cloud, colouring (see sections below)
|
||
```
|
||
|
||
**Sanity check after setup:**
|
||
|
||
```bash
|
||
ls ~/Calib-data/ # your calibration projects
|
||
ls ~/3D-Scans/ # your raw scan projects
|
||
ls ~/Speckle-Scanner/config.py # repo present
|
||
```
|
||
|
||
---
|
||
|
||
## System requirements
|
||
|
||
| Requirement | Version |
|
||
|-------------|---------|
|
||
| OS | Ubuntu 20.04 / 22.04 (Linux) |
|
||
| Python | 3.9 or newer |
|
||
| NVIDIA GPU + CUDA | 11.x or 12.x (required for libSGM; optional for GPU ZNCC) |
|
||
| CMake | ≥ 3.18 (for libSGM build) |
|
||
| Conda | Recommended for environment management |
|
||
|
||
---
|
||
|
||
## Directory layout — where folders live on the machine
|
||
|
||
All four top-level directories sit directly inside the home folder (`~/`). Only **`Speckle-Scanner`** comes from git; the others are local data or auto-generated output.
|
||
|
||
```
|
||
~/
|
||
├── Speckle-Scanner/ ← clone this repository here
|
||
├── 3D-Scans/ ← raw scan images (copy/mount data here)
|
||
├── Calib-data/ ← calibration session images (copy here)
|
||
└── Speckle-Scanner_Processing_data/ ← auto-created by the rectification step
|
||
```
|
||
|
||
**Never move Speckle-Scanner out of `~/`.** All path resolution is anchored to `~/` via `config.py` — no paths ever need manual editing.
|
||
|
||
### Folder structure inside each directory
|
||
|
||
```
|
||
~/3D-Scans/
|
||
└── <project>/
|
||
└── <date>/
|
||
└── <session>/
|
||
└── <scan>/
|
||
└── 01_raw_images/ ← lc_ts*.png rc_ts*.png ir_*.png rg_*.png
|
||
|
||
~/Calib-data/
|
||
└── <project>/
|
||
└── <date>/
|
||
└── <calib_name>/ ← e.g. calib1
|
||
├── lc/ ← left camera calibration images
|
||
├── rc/ rg/ ir/ ← right camera images
|
||
└── params/ ← calibration outputs (auto-created)
|
||
|
||
~/Speckle-Scanner_Processing_data/ ← created automatically by step 2 (rectification)
|
||
└── <project>/
|
||
└── <date>/
|
||
└── <session>/
|
||
├── params_link/ ← calibration files copied here once per session
|
||
└── <scan>/
|
||
├── 01_raw_images/
|
||
├── 02_rect_images/
|
||
├── 03_sgm_disp_map/
|
||
├── 04_zncc_disp_map/
|
||
├── 05_sgm_pcl/
|
||
├── 06_zncc_pcl/
|
||
├── 07_sgm_pcl_col/
|
||
└── 08_zncc_pcl_col/
|
||
```
|
||
|
||
---
|
||
|
||
## config.py — how path resolution works
|
||
|
||
`config.py` (at the root of this repo) is the single source of truth for all directory paths. It anchors everything to `Path.home()`:
|
||
|
||
```python
|
||
HOME_DIR = Path.home()
|
||
SOURCE_CODE_DIR = HOME_DIR / "Speckle-Scanner"
|
||
RAW_DATA_DIR = HOME_DIR / "3D-Scans"
|
||
CALIB_DATA_DIR = HOME_DIR / "Calib-data"
|
||
PROCESSING_DIR = HOME_DIR / "Speckle-Scanner_Processing_data"
|
||
```
|
||
|
||
Every pipeline script imports `config` and uses these variables — no script ever has a hardcoded path. To move the pipeline to a new machine, just clone the repo to `~/Speckle-Scanner/` and copy the data folders.
|
||
|
||
### Per-step `requirements.txt`
|
||
|
||
Each pipeline step has its own `requirements.txt` for minimal installs:
|
||
|
||
| Step | File |
|
||
|------|------|
|
||
| Calibration | `02_Calibration/requirements.txt` |
|
||
| Rectification | `04_Rectification/requirements.txt` |
|
||
| SGM disparity | `05_disparity/libsgm/requirements.txt` (build deps; no pip) |
|
||
| ZNCC disparity | `05_disparity/zncc/requirements.txt` |
|
||
| Point cloud | `06_Pointcloud/requirements.txt` |
|
||
| Colouring | `09_coloring/requirements.txt` |
|
||
|
||
The root `requirements.txt` is the **merged union** of all Python steps — use it when setting up one conda/venv for the full pipeline.
|
||
|
||
---
|
||
|
||
## Setup on a new machine (summary)
|
||
|
||
If you already followed [Quick start](#quick-start--clone-and-prepare-your-machine) above, you can skip this block. It repeats the same steps in one place:
|
||
|
||
```bash
|
||
cd ~
|
||
git clone https://gitea.subseascanning.com/AsadUllah/Speckle-Scanner.git
|
||
mkdir -p ~/3D-Scans ~/Calib-data
|
||
|
||
conda create -n speckle python=3.9 -y
|
||
conda activate speckle
|
||
pip install -r ~/Speckle-Scanner/requirements.txt
|
||
|
||
# GPU ZNCC (optional — match your CUDA version)
|
||
# pip install cupy-cuda11x # CUDA 11.x
|
||
# pip install cupy-cuda12x # CUDA 12.x
|
||
|
||
cd ~/Speckle-Scanner/05_disparity/libsgm
|
||
mkdir -p build && cd build
|
||
cmake .. -DENABLE_SAMPLES=on
|
||
make stereosgm_new -j4
|
||
```
|
||
|
||
Copy calibration images to `~/Calib-data/` and raw scans to `~/3D-Scans/`, then run the pipeline steps below.
|
||
|
||
> **Note:** The libSGM binary is machine-specific (compiled CUDA C++). It must be rebuilt on every new machine. The Python code is fully portable.
|
||
|
||
---
|
||
|
||
## Full pipeline — step by step
|
||
|
||
Replace `Olsen_wings`, `2026-05-12`, `calib1` with your actual project, date, and calibration folder name.
|
||
|
||
### Step 1 — Calibrate
|
||
|
||
Two-step calibration (detect features → JSON, then mono + stereo). One command calibrates **lc vs rc, rg, and ir**:
|
||
|
||
```bash
|
||
cd ~/Speckle-Scanner/02_Calibration
|
||
|
||
python main.py \
|
||
--project Olsen_wings --date 2026-05-12 --calib_name calib1 \
|
||
--chessboard_size 8,7 --square_size 0.045
|
||
```
|
||
|
||
Or run steps separately: `detect_features.py` then `calibrate.py`. Results go to `~/Calib-data/<project>/<date>/<calib_name>/params/` (`lc-rc_*`, `lc-rg_*`, `lc-ir_*`).
|
||
|
||
See [02_Calibration/README.md](02_Calibration/README.md) for all options.
|
||
|
||
---
|
||
|
||
### Step 2 — Rectify
|
||
|
||
Reads raw scans from `~/3D-Scans/`, applies calibration, and creates the full `Speckle-Scanner_Processing_data/` structure (including `params_link/`, `01_raw_images/`, `02_rect_images/`).
|
||
|
||
```bash
|
||
cd ~/Speckle-Scanner/04_Rectification
|
||
|
||
# All sessions under a date
|
||
python main.py --project Olsen_wings --date 2026-05-12 --calib_name calib1
|
||
|
||
# Single session only
|
||
python main.py --project Olsen_wings --date 2026-05-12 --calib_name calib1 --session session1
|
||
```
|
||
|
||
See [04_Rectification/README.md](04_Rectification/README.md) for all options.
|
||
|
||
---
|
||
|
||
### Step 3 — Disparity
|
||
|
||
Reads rectified images from `02_rect_images/`. Choose SGM, ZNCC, or run both.
|
||
|
||
**SGM (libSGM — CUDA C++):**
|
||
```bash
|
||
cd ~/Speckle-Scanner/05_disparity/libsgm
|
||
|
||
python run_sgm_pipeline.py --project Olsen_wings --date 2026-05-12
|
||
# Single session:
|
||
python run_sgm_pipeline.py --project Olsen_wings --date 2026-05-12 --session session1
|
||
# Single scan:
|
||
python run_sgm_pipeline.py --project Olsen_wings --date 2026-05-12 --session session1 --scan Scan000001
|
||
```
|
||
|
||
**ZNCC (CPU/GPU — Python):**
|
||
```bash
|
||
cd ~/Speckle-Scanner/05_disparity/zncc
|
||
|
||
python run_zncc_pipeline.py --project Olsen_wings --date 2026-05-12
|
||
# Single scan — use only the last 3 image pairs (highest timestamps):
|
||
python run_zncc_pipeline.py \
|
||
--project Olsen_wings --date 2026-05-12 --session session1 --scan Scan000001 \
|
||
--num_images 3 --window_size 7 --zncc_threshold 0.5
|
||
```
|
||
|
||
See [05_disparity/libsgm/README.md](05_disparity/libsgm/README.md) and [05_disparity/zncc/Readme.md](05_disparity/zncc/Readme.md) for all options.
|
||
|
||
---
|
||
|
||
### Step 4 — Point Cloud
|
||
|
||
Converts disparity maps to 3D point clouds (`.ply` by default; `.txt` with `--troubleshooting`).
|
||
|
||
```bash
|
||
cd ~/Speckle-Scanner/06_Pointcloud
|
||
|
||
# Both SGM and ZNCC (PLY only)
|
||
python run_pcl_pipeline.py --project Olsen_wings --date 2026-05-12
|
||
|
||
# SGM only / ZNCC only
|
||
python run_pcl_pipeline.py --project Olsen_wings --date 2026-05-12 --mode sgm
|
||
python run_pcl_pipeline.py --project Olsen_wings --date 2026-05-12 --mode zncc
|
||
|
||
# Single scan
|
||
python run_pcl_pipeline.py \
|
||
--project Olsen_wings --date 2026-05-12 --session session1 --scan Scan000001
|
||
|
||
# Also save ASCII .txt exports
|
||
python run_pcl_pipeline.py \
|
||
--project Olsen_wings --date 2026-05-12 --troubleshooting
|
||
```
|
||
|
||
See [06_Pointcloud/README.md](06_Pointcloud/README.md) for all options.
|
||
|
||
---
|
||
|
||
### Step 5 — Colour Point Cloud
|
||
|
||
Projects IR and RGB camera images onto the point cloud. Saves dual-texture binary PLY (RGB + IR channels).
|
||
|
||
```bash
|
||
cd ~/Speckle-Scanner/09_coloring
|
||
|
||
# Both SGM and ZNCC point clouds
|
||
python run_coloring_pipeline.py --project Olsen_wings --date 2026-05-12
|
||
|
||
# SGM only / ZNCC only
|
||
python run_coloring_pipeline.py --project Olsen_wings --date 2026-05-12 --mode sgm
|
||
python run_coloring_pipeline.py --project Olsen_wings --date 2026-05-12 --mode zncc
|
||
|
||
# Single scan
|
||
python run_coloring_pipeline.py \
|
||
--project Olsen_wings --date 2026-05-12 --session session1 --scan Scan000001
|
||
```
|
||
|
||
See [09_coloring/README.md](09_coloring/README.md) for all options.
|
||
|
||
---
|
||
|
||
## Output files per scan (summary)
|
||
|
||
| Folder | Contents | Created by |
|
||
|--------|----------|-----------|
|
||
| `01_raw_images/` | Copy of source images | Step 2 |
|
||
| `02_rect_images/` | Rectified `lc_*`, `rc_*`, `rg_*`, `ir_*` images | Step 2 |
|
||
| `03_sgm_disp_map/` | `disparity.xml`, `disparity_color.png` | Step 3 SGM |
|
||
| `04_zncc_disp_map/` | `disparity.npy`, colorbar PNGs | Step 3 ZNCC |
|
||
| `05_sgm_pcl/` | `Point_cloud.ply` (+ `.txt` with `--troubleshooting`) | Step 4 |
|
||
| `06_zncc_pcl/` | `Point_cloud.ply` (+ `.txt` with `--troubleshooting`) | Step 4 |
|
||
| `07_sgm_pcl_col/` | `Point_cloud_colored.ply` (RGB + IR) | Step 5 |
|
||
| `08_zncc_pcl_col/` | `Point_cloud_colored.ply` (RGB + IR) | Step 5 |
|
||
|
||
---
|
||
|
||
## Common `--session` and `--scan` options
|
||
|
||
All step 3–5 pipeline runners share the same scope arguments:
|
||
|
||
| Argument | Default | Effect |
|
||
|----------|---------|--------|
|
||
| `--session` | (omit) | Process **all sessions** under the date |
|
||
| `--session session1` | — | Process only `session1` |
|
||
| `--scan` | (omit) | Process all scans in the session |
|
||
| `--scan Scan000001` | — | Process only `Scan000001` |
|
||
|
||
---
|
||
|
||
## Quick check after setup
|
||
|
||
```bash
|
||
# Verify calibration outputs
|
||
ls ~/Calib-data/Olsen_wings/2026-05-12/calib1/params/
|
||
# Expected: lc-rc_stereo_cam_model.yaml lc-rc_Q.cvstore lc-rg_* lc-ir_*
|
||
|
||
# Verify rectification created the processing structure
|
||
ls ~/Speckle-Scanner_Processing_data/Olsen_wings/2026-05-12/session1/params_link/
|
||
ls ~/Speckle-Scanner_Processing_data/Olsen_wings/2026-05-12/session1/Scan000001/02_rect_images/ | head
|
||
|
||
# Verify libSGM binary
|
||
~/Speckle-Scanner/05_disparity/libsgm/build/sample/stereosgm_new --help
|
||
```
|