# Speckle-Scanner — 3D Scanning Pipeline

A modular pipeline for stereo 3D scanning: camera calibration → image rectification → disparity computation → point cloud generation → point cloud colouring.

---

## Quick start — clone and prepare your machine

### 1. Clone into home

The pipeline expects this repository at **`~/Speckle-Scanner`**. All paths in `config.py` are anchored to your home directory.

```bash
cd ~
git clone https://gitea.subseascanning.com/AsadUllah/Speckle-Scanner.git
```

This creates `~/Speckle-Scanner/` with the code only — **no scan or calibration data** is included in the repo.

### 2. Create the required data folders (siblings of the repo)

Three folders must exist **next to** `Speckle-Scanner` under `~/`. They are **not** part of the git repository; you create or copy data into them locally.

```bash
mkdir -p ~/3D-Scans ~/Calib-data
# ~/Speckle-Scanner_Processing_data/ is created automatically by rectification (step 2)
```

| Folder | Required? | Purpose | You provide |
|--------|-----------|---------|-------------|
| `~/Speckle-Scanner/` | yes (clone) | Pipeline source code, scripts, READMEs | `git clone` |
| `~/3D-Scans/` | yes | Raw stereo scan images | Copy/mount your project data |
| `~/Calib-data/` | yes | Calibration chessboard images | Copy/mount calibration sessions |
| `~/Speckle-Scanner_Processing_data/` | auto | Rectified images, disparity, point clouds | Created by step 2 (rectification) |

**Layout after clone:**

```
~/
├── Speckle-Scanner/                 ← git clone (this repo)
├── 3D-Scans/                        ← you create + add raw scans
│   └── <project>/<date>/<session>/<scan>/01_raw_images/
├── Calib-data/                      ← you create + add calibration images
│   └── <project>/<date>/<calib_name>/{lc,rc,rg,ir}/ + params/
└── Speckle-Scanner_Processing_data/ ← auto-created on first rectification run
    └── <project>/<date>/<session>/...
```

> **Do not move the repo out of `~/`.** Scripts resolve paths via `config.py` using `Path.home() / "Speckle-Scanner"`. Cloning elsewhere will break path resolution unless you change `config.py`.

### 3. Python environment and dependencies

```bash
conda create -n speckle python=3.9 -y
conda activate speckle

pip install -r ~/Speckle-Scanner/requirements.txt
```

Per-step installs are also available (see [Per-step requirements](#per-step-requirementstxt) below).

### 4. Build libSGM (GPU disparity — one-time per machine)

Requires NVIDIA CUDA + CMake ≥ 3.18. The binary is **not** committed to git; rebuild on each machine.

```bash
cd ~/Speckle-Scanner/05_disparity/libsgm
mkdir -p build && cd build
cmake .. -DENABLE_SAMPLES=on
make stereosgm_new -j4
```

Verify:

```bash
ls ~/Speckle-Scanner/05_disparity/libsgm/build/sample/stereosgm_new
```

### 5. Copy your data and run the pipeline

**Calibration data** → `~/Calib-data/<project>/<date>/<calib_name>/`  
Folders: `lc/`, `rc/`, `rg/` (or `rgb/`), `ir/` with chessboard images.

**Raw scans** → `~/3D-Scans/<project>/<date>/<session>/<scan>/01_raw_images/`  
Files: `lc_ts*.png`, `rc_ts*.png`, `ir_*.png`, `rg_*.png`, etc.

Then run the pipeline (replace names with yours):

```bash
# Step 1 — calibrate
cd ~/Speckle-Scanner/02_Calibration
python main.py --project Olsen_wings --date 2026-05-12 --calib_name calib1

# Step 2 — rectify (creates ~/Speckle-Scanner_Processing_data/...)
cd ~/Speckle-Scanner/04_Rectification
python main.py --project Olsen_wings --date 2026-05-12 --calib_name calib1

# Steps 3–5 — disparity, point cloud, colouring (see sections below)
```

**Sanity check after setup:**

```bash
ls ~/Calib-data/                    # your calibration projects
ls ~/3D-Scans/                      # your raw scan projects
ls ~/Speckle-Scanner/config.py      # repo present
```

---

## System requirements

| Requirement | Version |
|-------------|---------|
| OS | Ubuntu 20.04 / 22.04 (Linux) |
| Python | 3.9 or newer |
| NVIDIA GPU + CUDA | 11.x or 12.x (required for libSGM; optional for GPU ZNCC) |
| CMake | ≥ 3.18 (for libSGM build) |
| Conda | Recommended for environment management |

---

## Directory layout — where folders live on the machine

All four top-level directories sit directly inside the home folder (`~/`). Only **`Speckle-Scanner`** comes from git; the others are local data or auto-generated output.

```
~/
├── Speckle-Scanner/                       ← clone this repository here
├── 3D-Scans/                              ← raw scan images  (copy/mount data here)
├── Calib-data/                            ← calibration session images (copy here)
└── Speckle-Scanner_Processing_data/       ← auto-created by the rectification step
```

**Never move Speckle-Scanner out of `~/`.** All path resolution is anchored to `~/` via `config.py` — no paths ever need manual editing.

### Folder structure inside each directory

```
~/3D-Scans/
└── <project>/
    └── <date>/
        └── <session>/
            └── <scan>/
                └── 01_raw_images/        ← lc_ts*.png  rc_ts*.png  ir_*.png  rg_*.png

~/Calib-data/
└── <project>/
    └── <date>/
        └── <calib_name>/                 ← e.g. calib1
            ├── lc/                       ← left camera calibration images
            ├── rc/  rg/  ir/             ← right camera images
            └── params/                   ← calibration outputs (auto-created)

~/Speckle-Scanner_Processing_data/        ← created automatically by step 2 (rectification)
└── <project>/
    └── <date>/
        └── <session>/
            ├── params_link/              ← calibration files copied here once per session
            └── <scan>/
                ├── 01_raw_images/
                ├── 02_rect_images/
                ├── 03_sgm_disp_map/
                ├── 04_zncc_disp_map/
                ├── 05_sgm_pcl/
                ├── 06_zncc_pcl/
                ├── 07_sgm_pcl_col/
                └── 08_zncc_pcl_col/
```

---

## config.py — how path resolution works

`config.py` (at the root of this repo) is the single source of truth for all directory paths. It anchors everything to `Path.home()`:

```python
HOME_DIR       = Path.home()
SOURCE_CODE_DIR = HOME_DIR / "Speckle-Scanner"
RAW_DATA_DIR    = HOME_DIR / "3D-Scans"
CALIB_DATA_DIR  = HOME_DIR / "Calib-data"
PROCESSING_DIR  = HOME_DIR / "Speckle-Scanner_Processing_data"
```

Every pipeline script imports `config` and uses these variables — no script ever has a hardcoded path. To move the pipeline to a new machine, just clone the repo to `~/Speckle-Scanner/` and copy the data folders.

### Per-step `requirements.txt`

Each pipeline step has its own `requirements.txt` for minimal installs:

| Step | File |
|------|------|
| Calibration | `02_Calibration/requirements.txt` |
| Rectification | `04_Rectification/requirements.txt` |
| SGM disparity | `05_disparity/libsgm/requirements.txt` (build deps; no pip) |
| ZNCC disparity | `05_disparity/zncc/requirements.txt` |
| Point cloud | `06_Pointcloud/requirements.txt` |
| Colouring | `09_coloring/requirements.txt` |

The root `requirements.txt` is the **merged union** of all Python steps — use it when setting up one conda/venv for the full pipeline.

---

## Setup on a new machine (summary)

If you already followed [Quick start](#quick-start--clone-and-prepare-your-machine) above, you can skip this block. It repeats the same steps in one place:

```bash
cd ~
git clone https://gitea.subseascanning.com/AsadUllah/Speckle-Scanner.git
mkdir -p ~/3D-Scans ~/Calib-data

conda create -n speckle python=3.9 -y
conda activate speckle
pip install -r ~/Speckle-Scanner/requirements.txt

# GPU ZNCC (optional — match your CUDA version)
# pip install cupy-cuda11x   # CUDA 11.x
# pip install cupy-cuda12x   # CUDA 12.x

cd ~/Speckle-Scanner/05_disparity/libsgm
mkdir -p build && cd build
cmake .. -DENABLE_SAMPLES=on
make stereosgm_new -j4
```

Copy calibration images to `~/Calib-data/` and raw scans to `~/3D-Scans/`, then run the pipeline steps below.

> **Note:** The libSGM binary is machine-specific (compiled CUDA C++). It must be rebuilt on every new machine. The Python code is fully portable.

---

## Full pipeline — step by step

Replace `Olsen_wings`, `2026-05-12`, `calib1` with your actual project, date, and calibration folder name.

### Step 1 — Calibrate

Two-step calibration (detect features → JSON, then mono + stereo). One command calibrates **lc vs rc, rg, and ir**:

```bash
cd ~/Speckle-Scanner/02_Calibration

python main.py \
  --project Olsen_wings --date 2026-05-12 --calib_name calib1 \
  --chessboard_size 8,7 --square_size 0.045
```

Or run steps separately: `detect_features.py` then `calibrate.py`. Results go to `~/Calib-data/<project>/<date>/<calib_name>/params/` (`lc-rc_*`, `lc-rg_*`, `lc-ir_*`).

See [02_Calibration/README.md](02_Calibration/README.md) for all options.

---

### Step 2 — Rectify

Reads raw scans from `~/3D-Scans/`, applies calibration, and creates the full `Speckle-Scanner_Processing_data/` structure (including `params_link/`, `01_raw_images/`, `02_rect_images/`).

```bash
cd ~/Speckle-Scanner/04_Rectification

# All sessions under a date
python main.py --project Olsen_wings --date 2026-05-12 --calib_name calib1

# Single session only
python main.py --project Olsen_wings --date 2026-05-12 --calib_name calib1 --session session1
```

See [04_Rectification/README.md](04_Rectification/README.md) for all options.

---

### Step 3 — Disparity

Reads rectified images from `02_rect_images/`. Choose SGM, ZNCC, or run both.

**SGM (libSGM — CUDA C++):**
```bash
cd ~/Speckle-Scanner/05_disparity/libsgm

python run_sgm_pipeline.py --project Olsen_wings --date 2026-05-12
# Single session:
python run_sgm_pipeline.py --project Olsen_wings --date 2026-05-12 --session session1
# Single scan:
python run_sgm_pipeline.py --project Olsen_wings --date 2026-05-12 --session session1 --scan Scan000001
```

**ZNCC (CPU/GPU — Python):**
```bash
cd ~/Speckle-Scanner/05_disparity/zncc

python run_zncc_pipeline.py --project Olsen_wings --date 2026-05-12
# Single scan — use only the last 3 image pairs (highest timestamps):
python run_zncc_pipeline.py \
  --project Olsen_wings --date 2026-05-12 --session session1 --scan Scan000001 \
  --num_images 3 --window_size 7 --zncc_threshold 0.5
```

See [05_disparity/libsgm/README.md](05_disparity/libsgm/README.md) and [05_disparity/zncc/Readme.md](05_disparity/zncc/Readme.md) for all options.

---

### Step 4 — Point Cloud

Converts disparity maps to 3D point clouds (`.ply` by default; `.txt` with `--troubleshooting`).

```bash
cd ~/Speckle-Scanner/06_Pointcloud

# Both SGM and ZNCC (PLY only)
python run_pcl_pipeline.py --project Olsen_wings --date 2026-05-12

# SGM only / ZNCC only
python run_pcl_pipeline.py --project Olsen_wings --date 2026-05-12 --mode sgm
python run_pcl_pipeline.py --project Olsen_wings --date 2026-05-12 --mode zncc

# Single scan
python run_pcl_pipeline.py \
  --project Olsen_wings --date 2026-05-12 --session session1 --scan Scan000001

# Also save ASCII .txt exports
python run_pcl_pipeline.py \
  --project Olsen_wings --date 2026-05-12 --troubleshooting
```

See [06_Pointcloud/README.md](06_Pointcloud/README.md) for all options.

---

### Step 5 — Colour Point Cloud

Projects IR and RGB camera images onto the point cloud. Saves dual-texture binary PLY (RGB + IR channels).

```bash
cd ~/Speckle-Scanner/09_coloring

# Both SGM and ZNCC point clouds
python run_coloring_pipeline.py --project Olsen_wings --date 2026-05-12

# SGM only / ZNCC only
python run_coloring_pipeline.py --project Olsen_wings --date 2026-05-12 --mode sgm
python run_coloring_pipeline.py --project Olsen_wings --date 2026-05-12 --mode zncc

# Single scan
python run_coloring_pipeline.py \
  --project Olsen_wings --date 2026-05-12 --session session1 --scan Scan000001
```

See [09_coloring/README.md](09_coloring/README.md) for all options.

---

## Output files per scan (summary)

| Folder | Contents | Created by |
|--------|----------|-----------|
| `01_raw_images/` | Copy of source images | Step 2 |
| `02_rect_images/` | Rectified `lc_*`, `rc_*`, `rg_*`, `ir_*` images | Step 2 |
| `03_sgm_disp_map/` | `disparity.xml`, `disparity_color.png` | Step 3 SGM |
| `04_zncc_disp_map/` | `disparity.npy`, colorbar PNGs | Step 3 ZNCC |
| `05_sgm_pcl/` | `Point_cloud.ply` (+ `.txt` with `--troubleshooting`) | Step 4 |
| `06_zncc_pcl/` | `Point_cloud.ply` (+ `.txt` with `--troubleshooting`) | Step 4 |
| `07_sgm_pcl_col/` | `Point_cloud_colored.ply` (RGB + IR) | Step 5 |
| `08_zncc_pcl_col/` | `Point_cloud_colored.ply` (RGB + IR) | Step 5 |

---

## Common `--session` and `--scan` options

All step 3–5 pipeline runners share the same scope arguments:

| Argument | Default | Effect |
|----------|---------|--------|
| `--session` | (omit) | Process **all sessions** under the date |
| `--session session1` | — | Process only `session1` |
| `--scan` | (omit) | Process all scans in the session |
| `--scan Scan000001` | — | Process only `Scan000001` |

---

## Quick check after setup

```bash
# Verify calibration outputs
ls ~/Calib-data/Olsen_wings/2026-05-12/calib1/params/
# Expected: lc-rc_stereo_cam_model.yaml  lc-rc_Q.cvstore  lc-rg_*  lc-ir_*

# Verify rectification created the processing structure
ls ~/Speckle-Scanner_Processing_data/Olsen_wings/2026-05-12/session1/params_link/
ls ~/Speckle-Scanner_Processing_data/Olsen_wings/2026-05-12/session1/Scan000001/02_rect_images/ | head

# Verify libSGM binary
~/Speckle-Scanner/05_disparity/libsgm/build/sample/stereosgm_new --help
```