Speckle-Scanner/02_Calibration/README.md

# 02 Calibration

Two-step calibration pipeline:

| Step | Script | What it does |
|------|--------|--------------|
| **1. Detection** | `detect_features.py` | Chessboard corners / IR ellipses → **JSON next to each image** |
| **2. Calibration** | `calibrate.py` | Mono intrinsics per camera + stereo **lc vs rc/rg/ir** |

`main.py` runs both steps by default (`--step all`).

---

## Troubleshooting flag

All calibration scripts accept `--troubleshooting` (default: **off**).

| `--troubleshooting` | Logs | Disk output |
|---------------------|------|-------------|
| **False** (default) | Minimal summary per camera / stereo pair | Step 1: `*.json` only (required for step 2). Step 2: **`params/` only** |
| **True** | Detailed per-image / per-pair logs, progress bars | Step 1: + `corners/<camera>/` overlays. Step 2: + `pairing_reports/`, `rectified/` |

```bash
# Default — minimal logs, only params/ from step 2
python main.py --project Olsen_wings --date 2026-05-12 --calib_name calib1

# Debug — verbose logs + intermediate folders
python main.py --project Olsen_wings --date 2026-05-12 --calib_name calib1 --troubleshooting
```

Legacy mode (`--legacy`) also respects `--troubleshooting` (corners, local_coords, images_ncb, rectified).

---

## All CLI parameters (reference)

| Parameter | Default | Used in |
|-----------|---------|---------|
| `--project` | required | all |
| `--date` | required | all |
| `--calib_name` | `calib1` | all |
| `--chessboard_size` | `8,7` | all |
| `--square_size` | `0.045` | all |
| `--left_chessboard_size` | = `--chessboard_size` | all |
| `--right_chessboard_size` | = `--chessboard_size` | all |
| `--left_square_size` | = `--square_size` | all |
| `--right_square_size` | = `--square_size` | all |
| `--preprocessing` | `None` | step 1 (`G`, `C`, `T` chain) |
| `--cameras` | all present | `detect_features.py` |
| `--ir_mode` | `auto` | step 1 (`auto` / `chessboard` / `ellipse`) |
| `--step` | `all` | `main.py` (`detect`/`calibrate`/`all`); `calibrate.py` (`mono`/`stereo`/`all`) |
| `--left_camera` | `lc` | step 2 stereo (`lc` / `lc-ir`) |
| `--time_window` | `0.1` | step 2 stereo (seconds) |
| `--partners` | `rc,rg,ir` | step 2 stereo |
| `--legacy` | off | `main.py` only |
| `--right_camera` | `rc` | `main.py --legacy` only |
| `--troubleshooting` | off | all (`False` = minimal; `True` = debug output) |

---

## Folder structure

```
~/Calib-data/<project>/<date>/<calib_name>/
├── lc/
│   ├── lc_1778599872850705.bmp
│   └── lc_1778599872850705.json      ← step 1 (always)
├── rc/
├── rg/   (or rgb/)
├── ir/   (or IR/)
├── corners/                          ← step 1, only with --troubleshooting
├── pairing_reports/                  ← step 2, only with --troubleshooting
├── rectified/                        ← step 2, only with --troubleshooting
└── params/                           ← step 2 (always)
    ├── lc_intrinsics.npz
    ├── rc_intrinsics.npz
    ├── lc-rc_parameters.npz
    ├── lc-rc_stereo_cam_model.yaml
    ├── lc-rc_Q.cvstore
    ├── lc-rg_*
    └── lc-ir_*
```

Nested layout (`<calib_name>/images/lc/`, …) is also supported.

---

## Quick start (full pipeline)

```bash
cd ~/Speckle-Scanner/02_Calibration

python main.py \
  --project Olsen_wings --date 2026-05-12 --calib_name calib1 \
  --chessboard_size 8,7 --square_size 0.045
```

Or run steps separately:

```bash
# Step 1 — detect features, write JSON
python detect_features.py \
  --project Olsen_wings --date 2026-05-12 --calib_name calib1 \
  --chessboard_size 8,7 --square_size 0.045

# Step 2 — calibrate from JSON (writes params/ only)
python calibrate.py \
  --project Olsen_wings --date 2026-05-12 --calib_name calib1 \
  --chessboard_size 8,7 --square_size 0.045 \
  --time_window 0.1
```

---

## Step 1 — Feature detection (per camera)

For every image in each camera folder (`lc`, `rc`, `rg`, `ir`, `lc-ir`):

- Detects **chessboard corners** (default for lc/rc/rg)
- For **IR**: tries chessboard first (`--ir_mode auto`), falls back to **ellipse center**
- Writes `<image>.json` in the **same folder** as the image (always, even without `--troubleshooting`)

### LC only

```bash
python detect_features.py \
  --project Olsen_wings --date 2026-05-12 --calib_name calib1 \
  --cameras lc \
  --chessboard_size 8,7 --square_size 0.045 \
  --left_chessboard_size 8,7 --left_square_size 0.045 \
  --preprocessing None
```

### RC only

```bash
python detect_features.py \
  --project Olsen_wings --date 2026-05-12 --calib_name calib1 \
  --cameras rc \
  --chessboard_size 8,7 --square_size 0.045 \
  --right_chessboard_size 8,7 --right_square_size 0.045 \
  --preprocessing None
```

### RG only

```bash
python detect_features.py \
  --project Olsen_wings --date 2026-05-12 --calib_name calib1 \
  --cameras rg \
  --chessboard_size 8,7 --square_size 0.045 \
  --right_chessboard_size 8,7 --right_square_size 0.045 \
  --preprocessing None
```

### IR only

```bash
python detect_features.py \
  --project Olsen_wings --date 2026-05-12 --calib_name calib1 \
  --cameras ir \
  --chessboard_size 8,7 --square_size 0.045 \
  --right_chessboard_size 8,7 --right_square_size 0.045 \
  --preprocessing C \
  --ir_mode auto
```

### LC-IR folder only

```bash
python detect_features.py \
  --project Olsen_wings --date 2026-05-12 --calib_name calib1 \
  --cameras lc-ir \
  --chessboard_size 8,7 --square_size 0.045 \
  --left_chessboard_size 8,7 --left_square_size 0.045 \
  --preprocessing None
```

### All cameras

```bash
python detect_features.py \
  --project Olsen_wings --date 2026-05-12 --calib_name calib1 \
  --chessboard_size 8,7 --square_size 0.045 \
  --left_chessboard_size 8,7 --left_square_size 0.045 \
  --right_chessboard_size 8,7 --right_square_size 0.045 \
  --preprocessing None \
  --ir_mode auto
```

### Step 1 with troubleshooting

```bash
python detect_features.py \
  --project Olsen_wings --date 2026-05-12 --calib_name calib1 \
  --cameras lc,rc,ir \
  --chessboard_size 8,7 --square_size 0.045 \
  --preprocessing C \
  --ir_mode auto \
  --troubleshooting
```

### JSON contents (chessboard example)

```json
{
  "version": 1,
  "image": "lc_1778599872850705.bmp",
  "camera_folder": "lc",
  "feature_type": "chessboard",
  "success": true,
  "board_size": [8, 7],
  "square_size": 0.045,
  "timestamp_sec": 1778599872.850705,
  "pair_key": "1778599872850705",
  "corners": [[412.3, 287.1], ...]
}
```

---

## Step 2 — Calibration

### 2a. Mono intrinsics

Reads chessboard JSONs from each camera folder, runs `cv2.calibrateCamera`, saves:

- `params/<camera>_intrinsics.npz`
- `params/<camera>_intrinsics.yaml`

Requires **≥ 3** successful chessboard detections per camera.

### 2b. Stereo calibration

- **Left camera:** `lc` by default (`--left_camera`)
- **Partners:** `rc`, `rg`, `ir` — each available folder is calibrated against lc
- **Pairing:** time-window match (`--time_window`, default **0.1 s**), then filename `pair_key` fallback for IR scan ids
- Uses mono intrinsics with `CALIB_FIX_INTRINSIC`
- Saves `lc-rc_*`, `lc-rg_*`, `lc-ir_*` under `params/`

### Full step 2 (mono + all stereo pairs)

```bash
python calibrate.py \
  --project Olsen_wings --date 2026-05-12 --calib_name calib1 \
  --step all \
  --left_camera lc \
  --partners rc,rg,ir \
  --time_window 0.1 \
  --chessboard_size 8,7 --square_size 0.045 \
  --left_chessboard_size 8,7 --left_square_size 0.045 \
  --right_chessboard_size 8,7 --right_square_size 0.045
```

### Stereo: LC ↔ RC only

```bash
python calibrate.py \
  --project Olsen_wings --date 2026-05-12 --calib_name calib1 \
  --step stereo \
  --left_camera lc \
  --partners rc \
  --time_window 0.1 \
  --chessboard_size 8,7 --square_size 0.045
```

### Stereo: LC ↔ RG only

```bash
python calibrate.py \
  --project Olsen_wings --date 2026-05-12 --calib_name calib1 \
  --step stereo \
  --left_camera lc \
  --partners rg \
  --time_window 0.1 \
  --chessboard_size 8,7 --square_size 0.045
```

### Stereo: LC ↔ IR only

```bash
python calibrate.py \
  --project Olsen_wings --date 2026-05-12 --calib_name calib1 \
  --step stereo \
  --left_camera lc \
  --partners ir \
  --time_window 0.1 \
  --chessboard_size 8,7 --square_size 0.045
```

### Stereo: left = LC-IR folder

```bash
python calibrate.py \
  --project Olsen_wings --date 2026-05-12 --calib_name calib1 \
  --step stereo \
  --left_camera lc-ir \
  --partners rc,rg,ir \
  --time_window 0.1 \
  --chessboard_size 8,7 --square_size 0.045 \
  --left_chessboard_size 8,7 --left_square_size 0.045 \
  --right_chessboard_size 8,7 --right_square_size 0.045
```

### Mono intrinsics only

```bash
python calibrate.py \
  --project Olsen_wings --date 2026-05-12 --calib_name calib1 \
  --step mono \
  --chessboard_size 8,7 --square_size 0.045 \
  --left_chessboard_size 8,7 --left_square_size 0.045 \
  --right_chessboard_size 8,7 --right_square_size 0.045
```

### Step 2 with troubleshooting

```bash
python calibrate.py \
  --project Olsen_wings --date 2026-05-12 --calib_name calib1 \
  --step all \
  --time_window 0.1 \
  --chessboard_size 8,7 --square_size 0.045 \
  --troubleshooting
```

Writes `params/` plus `pairing_reports/<pair>.txt` and `rectified/<pair>/`.

---

## Full pipeline (`main.py`)

```bash
python main.py \
  --project Olsen_wings --date 2026-05-12 --calib_name calib1 \
  --step all \
  --left_camera lc \
  --partners rc,rg,ir \
  --time_window 0.1 \
  --chessboard_size 8,7 --square_size 0.045 \
  --left_chessboard_size 8,7 --left_square_size 0.045 \
  --right_chessboard_size 8,7 --right_square_size 0.045 \
  --preprocessing None \
  --ir_mode auto
```

With troubleshooting:

```bash
python main.py \
  --project Olsen_wings --date 2026-05-12 --calib_name calib1 \
  --troubleshooting \
  --chessboard_size 8,7 --square_size 0.045
```

---

## Legacy one-shot mode

The old in-memory flow (single `--right_camera`, filename pairing) still works:

```bash
# LC-RC
python main.py --legacy \
  --project Olsen_wings --date 2026-05-12 --calib_name calib1 \
  --left_camera lc --right_camera rc \
  --chessboard_size 8,7 --square_size 0.045

# LC-RG
python main.py --legacy \
  --project Olsen_wings --date 2026-05-12 --calib_name calib1 \
  --left_camera lc --right_camera rg \
  --chessboard_size 8,7 --square_size 0.045

# LC-IR
python main.py --legacy \
  --project Olsen_wings --date 2026-05-12 --calib_name calib1 \
  --left_camera lc --right_camera ir \
  --chessboard_size 8,7 --square_size 0.045 \
  --preprocessing C

# LC-IR folder + IR partner (with debug output)
python main.py --legacy \
  --project Olsen_wings --date 2026-05-12 --calib_name calib1 \
  --left_camera lc-ir --right_camera ir \
  --chessboard_size 8,7 --square_size 0.045 \
  --preprocessing C --troubleshooting
```

---

## Dependencies

```bash
pip install -r ~/Speckle-Scanner/02_Calibration/requirements.txt
# or full pipeline:
pip install -r ~/Speckle-Scanner/requirements.txt
```

---

## Notes

- Stereo pairing uses **timestamps** parsed from filenames (`ts…` tokens or long numeric ids); `ck…` suffixes are ignored.
- **Ellipse-only** IR JSONs are stored but cannot produce mono intrinsics (need full chessboard grids). Use chessboard IR images for calibration.
- Per-camera board overrides apply to detection and calibration (`--left_chessboard_size`, etc.).
- Re-run **step 1** if images change; re-run **step 2** freely when tuning `time_window` or partners.
- With `--troubleshooting` off, step 2 writes **only** `params/` (no `pairing_reports/`, no `rectified/`).