Stereo Camera SDK (+Viewer) with Pluggable Disparity Engines

Summary

Build a minimal but scalable SDK for a custom stereo camera that provides:

1. a pluggable disparity interface (algorithms: SGBM, LightStereo, CREStereo; MVP: LightStereo CPU + LightStereo CUDA),

2. a simple viewer (stereo + depth), and

Deliver an MVP in 3 weeks, production-ready structure, clear extension points, and CI. The SDK must be designed to become cross-platform and language-agnostic (future bindings for Python, Rust, Java).

Target MVP Scope (3 weeks)

Platforms (MVP)

• Primary dev/test: Linux x86_64 (Ubuntu 22.04 or 24.04).

• Optional (nice-to-have if time allows): Windows 11.

• GPU: NVIDIA CUDA (12.x) for LightStereo CUDA variant (if GPU present). CPU-only still required.

Core Modules

1. Disparity Engine Abstraction

   • MVP implementations:

     • LightStereoCPU (no GPU)

     • LightStereoCUDA (CUDA 12.x)

   • Stubs only (no implementation) for SGBM and CREStereo to demonstrate the extension points (return “not implemented” but compile).

2. Viewer (simple)

   • Minimal UI

   • Panels:

     • Live Stereo view (left/right rectified).

     • Depth view: colorized depth, with dynamic range slider + click-to-probe pixel to show depth (m) and disparity.

   • FPS + latency overlay.

   • Source selector (Dummy folder / camera API placeholder).

   • Algorithm selector (LightStereo CPU/CUDA).

   • Record/Playback:

     • Optional if time permits: write/read simple dataset format (pngs + timestamps + json).

3. Public C API (for future language bindings)

   • Provide a thin C layer on top of the C++ core

   • Python demo binding via pybind11 around the C API. This proves multi-language viability now; Rust/Java to be added later using same C ABI.

Non-Goals (MVP)

• Full cross-platform parity (macOS, ARM) — just design for it.

• Production-grade camera driver integration — we’ll use a clean interface and a dummy source now.

• Advanced UI features (ROI tools, 3D point cloud) — can be future work.

Tech Stack

• Language: C++20 for core; C ABI for FFI; minimal C wrappers.

• Build: CMake (exported targets, find_package ready).

• Computer Vision: OpenCV 4.8+.

• GPU: CUDA 12.x (optional at runtime).

• UI: Dear ImGui + GLFW (or Qt 6 if contractor prefers, but must keep layering clean).

• Bindings: pybind11 (MVP demo).

• Testing: GoogleTest + data-driven tests; clang-format/clang-tidy.

• CI: GitHub Actions (Linux build + unit tests; optionally Windows).

Deliverables

1. Source code repo.

2. Build scripts (CMake) and README with:

   • prerequisites, build, run, and sample commands,

   • how to call the C API and Python binding.

3. CLI tools

   • stereo_viewer (run viewer; select algorithm; show stereo+depth).

   • stereo_benchmark (optional: measures FPS/latency on sample data).

4. Python demo

   • Minimal example examples/python/run_lightstereo.py that loads two images + calibration (hardcoded) and returns depth.

5. Unit tests

   • Rectification consistency.

   • Disparity API smoke tests for CPU/CUDA variants.

6. CI pipeline (Linux build + tests).

7. Documentation

   • API reference (doxygen or md files).

   • Architecture overview diagram.

   • Extension guide: “How to add a new IDisparity implementation”.

8. Release artifacts

   • libstereo_sdk.so (or .dll), CLI tools, and Python wheel for Linux (cp310+).

Timeline & Milestones (3 Weeks)

• Week 1

  • Repo scaffolding, CMake, core types, C API skeleton.

• Week 2

  • IDisparity interface + LightStereoCPU implementation.

  • Viewer (stereo + depth; selector; FPS overlay).

  • Python binding (run path), initial tests, CI.

• Week 3

  • LightStereoCUDA implementation and runtime selection.

  • Stabilize C API; docs; examples.

  • Performance pass; unit tests; packaging & delivery.

Contractor Qualifications

• Strong C++ (C++17/20) and CMake.

• OpenCV experience.

• Experience with deep/disparity models (LightStereo/CREStereo/SGBM) and CUDA kernels or deployment.

• Familiar with FFI and building C ABI for Python/Rust/Java bindings.

• Comfortable with Dear ImGui (or Qt) for quick viewers.

• Bonus: packaging (Python wheels), Windows builds, plugin architectures.