Shape Up Pitch
# AnySurface – Acequia Camera

## Problem
We want a robust calibration, scene capture and reprojection for AnySurface. The challenge is to accurately calibrate the camera-projector system, generate RGBD data streams, and fuse them into usable point clouds. 

This work will also benefit the Simtable triggerless effort, learning to adjust shutter, gain/iso, improving correspondence algorithm and depth scanning. It will also benefit Realtime.Earth camera observations.

## Appetite
6 weeks (1 cycle). Deliver a working prototype focusing on camera-projector calibration, structured light scanning, and fused point cloud generation.

## Solution
- **Calibration Core**
  - Use **two Android cameras + structured light projector**
  - compare benefit of high vs low resolution android camera scans
  - Estimate intrinsic and extrinsic poses for cameras and projector.
  - Compute **3 fundamental matrices** and the related **trifocal tensor** to establish geometric constraints.
  - Validate models against known objects and measured placements
  = reproject the depth model from the projectors view
  - Compare against other methods if time allows (monocular depth, poly.cam photgrammetry, etc)
- **RGBD Data & Point Clouds**
  - Create a three.js **RGBD projective pixels**  for cameras and projector (three.js viewer like the railyard depth pixels)
  - Create rgbd images for cameras and a derived rgbd image for projector. 
  - Create **per-view point clouds** and a **fused global point cloud**.
- **Acequia Integration**
  - **Work with Redfish team (Josh and Stephen)** to integrate **Acequia GeoCamera** to pub/sub to network cameras


## Deliverables
- **deployed page on AnySurface**
- **Prototype pipeline using Acequia GeoCamera.**
- **Camera-projector calibration module:** Fundamental matrices + trifocal tensor computation.
- **RGBD + Point Cloud Outputs:** Individual viewpoints and fused global point cloud.
- **Integration with Redfish team (Josh & Stephen) to refine Acequia GeoCamera.**
- **lab notes write up**

this: https://guerin.acequia.io/pitches/anysurface-geocamera.md