06 - 3D Sensors

Question 1

Mention a few optical 3d sensing modalities

Answer 1

Active (use of light source):
- Reflective
- Triangulation based: Structured light (multipattern & one shot), Laser Line Scanner
- Time Of Flight, Depth from defocus etc
- Transmissive

Passive:
- Stereo Vision
- Shape from Texture, silhouettes, shadows, specularities

Which one we choose depends on the time, money and situation we have. Fx stereo tends to cost less than structured light&tof and is better for outdoor use, while it is not as good as the others in low light scenarios and not as fast.

Question 2

Explain the TOF principle and the phase difference method

Answer 2

We send out a light pulse and use the time it takes for the reflection to come back to calculate the distance to the object/point in space
- Problems: timer needs extremely good resolution bc light moves so fast. The closer an object is, the faster the light reflects back, so for very close objects nanosecond precision is needed. Also, the pulse needs to be very very short & really big/intense to be able to be seen from the sensor and still be precise

Continuous Wave Modulation (Typically used)
- Instead of a pulse, temporally modulated light is emitted
- The depth is calculated with help of the phase shift
- we can adapt the frequency to the distance to the object

Question 2

What are rectifying homographies and how can they be computed?

Answer 2

Rectifying homographies are transformation matrices used to correct images such that scanlines can be matched directly (the epipolar lines will be parallel).

To estimate the rectification homography, we need to find a plane in space (the gray plane on image below) and then the homography to map from that plane to the camera planes (green and red). The 3D plane is given by Q = aA + bB + C.

Good choice for A: vector that is connecting the camera centers

Good choice for B: normalized corssproduct of A and N (average of the two imageplane normals)

Good choice for C: find the biggest overlap between the two camera views.

Question 2

What properties/advantages do active triangulation-based sensors have?

Answer 2

Fast, good low light performance, bad for outdoors and can be costly.

Question 3

How does active scene coding provide correspondences (binary coding and phase shifting)?

Answer 3

Each prjector column gets an individual binary code, where a 0 means no light is send and 1 means light is send.
Now we are able to do 2^n-1 stripes in n images.
Possible problems: light bleeding

We can also do ternary coding (on, half on, off or RGB) and k-ary, where we can do k^n stripes for n images

Phase Shifting:

instead of using descrete levels we now use continuous functions. It is possible to just use multiple functions, but in phase shifting the idea is to use one single function and just shift its phase

Phase shift is way faster (needs way less images) to get info about all pixels BUT needs very good photometric calibration between projector & camera

Question 4

Which directions need to be coded?

Answer 4

If we have epipolar constriants, we only need to code in one direction (along the epipolar lines) I think there is also something