Stereo Vision

Question 1

StereoVision:
How does the human visual system work?

Answer 1

Many of the perceptual cues we use to visualize 3D structure are
available in 2D projections

Question 2

Which cues are we talking about?

Answer 2

– occlusion (one object partially covering another)
– perspective (point of view)
– familiar size (we know the real-world sizes of many objects)
– atmospheric haze (objects further away look more washed out)

Question 3

What are the four cues that are missing from single 2D views?

Answer 3

– stereo parallax - seeing a different image with each eye
– movement parallax -seeing different images when we move the
head
– Accommodation - the eyes’ lenses focus on the object of interest
– Convergence - both eyes converge on the object of interest

Question 4

What does Stereopsis mean?

Answer 4

• stereo means solid or three dimensional
• opsis means appearance or sight

Question 5

What is Stereopsis?

Answer 5

• impression of depth that is perceived when a scene is viewed with both eyes
• binocular disparity is due to the different position of our two eyes

Question 6

What are some common ways to produce 3D sensation?

Answer 6

• anaglyphs: two colored images and color coded glasses ((red/cyan(green))
• two images with different light polarization and polarizing glasses-linear and circular
• double frame-rate displays combined with LCD shutter glasses
• autostereoscopic displays:
  - such as parallax barrier and lenticular lens
• HMDs (Head Mounted Displays)
• “exotic displays”

Question 7

What is StereoVision?

Answer 7

• capability to define depth from two images
• possible by computing correspondences between two images

Question 8

What is the frontal parallel arrangement?

Answer 8

-Frontal-parallel arragement of an image refers to the result you get after rectifying warped images.
- Z = f*T / d ; d = xl - xr
- stereo system have good depth resolution for close objects since depth is inversely proportional to disparity
-it is easy to relate correspondence to depth in
frontal parallel arrangement

Question 9

What is the problem of frontal parallel arrangement?

Answer 9

It is how to map real configuration into it

Question 10

What is an epipole?

Answer 10

• epipole is a projection of the optical center of a camera on the other image plane

Question 11

What is epipolar geometry useful for?

Answer 11

-given a point in an image, its corresponding point in the other image lies on the corresponding epipolar line
- order is preserved
- transforms a 2D search into a 1D search saving resources and avoiding errors

Question 12

How is epipolar geometry defined?

Answer 12

It is defined using Essential (E) and Fundamental (F) Matrices

Question 13

What information does the Essential Matrix gives?

Answer 13

• information about relative position between cameras (rotation and translation) [extrinsic]
• maps a 3D point in one image with its corresponding 3d point on the other image considering translation and rotation

Question 14

What information does the Fundamental Matrix gives?

Answer 14

• intrinsic parameters of the cameras (focal
  length, lens distortion, optical centre, etc…)
• maps a pixel in one image with its corresponding pixel on the other image

Question 15

What do Fundamental and Essential matrices have in common?

Answer 15

They represent the transformation between the
stereo pair images.

Question 16

Give me a difference between the Fundamental and Essential matrices

Answer 16

Fundamental matrix operates in image coordinates (pixels) while the Essential matrix operates in physical coordinates.

Question 17

What is Image rectification?

Answer 17

• rectify an image by aligning epipolar lines in rows on the two rectified images getting a frontal parallel arranjement

Question 18

What is disparity in rectified images?

Answer 18

It is simply difference between pixel coordinates xl and xr

Question 19

How does Template matching work?

Answer 19

-It evaluates how well template matches image in that position.
-Template matching is moved to all positions (𝑢, 𝑣) in image and computes 𝑝 (𝑎, 𝑏) to evaluate how well template matches image in that position.

Question 20

How to know if object is on image at some position?

Answer 20

The position is where occurs max 𝑝(𝑎, 𝑏) if max(𝑝 𝑎, 𝑏) > threshold

Question 21

What are some possible functions to compare template and
image 𝑝( 𝑎, 𝑏)?

Answer 21

• square difference matching
• correlation matching
  correlation coefficient matching

Question 22

What is Stereo Matching?

Answer 22

It consists on matching pixels in conjugate epipolar lines

Question 23

How does the basic stereo algorithm work?

Answer 23

• for each epipolar line and for each pixel in the left image:
  - compare with the pixel on the same epipolar line in the right image
  - pick pixel with minimum match cost

Question 24

What other algorithm is there?

Answer 24

There is also the block matching algorithm

Question 25

How does the Block matching algorithm work?

Answer 25

• divides an image into macrobooks
• compare each with a corresponding block and its neighbours in the another image

Question 26

Which metrics can be used in the Block matching algorithm?

Answer 26

• mean difference or mean absolute difference (MAD) ;
• mean squared error (MSE)

Question 27

Does the window size affects the results of stereo?

Answer 27

Yes.
In fact, there are better results with adaptive window.

Question 28

Where do Errors in stereo system come from?

Answer 28

– Camera calibration errors
– Poor image resolution
– Occlusions
– Violations of brightness constancy (specular reflections)
– Large motions
– Low-contrast image regions

Question 29

What are the steps on Stereo Vision?

Answer 29

• calibrate cameras
• rectify images
• compute disparity
• estimate depth

Question 30

What are some StereoVision functions on OpenCV?

Answer 30

• cvFindChessboardCorners: detect chessboard corner in stereo images
• cvStereoCalibrate: calibrates stereo rig
• cvStereoRectify: computes rotations that make both camera planes the same.
• cvInitUndistortRectifyMap and cvRemap: use to compute undistortion map and rectified images
• Stereo correspondence (ex:cvFindStereoCorrespondenceBM): computes the disparity map.
• cvReprojectImageTo3D: disparity map to 3D with calibrated cameras