Stereo Vision Flashcards
StereoVision:
How does the human visual system work?
Many of the perceptual cues we use to visualize 3D structure are
available in 2D projections
Which cues are we talking about?
– occlusion (one object partially covering another)
– perspective (point of view)
– familiar size (we know the real-world sizes of many objects)
– atmospheric haze (objects further away look more washed out)
What are the four cues that are missing from single 2D views?
– stereo parallax - seeing a different image with each eye
– movement parallax -seeing different images when we move the
head
– Accommodation - the eyes’ lenses focus on the object of interest
– Convergence - both eyes converge on the object of interest
What does Stereopsis mean?
- stereo means solid or three dimensional
- opsis means appearance or sight
What is Stereopsis?
- impression of depth that is perceived when a scene is viewed with both eyes
- binocular disparity is due to the different position of our two eyes
What are some common ways to produce 3D sensation?
- anaglyphs: two colored images and color coded glasses ((red/cyan(green))
- two images with different light polarization and polarizing glasses-linear and circular
- double frame-rate displays combined with LCD shutter glasses
- autostereoscopic displays:
- such as parallax barrier and lenticular lens - HMDs (Head Mounted Displays)
- “exotic displays”
What is StereoVision?
- capability to define depth from two images
- possible by computing correspondences between two images
What is the frontal parallel arrangement?
-Frontal-parallel arragement of an image refers to the result you get after rectifying warped images.
- Z = f*T / d ; d = xl - xr
- stereo system have good depth resolution for close objects since depth is inversely proportional to disparity
-it is easy to relate correspondence to depth in
frontal parallel arrangement
What is the problem of frontal parallel arrangement?
It is how to map real configuration into it
What is an epipole?
- epipole is a projection of the optical center of a camera on the other image plane
What is epipolar geometry useful for?
-given a point in an image, its corresponding point in the other image lies on the corresponding epipolar line
- order is preserved
- transforms a 2D search into a 1D search saving resources and avoiding errors
How is epipolar geometry defined?
It is defined using Essential (E) and Fundamental (F) Matrices
What information does the Essential Matrix gives?
- information about relative position between cameras (rotation and translation) [extrinsic]
- maps a 3D point in one image with its corresponding 3d point on the other image considering translation and rotation
What information does the Fundamental Matrix gives?
- intrinsic parameters of the cameras (focal
length, lens distortion, optical centre, etc…) - maps a pixel in one image with its corresponding pixel on the other image
What do Fundamental and Essential matrices have in common?
They represent the transformation between the
stereo pair images.
Give me a difference between the Fundamental and Essential matrices
Fundamental matrix operates in image coordinates (pixels) while the Essential matrix operates in physical coordinates.
What is Image rectification?
- rectify an image by aligning epipolar lines in rows on the two rectified images getting a frontal parallel arranjement
What is disparity in rectified images?
It is simply difference between pixel coordinates xl and xr
How does Template matching work?
-It evaluates how well template matches image in that position.
-Template matching is moved to all positions (𝑢, 𝑣) in image and computes 𝑝 (𝑎, 𝑏) to evaluate how well template matches image in that position.
How to know if object is on image at some position?
The position is where occurs max 𝑝(𝑎, 𝑏) if max(𝑝 𝑎, 𝑏) > threshold
What are some possible functions to compare template and
image 𝑝( 𝑎, 𝑏)?
- square difference matching
- correlation matching
correlation coefficient matching
What is Stereo Matching?
It consists on matching pixels in conjugate epipolar lines
How does the basic stereo algorithm work?
- for each epipolar line and for each pixel in the left image:
- compare with the pixel on the same epipolar line in the right image
- pick pixel with minimum match cost
What other algorithm is there?
There is also the block matching algorithm
How does the Block matching algorithm work?
- divides an image into macrobooks
- compare each with a corresponding block and its neighbours in the another image
Which metrics can be used in the Block matching algorithm?
- mean difference or mean absolute difference (MAD) ;
- mean squared error (MSE)
Does the window size affects the results of stereo?
Yes.
In fact, there are better results with adaptive window.
Where do Errors in stereo system come from?
– Camera calibration errors
– Poor image resolution
– Occlusions
– Violations of brightness constancy (specular reflections)
– Large motions
– Low-contrast image regions
What are the steps on Stereo Vision?
- calibrate cameras
- rectify images
- compute disparity
- estimate depth
What are some StereoVision functions on OpenCV?
- cvFindChessboardCorners: detect chessboard corner in stereo images
- cvStereoCalibrate: calibrates stereo rig
- cvStereoRectify: computes rotations that make both camera planes the same.
- cvInitUndistortRectifyMap and cvRemap: use to compute undistortion map and rectified images
- Stereo correspondence (ex:cvFindStereoCorrespondenceBM): computes the disparity map.
- cvReprojectImageTo3D: disparity map to 3D with calibrated cameras
