w8 gemini Flashcards
List the constraints typically applied to solving the video correspondence problem.
Spatial coherence (neighboring points have similar optical flow) and Small motion (optical flow vectors have small magnitude).
Note circumstances in which the spatial coherence constraint fails.
It fails at discontinuities between surfaces at different depths, or surfaces with different motion.
Note circumstances in which the small motion constraint fails.
It fails if relative motion is fast or frame rate is slow.
Define what is meant by the “aperture problem”.
The aperture problem refers to the fact that the direction of motion of a small image patch can be ambiguous.
Suggest how the aperture problem can be overcome.
- Integrating information from many local motion detectors / image patches, or 2. by giving preference to image locations where image structure provides unambiguous information about optic flow (e.g. corners).
Consider a traditional barber’s pole. When the pole rotates on its axis, in which direction is the motion field?
Horizontal.
Consider a traditional barber’s pole. When the pole rotates on its axis, in which direction is the optic flow?
Vertical.
Explain why the optic flow of a rotating barber’s pole appears vertical, despite the actual motion being horizontal, in the context of the aperture problem.
The stripes of the pole provide ambiguous information. Only where the stripes meet the top and bottom and the sides of the pole are corners present, and hence, there is (seemingly) unambiguous information.
What is video in the context of computer vision?
A series of N images, or frames, acquired at discrete time instants tk=to+kΔt, where Δt is a fixed time interval and k=0,1,…,N-1.
How is video similar to stereo vision?
Both deal with more than one image.
What are some advantages of using video compared to static images or stereo vision?
Inference of 3D structure, segmentation of objects from background without recovery of depth, and inference of self and object motion.
What are the two main types of motion that cause changes in the projection of a scene point in a video?
Object motion and camera motion (or ‘ego motion’).
What is optic flow?
The apparent motion of objects, surfaces, and edges in a visual scene caused by the relative motion between an observer (an eye or a camera) and the scene.
Distinguish between an optic flow vector and an optic flow field.
An optic flow vector represents the image motion of a single scene point. An optic flow field is the collection of all optic flow vectors in an image.
What are the two main categories of optic flow fields?
Sparse (vectors defined only for specified features) and dense (vectors defined everywhere).
What is the analogy between optic flow vectors and disparity vectors?
Optic flow vectors are analogous to disparity vectors in stereo vision.
What is required to measure optic flow?
Finding correspondences between images.
What is the motion field?
The true image motion of a scene point, which is the actual projection of the relative motion between the camera and the 3D scene.
Explain why optic flow is often an approximation of the motion field.
Optic flow is what we can measure in the image, but it might not perfectly represent the actual 3D motion due to factors like illumination changes or non-Lambertian surfaces.
Give an example where the motion field is non-zero, but the optic flow is zero.
A smooth, Lambertian, uniform sphere rotating around a diameter.
Give an example where the motion field is zero, but the optic flow is non-zero.
A stationary, specular sphere and a moving light source.
Give an example where the motion field and optic flow differ.
A barber’s pole: motion field is horizontal, optic flow is vertical.
Why must we estimate the motion field by observing the optic flow?
Because the motion field cannot be directly observed.
What is the video correspondence problem?
The problem of finding corresponding points in different frames of a video to measure optic flow.
What are the two main types of methods used to solve the video correspondence problem?
Feature-based methods and Direct methods.
Describe feature-based methods for solving the video correspondence problem.
Extract descriptors from around interest points and find similar features in the next frame.
Describe direct methods for solving the video correspondence problem.
Directly recover image motion at each pixel from temporal variations of the image brightness.
What are the basic requirements for using feature-based methods to solve the correspondence problem?
- Most scene points visible in both images. 2. Corresponding image regions appear “similar”.
What is the spatial coherence constraint in video correspondence?
Similar neighboring flow vectors are preferred over dissimilar ones.
When does the spatial coherence constraint fail?
At discontinuities between surfaces at different depths or with different motions.
What is the small motion constraint in video correspondence?
Small optic flow vectors are preferred over large ones.
When does the small motion constraint fail?
If relative motion is fast or the frame rate is slow.
Explain the aperture problem using the example of a moving rectangle.
When observing a small patch of a moving edge, the motion parallel to the edge is ambiguous.
How does the brain deal with the aperture problem?
The brain combines local motion measurements across space or relies on unambiguous features like corners.
What are two potential reasons why we might perceive motion perpendicular to an edge in the aperture problem?
It might be the average of all possibilities or it predicts the slowest movement.
Describe two solutions to the aperture problem.
- Combine local motion measurements across space. 2. Utilize locations where the direction of motion is unambiguous.
List some applications of optic flow.
Estimating the layout of the environment, estimating ego-motion, estimating object motions, and obtaining predictive information for action control.
Describe the simple case 1 for depth from optic flow with known ego-motion.
Direction of motion is perpendicular to the optical axis, and the velocity of the camera (Vx) is known.
Describe the simple case 2 for depth from optic flow with known ego-motion.
Direction of motion is along the camera optical axis, and the velocity of the camera (Vz) is known.
In the context of time-to-collision from optic flow (simple case 2), what is assumed about the camera’s velocity?
The camera velocity is unknown.
What is the formula for time-to-collision derived from optic flow?
Time-to-collision = x / vx
What is a significant advantage of calculating time-to-collision from optic flow in this way?
It can be calculated purely from the image without knowing the camera’s velocity or the object’s depth.
How can optic flow be used for robot navigation?
By moving so that the destination is the Focus of Expansion (FOE).
How can optic flow be used to judge relative depths?
By relative magnitudes of optic flow vectors; points closer to the camera move more quickly across the image plane.
How can optic flow be used to measure absolute depths?
With knowledge of camera velocity.
How can optic flow be used to judge camera speed?
By the rates of expansion/contraction.
How can optic flow be used for segmentation?
Discontinuities in the optic flow field indicate different depths and, hence, different objects.
What are camera translations that induce characteristic patterns of optic flow for a static scene?
Turn/translate left, turn/translate right, move forward, move backward.
What are the characteristics of a Parallel Optic Flow Field (Vz = 0)?
All optic flow vectors are parallel, direction of camera movement is opposite to the direction of the optic flow field.
What are the characteristics of a Radial Optic Flow Field (Vz ≠ 0)?
All optic flow vectors point towards/away from a vanishing point (po).
How is relative depth determined in a Parallel Optic Flow Field?
Depth is inversely proportional to the magnitude of the optic flow vector.
How is relative depth determined in a Radial Optic Flow Field?
Depth of a point p is inversely proportional to the magnitude of the optic flow vector and also proportional to the distance from p to po.
Explain how optic flow discontinuities can be used for segmentation.
Large differences in optic flow between adjacent regions often indicate boundaries between objects moving independently.
What is the key idea behind segmentation from motion?
Moving objects can be distinguished from a static background or other moving objects based on differences in their motion patterns.
What are tracking algorithms?
Algorithms used when high-level features, such as objects, are matched across several frames.
How do tracking algorithms typically work?
They use previous frames to PREDICT the location of an object in the next frame.
What are the intents behind using tracking algorithms?
To do less work looking for the object and to get improved estimates since measurement noise is averaged out.
Name two common methods used in tracking algorithms.
Kalman filtering and Particle filtering.
How do humans predict movement?
By extrapolating object trajectory from previous locations and from high-level knowledge of typical movement.
Describe the concept of ‘segmentation from motion’.
Using motion cues to separate different objects or regions in a video sequence.
Give an example of segmentation from motion.
Camouflaged objects are more easily seen when they move.
Explain the technique of ‘image differencing’ for segmentation.
For a static camera, successive images are subtracted pixel by pixel.
What is a problem with the image differencing technique for segmentation?
The grey level changes most where background is replaced by the object.
Explain the ‘background subtraction’ technique for segmentation.
A reference image of the static background is learned and subtracted from the current frame.
What is an advantage of background subtraction over image differencing?
Objects that are temporarily stationary are still seen as foreground.
What are some common methods for learning the static background in background subtraction?
Adjacent Frame Difference, Off-line average, and Moving average.
Describe the ‘Adjacent Frame Difference’ method for background subtraction.
Each image is subtracted from the previous image in the sequence.
Describe the ‘Off-line average’ method for background subtraction.
Pixel-wise mean values are computed during a separate training phase.
Describe the ‘Moving average’ method for background subtraction.
The background model is a linear weighted sum of previous frames.
List the steps in a typical algorithm for motion segmentation using background subtraction.
- Update background model. 2. Compute frame difference. 3. Threshold frame difference. 4. Noise removal.
What is the goal of the correspondence step in optic flow analysis?
Determining which points in different frames are projections of the same point in the scene.
What constraints are typically used in the correspondence step of optic flow?
Small motion and spatial coherence.
Describe the ‘Moving average’ method for background subtraction.
The background model is a linear weighted sum of previous frames, which can overcome slow illumination changes.
What constraints are typically used in the correspondence step of optic flow?
Small motion and Spatial coherence.
What is the goal of the reconstruction step in optic flow analysis?
Given the correspondence between points, calculate 3D structure and motion of the observed scene.
What can be calculated in the reconstruction step with knowledge of ego-motion?
Absolute depth.
What can be calculated in the reconstruction step without knowledge of ego-motion?
Relative depths, time-to-collision, and direction of ego-motion.
Summarize the process of segmenting moving objects from a stationary background.
Calculate the absolute difference between the current image and a background model (abs(I(x,y,t)-B(x,y))>T).
What are two common approaches to background subtraction?
Image differencing and explicit background subtraction.
Describe the formula for the background model in image differencing.
B(x,y) = I(x,y,t-1).
Describe the formula for the background model in off-line average background subtraction.
B(x,y) = (1/N) * Σ(from t=1 to N) I(x,y,t).
Describe the formula for the background model in moving average background subtraction.
B(x,y) ← (1-β)B(x,y) + βI(x,y,t).
