Robot Learning

Question 1

What is epipolar geometry?

Answer 1

Epipolar geometry is the intrinsic projective geometry between two views. It is
independent of scene structure, and only depends on the cameras’ internal parameters
and relative pose.

Question 2

What is the fundamental matrix in relation to the geometry of two views?

Answer 2

The fundamental matrix F encapsulates the intrinsic geometry of the views, and is a 3x3 matrix with rank 2. If a point in 3-space X is imaged as x in the first view, and x′ in the second, then the image points satisfy the relation x′(transposed) Fx = 0.

Question 3

What is the relation between the corresponding image points x and x′?

Answer 3

the image points x and x′, space point X, and camera centres are coplanar. Denote this plane as π. Clearly, the rays back-projected from x and x′ intersect at X, and the rays are coplanar, lying in π.

Question 4

how is the corresponding point x′ constrained?

Answer 4

The plane π is determined by the baseline and the ray defined by x. We know that the ray corresponding to the (unknown) point x′ lies in π, hence the point x′ lies on the line of intersection l’ of π with the second image plane.

Question 5

What is the epipolar line corresponding to x?

Answer 5

the point x′ lies on the line of intersection l′ of π with the second image plane. This line l′ is the image in the second view of the ray back-projected from x

Question 6

What is the benefit of utilizing epipolar geometry for a stereo correspondence algorithm?

Answer 6

The search for a particular point corresponding to x is constrained to the epipolar line l’, instead of searching the entire image plane.

Question 7

What is the epipole?

Answer 7

The point of intersection of the line joining the camera centres (the baseline) with the image plane. Equivalently, the epipole is the image in one view of the camera centre of the other view. It is also the vanishing point of the baseline
(translation) direction.

Question 8

What is the epipolar plane?

Answer 8

A plane containing the baseline (the line joining the camera centers) - represented by a single parameter family of epipolar planes.

Question 9

What is the epipolar line?

Answer 9

The intersection between the epipolar plane with the image plane.

Question 10

What are deformable parts models(DPM)?

Answer 10

use a sliding window approach where a
classifier is run at evenly spaced locations over an entire image

Question 11

What are region proposal methods?

Answer 11

methods to first generate potential bounding boxes in an image and then run a classifier on these proposed boxes. After classification, post-processing is used to refine the bounding boxes, eliminate duplicate detections, and rescore the boxes based on other objects in the scene. R-CNNs are an example.

Question 12

What’s the key weakness of region proposal methods?

Answer 12

Pipelines can be too slow and complex to optimize since each component of the pipeline needs to be trained separately.

Question 13

What architecture does the YOLO (You only look once by Redmon, Divvala, Girshick, Farhadi) paper present?

Answer 13

Proposes a unified architecture that has a single CNN that predicts both the bounding boxes and the associated probabilities. YOLO trains on full images and directly optimizes detection performance

Question 14

What are the main strengths proposed by the YOLO paper?

Answer 14

Architecture is fast, learns generalizable representations of objects, and reasons globally about images instead of using methods such as region proposal methods and DPMs

Question 15

What are the main weaknesses with the YOLO architecture?

Answer 15

YOLO lags behind in accuracy compared to SOTA models and struggles with localizing particularly smaller objects in images.

Question 16

Describe YOLO’s architectural design

Answer 16

YOLO divides images into SxS grid cells, and then tries to predict B bounding boxes per cell and the associated confidences of each box. If the center of an object is in the center of the cell - it is responsible for detecting it. Each bounding box consists of 5 predictions: x, y, w, h, and confidence. Each grid cell also predicts C conditional class probabilities, Pr(Classi|Object)

Question 17

What does confidence mean for the YOLO architecture

Answer 17

confidence scores reflect how
confident the model is that the box contains an object and
also how accurate it thinks the box is that it predicts. Formally defined confidence is Pr(Object) ∗ IOU^truth/pred (intersection over union between prediction and ground truth)

Question 18

Instance Segmentation

Answer 18

Combines object detection and semantic segmentation to correctly detect all objects in an image while precisely segmenting each instance.

Question 19

Object Detection

Answer 19

Classifying and localizing individual objects using a bounding box.

Question 20

Semantic Segmentation

Answer 20

Computer vision task of classifying each pixel of an image into a fixed set of categories without differentiating object instances.

Question 21

How to add context when using a sliding window approach for semantic segmentation?

Answer 21

Extract regional patches around center pixels, and then feed these patches into CNNs to identify the center pixel. However, this is very costly computationally and very inefficient.