Image Alignment / Panorama Stitching - Week 8

Question 1

What is alignment for features?

Answer 1

Fitting a model to a transformation between pairs of features (matches) in two images

Question 2

What are the four 2D transformation models?

Answer 2

Euclidean (Translation + rotation)
Similarity (Translation + rotation + scale)
Affine (Translation + rotation + scale + shear)
Projective (Homography)

Question 3

What makes a transformation T global?

Answer 3

It is the same for any point p
Can be described by just a few numbers (parameters)

Question 4

With what matrix can linear transformations be represented?

Answer 4

A 2x2 matrix

Question 5

What are the linear transformations?

Answer 5

Scaling
Rotation
Shear
Mirror

Or any combination of the above

NOT transformation in 2D space

Question 6

What are the properties of linear transformations?

Answer 6

Origin maps to origin
Lines map to lines
Parallel lines remain parallel
Ratios are preserved
Closed under composition

Question 7

How is the homogenous image coordinate [x, y, w] converted to cartesian coordinates?

Answer 7

[x, y, w] => (x/w, y/w)

Question 8

How is 2D translation represented in 3x3 matrices?

Answer 8

[1 0 tx]
[0 1 ty]
[0 0 1]

Question 9

What is an affine transformation?

Answer 9

Any transformation that can be represented with last row
[0 0 1]

Combinations of linear transformations and translations

Question 10

What are the properties of affine transformations?

Answer 10

Origin does not necessarily map to origin

Lines maps to lines
Parallel lines remain parallel
Ratios are preserved
Closed under composition

Question 11

When 3x3 matrix transformations are extended so any value can be changed, what are these types of transformations called?

Answer 11

Plane Projective Transformations - Homographies - Planar Perspective Maps

They map one plane to another through a point

Question 12

What are the properties of homographies?

Answer 12

• Origin does not necessarily map to origin
• Lines map to lines
• Parallel lines do not necessarily remain parallel
• Ratios are not preserved
• Closed under composition

Question 13

What are inliers and outliers in feature matching?

Answer 13

Inliers are features that were matched correctly

Outliers are features that were incorrectly matched

Question 14

What is the idea behind RANSAC?

Answer 14

Find the line with the largest number of points that agree with the line (are within a small distance to the line) - also called inliers.

If the set of local feature matches contains a very high percentage of outliers:
All the inliers will agree with each other; the (hopefully) small number of outliers will (hopefully) disagree with eachother
- RANSAC only has guarantees if <50% of points are outliers

Question 15

What is the RANSAC loop?

Answer 15

1. Randomly select s sample matches
   - s == minimum sample size that lets you fit a transformation model
2. Compute transformation from sample group
3. Find inliers to this transformation
4. If the number of inliers is sufficiently large, re-compute least-squares estimate of transformation on all the inliers
5. Repeat N times
6. Keep the transformation with the largest number of inliers

N is determined based on the number of outliers expected and the probability of success we’d like to guarantee
- Say 20% outliers and we want the correct answer with at least 99% probability.

Question 16

How can we compute a transformed image I2 from I1 given the coordinate transform (x’, y’) = T(x, y)

Answer 16

Forward warping or Inverse warping

Question 17

What is forward warping?

Answer 17

Send each pixel (x, y) to it’s corresponding position in the warped image.

What if a pixel “lands” between two pixels?
add “contribution” to several pixels, normalise later (splatting).

Splatting can still result in holes

Question 18

What is inverse warping?

Answer 18

Get each pixel g(x’, y’) from it’s corresponding location
(x, y) = T^-1(x, y) in f(x,y)

What if a pixel “lands” between two pixels?
Resample colour value from the interpolated (pre-filtered) source image

Question 19

What are the steps for creating panoramas?

Answer 19

Step 1: Detect Features
Step 2: Match Features
Step 3: Compute a homography using RANSAC
Step 4: Combine the images together (image blending)

Question 20

How does alpha blending work?

Answer 20

Ia = alphaI1 + (1-alpha)I2 for each pixel with 0 <= alpha <= 1

Question 21

In alpha blending what is the window size?

Answer 21

How alpha changes over time

Question 22

What properties should the alpha window have?

Answer 22

To avoid seams: window = size of largest prominent feature
To avoid ghosting: window <= 2*size of smallest prominent feature