Motion

Question 1

Why is motion information usefull?

Answer 1

For action recognition and object tracking

Question 2

What are the basic assumptions behind optical flow?

Answer 2

Brightness constancy and small displasement

Question 3

What do we know about I(x+u, y+v, t+1) given displacement (u,v) and the brigthness constancy assumption?

Answer 3

It is the same as I(x,y,t)

Question 4

What is the optical flow constraint?

Answer 4

dI/du * u + DI/dv * v + dI/Dt = 0

Question 5

What assumption does the the Lucas Kanade method make?

Answer 5

Flow, or displacement, is constant in a local neighbourhood

Question 6

What equation do we have to solve in the Lucas Kanade method

Answer 6

Ax = b -> x = (A^TA)^-1 A^Tb

where A = [Ix(p1) iy(p1); Ix(p2) Iy(p2)….]
b = [-It(p1); I(t(p2); ……]

Question 7

What is the condition number of a 2-d matrix?

Answer 7

abs(lambda_max) / abs(lambda_min)

Question 8

How can the condition number be interpreted?

Answer 8

Large conditon number makes the matrix inverse sensitive to noise and small changes in values.

Question 9

Describe the implementation of Lucas Kanade method.

Answer 9

1) Calculate Ix, Iy and It.
2) For a small window calculate:
AA^T = sum_p [ Ix^2, IxIy
IxIy, Iy^2]
A^tb = sum_p [Ix It, Iy * It]
3) Calculate the optical flow:
[u,v] = (A^TA)^-1 A^Tx

Question 10

What is the idea behind the Horn- Schunk method?

Answer 10

Define a global energy function:

Integral_(x,y) ((Ix*u) + (Iy * v) + I_t)^2 +
alpha(abs(gradient(u))^2 + abs(gradient(u))^2) dxdy

And minimize the energy with respect to u(x,y), v(x,y).

Question 11

What can we do if the small displacement condition does not hold?

Answer 11

Use spatial pyramids of downsampled images. Estimate motion from the coarsest (smallest) image first and iteratively to larger versions.

Question 12

How does the Middlebury dataset create GT for optical flow?

Answer 12

It use textured UV paint and UV cameras.

Question 13

How does the Kitty dataset create GT for optical flow?

Answer 13

It uses a lidar.

Question 14

What metric can we use for flow evauation with ground truth?

Answer 14

Average end point error:
sum_x,y sqrt((u-u_gt)^2 + (v_v_gt)^2)
Averaged over all images

Question 15

What metric can we use for flow evauation without GT?

Answer 15

We can skip one frame, calculate optical flow between frame 0 and 2 and interpolate frame 1. We then use the true frame 1 for GT.

Question 16

Derive the optical flow constraint

Answer 16

Constant brightness: I(x+u, y+v, t+1) = I(x,y,t)
Taylor: I(x+u, y+v, t+1) = I(x,y,t) + Ixu + Iyu + It
Combining: Ix + Iy + It = 0