Week 5 - Model based Vision

Question 1

What is model based vision

Answer 1

Visual understanding and interpretation are achieved through the use of explicit models of objects and scenes

Question 2

Wha is Principle Component Analysis

Answer 2

Used for dimensionality reduction
Aims to find the directions (or principal components) that capture the maximum variance in the data

Question 3

What is dimensionality reduction

Answer 3

(2D → 1D)
Given some data
Find a line of best fit v1 which contains the largest variation in values
If v2 (perpendicular) variation is much smaller compared to v1
We: project all points onto v1
approximately reduce the data by only saving v1 (each point is no represented by a single value)

Can also do: 3D → 2D, 4D → 3D

Question 4

When does dimensionality reduction create a better representation

Answer 4

When the ellipse along v1 is narrower

Question 5

What is the PCA Algorithm

Answer 5

1) Assemble data into matrix
2) Compute the covariance matrix
3) Find the Eigenvalues (λi) and Eigenvectors (vi) of C
4) Choose the K largest eigenvalues to account for p% of T
(because we want to reduce the number of dimensions)
For example, we might choose p = 0.95

Question 6

What are the dimensions of the intial PCA matrix

Answer 6

number of samples x number of variables
row x column

Question 7

What is an Eigenvalue (λi)

Answer 7

Represents the magnitude of variance along each direction

Question 8

What is an Eigenvector (vi)

Answer 8

The directions of maximum variance in the dataset

Question 9

How do you calculate total variance along a direction

Answer 9

Σ λi

Question 10

What are the dimensions of the PCA covariance matrix

Answer 10

number of var x number of var
(symmetrical)

Question 11

what is in each segment of a covariance matrix, given 2 var:x,y

Answer 11

cov(x,x) cov(x,y)
cov(y,x) cov(y,y)

Question 12

What is cov(x,x) the same as

Answer 12

var(x)

Question 13

How do you calculate cov(x,y)

Answer 13

cov(x,y) = E[xy] - E[x]E[y]
E[] = average

Question 14

What equation to eigen values and vectors satisfy

Answer 14

Av = λv

To find the eigenvalues we solve

det( A - λI) = 0

where I is an identity matrix

Question 15

what is the formula for determinant

Answer 15

determinant = ad - bc

Question 16

What do the two eigenvalues represent

Answer 16

the magnitude of the variance in direction v1 and v2
(eigenvector v1 is perpendicular to eigenvector v2)

Question 17

How do you calculate the proportion of variance in the v1 direction

Answer 17

eigenvalue(v1) / (eigenvalue(v1) + eigenvalue(v2))
eg 0.86
means if we projected the data points onto v1 we would still retain 86% of the variation in the data

Question 18

What are Active Shape models

Answer 18

Allow for non-rigid shape matching
eg annotated dataset of female faces

Are able to generate different (valid) shapes based on supplied shapes in training dataset

Question 19

What do active shape models not allow

Answer 19

shapes to be scaled, rotated, translated etc

Question 20

How do we use PCA in ASMs

Answer 20

Assemble data in matrix (number of variables x number of data samples)
Compute covariance matrix (number of var x number of var)
Find eigenvalues solve det(C - λI) = 0

so there will be (number of var) eigenvalues
Which can be a lot so we choose the K largest eigenvalues

Question 21

How do we then generate new shapes using eigenvalues

Answer 21

x = x̄ + Vb

Where x̄ is the mean shape
Eigenvectors vi are the column vectors in V
and b is the shape parameter vector

the mean shape is calculated from the original matrix of data D

Question 22

What is the shape parameter vector b

Answer 22

Size (number of var x1)
A column vector of coefficients that scales the eigenvectors (or eigenmodes) of shape variation

Question 23

What are the modes of variation of a shape

Answer 23

The effect of changing b on the shape
PCA finds these automatically
eg b1 changes the vertical position

Question 24

How do we use shape fitting to find a shape in a new image

Answer 24

We can use an iterative localised search in the image:

• Place the shape into the image
• Search in the neighbourhood of current feature points for better locations
• Fit the model to the new suggested shape
• Repeat until convergence occurs

Question 25

What parameters do we need to be able to change when placing a model in an image

Answer 25

(s, θ, r, b)
As well as changing the values in b, we also need to allow for the “pose parameters”:
scaling (s), rotation (θ) and translation (r)

Question 26

What does b=0 mean about the shape

Answer 26

It is the mean shape

Question 27

How do we fit a model to the image

Answer 27

• Calculate normals to the model curve at each point (current points are x)
• search along the normal for the strongest edge (trying to find the edge of the actual shape)
• This gives a set of suggested points x’
  
  • But we do not just use as x’ as this may not give a valid face shape
• First, (s, θ, r) are found to best fit x to x’
  
  • rigid transformation to get closer, gives x’’
    Use the model backwards
  • b = V−1 (x’’ − ¯x)
  • ¯x is the mean face
• New b generates a new valid shape
• Iterate until convergence

Question 28

How is using the model backwards useful

Answer 28

originally: x = x̄ + Vb
When we calculate x’’ using rigid transformations and normals
we can calculate b using:
b = V−1 (x’’ − ¯x)

And then find a new set of suggested points using
x̄ + Vb (with updated b) to get closer to actual shape

Question 29

How is ASM used in the real world

Answer 29

Medical application: ASM are used a lot in the medical field
eg positioning an artificial hip

Question 30

What is the Active Appearance Model

Answer 30

• face is split into triangles
• texture within triangles is warped to fit the model
• imagine face printed on a rubber sheet and stretched about → used to change expressions

Question 31

How are different shapes creates in ASM

Answer 31

By changing b