- Local, meaningful, detectable parts of the image - Location of a sudden change - ‘salient patches’

Week 5 and 6 - Local Features Flashcards by Isabel Draper

What is the motivation of using local features

Global representations have limitations
Local features only describe and match local regions

How well did you know this?

Not at all

Perfectly

What are local invariant descriptors

another term for local features

How well did you know this?

Not at all

Perfectly

What is Image-based object recognition

objects are recognized based on their appearance in images rather than explicit geometric models

How well did you know this?

Not at all

Perfectly

What is Model-based object recognition

objects are recognized by comparing them to predefined models or templates

How well did you know this?

Not at all

Perfectly

What is a feature

Local, meaningful, detectable parts of the image
Location of a sudden change
‘salient patches’

How well did you know this?

Not at all

Perfectly

What is a salient patch

region or area within an image that stands out due to its significance

How well did you know this?

Not at all

Perfectly

Why do we use features

Information Content High
Invariant to change of view point, illumination
Reduces computational burden

How well did you know this?

Not at all

Perfectly

What is Visual SLAM

An application of features
“Simultaneous Localisation and Mapping”
Estimating the local geometry and fusing it into a 3D model
Used in augmented reality and autonomous vehicles
Eg holding up phone camera and it being able to map the geometry of the shape in screen

How well did you know this?

Not at all

Perfectly

What is Image matching

An application of features
Reverse google searching to find a monument
uses local image features

How well did you know this?

Not at all

Perfectly

How is NASA using features

Local features are currently being used on the NASA Mars Rover - trying to stitch together to find a panorama of Mars landscape

How well did you know this?

Not at all

Perfectly

What are feature points used for

Image alignment (homography, fundamental matrix)
3D reconstruction
Motion tracking
Indexing and database retrieval
Robot navigation
… other

How well did you know this?

Not at all

Perfectly

What is Image stitching

Procedure:
- Detect feature points in both images
- Find corresponding pairs
- Use these pairs to align the images

How well did you know this?

Not at all

Perfectly

General approach to feature-based image matching

Find a set of distinctive keypoints
Define a region around each keypoint
Extract and normalise the region content
Compute a local descriptor from the normalised region
1. Has to be unique
Match local descriptors

How well did you know this?

Not at all

Perfectly

What does it mean to normalise the region content

Make the descriptor invariant to certain transformations (e.g., scale, rotation, illumination)

How well did you know this?

Not at all

Perfectly

What are the 2 main problems in feature matching

1) Detect the same point independently in both image
2) For each point, correctly recognise the corresponding one

How well did you know this?

Not at all

Perfectly

What is a repeatable detector

Detector that guarantees that it will always find the interest point if present

How well did you know this?

Not at all

Perfectly

What are the 4 requirements of region extraction

Repeatable
Invariant to translation, rotation and scale changes
Robust or covariant to out-of-plane transformations
Robust to lighting variations, noise, blur and quantisation

How well did you know this?

Not at all

Perfectly

What are the 5 requirements of local features

-requirements of region extraction fulfilled
- Locality
- Quantity
- Distinctiveness
- Efficiency

How well did you know this?

Not at all

Perfectly

What are the 3 main types of detector:

Harris
Laplacian
DoG (difference of gaussians)

How well did you know this?

Not at all

Perfectly

What is the important first step in feature detection

Finding candidate locations

Why are edges not ideal candidate locations

Edges only localise in one direction

What are corners

Repeatable points, good candidate locations
Around a corner, image gradient has two or more dominant directions

How does the Harris Corner Detector work

Iterates through an image with a window
Calculate the change in intensity for the shift [u,v]
Minus the intensity from the shifted (new) intensity
Approximate this shift using a matrix M
Calculate the “corner response”

What are the two types of window function for harris detector

Option 1) Basic Harris Corner detector
binary: 1 in window, 0 outside
SVD
compute M and its eigenvalues

Option 2) Smooth with Gaussian Kernel
Bell curve over window
Use smoothed derivatives in M

What is the matrix M in harris corner detection

M = Σw(x,y) [Ix² IxIy, IxIy Iy²] Where IxIy is gradient w.r.t x times gradient w.r.t y

What is the equation for approximating the shift in harris corner detection

E(u,v) ~= [u,v] M [u, v] Second [u,v] is a column vector

What do we assume about the datapoints in harris detection

They are coming from a Multivariate Gaussian model So we can use the eqns for mean (miu^) and covariance (sigma^)

Why do we want to find the covariance matrix in harris corner detection

The covariance matrix describes how the intensity changes are distributed across these different directions This matrix gives us Eigenvectors (directions that remain unchanged except for a scaling factor by the corresponding eigenvalue) We essentially are shifting our data along the long axis of the ellipse (eigenvector) in order to calculate variance The full covariance situation is not ideal

Why are eigenvectors important for changes in intensity calculations

eigenvectors = principal axes along which the intensity changes have the most significant variance

What is Singular value decomposition

SVD A method of decomposing any matrix into 3 simpler matrices

What 3 matrices are used in SVD in harris corner detection and what do they represent

A = U . D . U^T u1, u2 (columns of U) capture the axes of the ellipse - Eigenvectors d1,d1 (columns of D) determine the scale and variance of the data - Eigenvalues

What is the purpose of SVD in Harris detection

SVD = eigenvalue decomposition of matrix M Tells us the direction and scales of the variation gives us a new coordinate system u1,u2

How do we use eigenvalues λ1,λ2 (d1,d2) to find a corner

- If both eigenvalues are large, it suggests the presence of a corner (E increases in all directions) - If one eigenvalue is large and the other is small, it indicates an edge - If both eigenvalues are small, it represents a flat region

What is the equation for R to measure the corner response

R = det(M) - k x (trace(M))² R = λ1λ2 - α(λ1 + λ2)²

What is detM

λ1λ2

What is traceM

λ1 + λ2

In the equation for R, what are the best values for constant k

k (also α) is a constant typically set to a value between 0.04 to 0.06

How do we interpret the value of R

R is large → corner R is negative → large magnitude for an edge |R| is small → flat region

What is the problem with taking using basic harris detector (no gaussian)

Not rotation invariant uniform window (1 in window, 0 outside)

How can we use Gaussian smoothing instead over window (option 2)

M = g(σ) * [Ix² IxIy, IxIy Iy²] instead of Σw(x,y) The result is rotation invariant

How does gaussian harris detection (Fast approximation) work (option 2)

1) Image derivatives (blur first) 2) Compute Ix², IxIy, IxIy, Iy² 3) Gaussian filter g(Ix²), g(IxIy),... 4) M(σl, σd) = g(σl) * [Ix²(σd) IxIy(σd), IxIy(σd) Iy²(σd)] Then use M(σl, σd) in R as before: R = det(M(σl, σd)) - k x (trace(M(σl, σd)))²

What are the properties of Harris operator

Rotation invariant Not scale invariant → scaling up will cause corners to be classed as edges The Harris operator does not tell us how much of the surrounding region is involved precise localisation high repeatability

What is locality of features

Features are local, therefore robust to occlusion and clutter

What is quantity of features

We need a sufficient number of regions to cover the object

What is distinctiveness of features

The region should contain ‘interesting’ structure

What is efficiency of features

close to real-time performance

What are local features most robust to than global

-Occlusions -Intra-category variations

σI

scale of Gaussian smoothing applied to the image before computing gradients

σD

scale of derivative operator