Invariances

Question 1

Why are invariances a problem?

Answer 1

• Human perceptual system very good at dealing with invariances (e.g translation, rotation, scaling, deformation, contrast, lighting); hard to make computers perceive
• Image and speech tasks require the input-output function to be insensitive to irrelevant variations of the input (e.g changes in position, orientation, illumination, or variations in pitch or accent) while being very sensitive to particular minute variations (e.g. white wolfs vs Samoyeds)
• LeCun, Bengio & Hinton (2015)
• Difficult for shallow classifiers - would need a good feature extractor that solves the selectivity-invariance dilemma

Question 2

Selectivity-invariance dilemma

Answer 2

Produces representations that are selective to the aspects of the image that are important for discrimination, but are invariant to irrelevant aspects such the pose of an animal

Question 3

Invariance Solutions

Answer 3

1. Network design - sharing weights
2. Using features
3. Normalization
4. Brute force