L10 - CNN’s 2

Question 1

1. What is a Feature Map?

Answer 1

1. Map of the match between filter and image at every spot.
   
   2. Represents where that particular pattern was found.
   3. Every filter results in 1 feature map

Question 2

2. What is the ReLU function? What’s its purpose?

Answer 2

1. An activation function that introduces non-linearity into a neural network
   
   2. Enables modelling of more complex relationships in data
   3. Determines quality of the filter match

Question 3

3. How does the ReLU activation function operate?

Answer 3

1. Converts all negative values to 0
   
   2. Leaves all positive values

Question 4

4. What is the ReLU applied to?

Answer 4

1. A feature map, resulting in a feature map with values of only 0 and above

Question 5

5. What is downsampling?

Answer 5

1. The process of reducing the dimensionality of data

Question 6

6. Why is downsampling important in Computer Vision?

Answer 6

1. Images are very data intensive, thus, have high dimensionality.
   
   2. Reducing dimensionality improves performance and makes processing more manageable

Question 7

7. What are some issues with downsampling?

Answer 7

1. Information loss -> Reducing dimensionality reduces the information of the image which can lead to poor model performance is vital details and patterns are removed.
   
   2. Overfitting -> Reducing data may make model prone to learning noise rather than underlying patterns of image.
   3. Introduce bias

Question 8

8. What is considered the correct technique of downsampling?

Answer 8

1. Pooling

Question 9

9. What is Pooling?

Answer 9

1. Technique for reducing dimensionality of image data.
   
   2. Applied after ReLU function has been applied to the feature map in order to further decrease the dimensions.

Question 10

10. What is the general idea of pooling?

Answer 10

1. Aggregates a group of pixel via their average or max.

Question 11

11. When is pooling applied?

Answer 11

1. After ReLU function has been applied to the feature map.

Question 12

12. What is the end result of downsampling?

Answer 12

1. A pooled feature map that is small enough so it becomes a traditional ML problem.

Question 13

13. What is Padding and Stride?

Answer 13

1. Padding -> Increases data dimensions so filter can better pick up border features of image.
   
   2. Stride -> Move the filter a customer amount over the image

Question 14

14. What are the effects of a higher stride or lower stride?

Answer 14

1. Higher -> Reduces feature map size, picks up less information, reduces computational cost.
   
   2. Lower -> Increases feature map size, picks up more info, higher computational cost.

Question 15

15. What is the purpose of an activation function? What are the 6 main ones?

Answer 15

1. To introduce non-linearity into a model, enabling the identification of more complex relationships.
   
   2. ReLU, Leaky ReLU, Tanh, Maxout, Sigmoid, ELU

Question 16

16. What is the Softmax Function? How does it work?

Answer 16

1. An activation function used in neural networks tasked with multi-classification problems.
   
   2. Transforms a real number vector into probabilities of belonging to certain classes.
   3. For example: Classes are defined by the networks fully connected layer, the Softmax function is applied to each of the class values to obtain the probabilities of an input being in a class.

Question 17

17. Define the learning rate of a model…

Answer 17

1. How much the model updates it parameters at every step.

Question 18

18. What is the trade off between large and low learning rates?

Answer 18

1. Large -> May overshoot optimal values
   
   2. Small -> Takes long time to final optimal and can get stuck at local optimal

Question 19

19. What technique can we use to ensure our learning model uses both large and small values appropriately?

Answer 19

1. Decaying at every step -> Initial large learning rate for speed, and decrease rate at each step so we don’t overshoot.
   
   2. I.e Decay the learning rate at every step