Losses

Question 1

What is the purpose of Mean Squared Error (MSE) in regression?

Answer 1

MSE calculates the average squared difference between predicted and actual values, penalizing larger errors more heavily.

Question 2

What does Mean Absolute Error (MAE) measure in regression tasks?

Answer 2

MAE measures the average absolute difference between predicted and actual values, providing a more robust metric against outliers compared to MSE.

Question 3

What is Log-Cosh Loss, and why is it used?

Answer 3

Log-Cosh Loss is which behaves as MSE at small values and as MAE at large values, and differentiable everywhere.

Question 4

What is Binary Cross-Entropy used for?

Answer 4

Binary Cross-Entropy measures the difference between predicted probabilities and true lables (turned into 1 and 0) for binary classification tasks.

Question 5

What does Categorical Cross-Entropy measure in classification?

Answer 5

Categorical Cross-Entropy is a loss function to evaluate the performance of a model by comparing predicted probabilities with actual one-hot encoded labels.

Question 6

What is Focal Loss designed to address in classification?

Answer 6

Focal Loss focuses on hard-to-classify examples by reducing the loss contribution from easy examples, helping with class imbalance.

Question 7

What is the purpose of Triplet Loss in ranking tasks?

Answer 7

Triplet Loss optimizes the relative distances among anchor, positive, and negative samples to improve ranking models.

Question 8

What is IoU Loss and how is it used in object detection?

Answer 8

IoU Loss measures the intersection-over-union between predicted and ground-truth. It is used in OD for bounding boxes, focusing on localization accuracy.

Question 9

What is Smooth L1 Loss, and where is it applied?

Answer 9

Smooth L1 Loss is used in object detection to be more robust to outliers.

Question 10

What is the role of Focal Loss in object detection?

Answer 10

Focal Loss helps address class imbalance by focusing training on hard-to-detect objects.

Question 11

What is Dice Loss?

Answer 11

Dice Loss is 2 times the intersection over the sum.

Question 12

What does Jaccard Loss measure?

Answer 12

Jaccard Loss, or IoU loss, is the intersection of union of 2 sets.

Question 13

What is Tversky Loss, and how does it differ from Dice Loss?

Answer 13

Tversky Loss generalizes Dice Loss by adding an option to give different weights to false positives and false negatives, making it suitable for imbalanced data.

Question 14

What is Adversarial Loss in generative models?

Answer 14

Adversarial Loss is used in GANs to optimize the generator and the discriminator together.

Question 15

What is Reconstruction Loss in generative models?

Answer 15

Reconstruction Loss measures how well the generated output matches the input, commonly used in Autoencoders.

Question 16

What is KL Divergence?

Answer 16

KL Divergence measures the difference between two probability distributions.

Question 17

What are Weighted Losses, and why are they used?

Answer 17

Weighted Losses assign different importance to specific samples or classes, addressing class imbalance or sample-specific priorities.

Question 18

What is Multi-Task Loss in custom loss functions?

Answer 18

Multi-Task Loss combines losses from multiple objectives, balancing the training of different tasks in a single model.

Question 19

What is the generator goal in the adversarial loss in GAN

Answer 19

To minimise log(1-D(G(z)))

Question 20

What is the discriminator goal in the adversarial loss in GAN

Answer 20

To maximise log(D(x)) + log(1-D(G(z)))

Question 21

Why are we using MSE for loss in reconstruction of images in Auto-Encoder?

Answer 21

Because we assume that the pixel values follow a Gaussian distribution and becacuse we want to maximise the negative log likelihood of the input with the output.

Question 22

How maximising the negative log likelihood turns into MSE?

Answer 22

Plugging the Gaussian exponent into the log function gives back the squared distance between the output and the label