Segment/Depth Anything

Question 1

How the focal loss helps in an imbalanced classification

Answer 1

Focal loss is takes Cross Entropy loss and adds a modulating factor that helps to bring the loss further down when an example is classified well. That makes it focus more on hard examples.

Question 2

Why is focal loss good for segmentation

Answer 2

In segmentation we have class a heavy class imbalance because most of the pixels are negative and only a few are positive for the specific class we want.

Question 3

What is the task proposal for segmenting anything

Answer 3

Return a valid segmentation mask given any prompt

Question 4

What can be ambiguous in a point prompt for a segmentation

Answer 4

A point can be the whole object, a part of the object or the sub-part of the object.

Question 5

In the training of the model, how did the vit evolved?

Answer 5

They started with vit-base and moved to vit-huge in the 2nd stage.

Question 6

Write the equation for focal loss & cross-entropy loss

Answer 6

FL=((1-p_t)^gamma)*CE
CE = -log(p_t)
p_t = p if y==1 else 1-p

Question 7

List depth anythin main contributions

Answer 7

Train on both labeled and unlabeled data to create a strong small student network.
Unlabeled training with strong perturbations: gaussian blur, color jitter & CutMix.
Semantic priors loss for encoder comparison.

Question 8

Explain the idea behind semantic prior method

Answer 8

Create a loss from the student model encoder to a frozen big encoder to enforce producing similar feature extraction.

Question 9

Explain in words the equation of the semantic prior method

Answer 9

take a cosine similarity of each feature value and divide it by the amount of features (take 1 minus for loss)

Question 10

How to combat semantic prior when pixels can be in different depths but of the same object

Answer 10

Create an alpha threshold. and when the cosine would be greater then the pixel would not be considered in the loss.

Question 11

List the strong perturbations they used

Answer 11

colorJitter
GaussianBlur
CutMix

Question 12

Explain the steps to create the CutMix loss.

Answer 12

We take 2 unlabeled images
We take the teacher model and predict on both images.
We initialise a mask
We use the student model to predict on the combined images with the mask.
We take the distance of the prediction between: Student network multiplied by mask. Teacher network multiplied by the mask.
The distance is the same depth loss distance.
Do the same for outside the mask.
Combine the results