Evaluating

Question 1

What is Bias-variance decomposition?

And what are the names of the differentces in errors between the sources?

Answer 1

The difference in the error between the sources.
So the difference between the irreducible error (the best that you can achieve - for example the human error - can get better than that becomes the data set is labeled with that)
And the train error - is avoidable bias (under fitting)
The difference between the train and validation error is overfitting
And the difference between the validation and test error is validation set overfitting

Question 2

Bias variance with distribution shift, how to evaluate the model when the train and test are from different distributions?

Answer 2

In the validation set make sure to add some of the test set, even if it’s small.than the difference between the regular validation error and this new set error will be the “distribution shift”