Linear Regression vs. kNN regression

Question 1

When it comes to kNN regression, what is the “curse of dimensionality”?

Answer 1

When you have a large # of predictors and relatively few observations, there often isn’t a nearby neighbor for each data point which leads to very poor predictions.

As a general rule, parametric methods will tend to outperform non-parametric approaches when there is a small number of observations per predictor.