Midterm Practice Problems

Question 1

Describe what the “Bayes classifier” is

Answer 1

The Bayes classifier is a classification procedure that reaches the true minimum misclassification rate. It can be thought of as the underlying model that generates the true categories of the observations.

Question 2

Will the Bayes Classifier result in 0 misclassifications?

Answer 2

Whilethere may be some cases where data is extremely well-separated and thus the Bayes classifier results in 0misclassifications, in general this is not expected

Question 3

Under what assumptions is Linear Discriminant Analysis the Bayes classifier?

Answer 3

If each group (or subpopulation) is assumed to be mul-tivariate normally distributed, and all groups have a common covariance matrix

Question 4

Under what assumptions is Quadratic Discriminant Analysis the Bayes classifier?

Answer 4

If each group is assumed to be multivariate normally distributed with uniquecovariance matrices

Question 5

What is clustering?

Answer 5

Clustering is attempting to separate observations into groups according to the predictors (X) — there is noknown response (Y) that we are actively modelling, it is an exploratory procedure.

Question 6

What is classification

Answer 6

Classification is the process of fitting a model using predictors (X) to predict a categorical response variable(Y)

Question 7

Whats the difference between clustering and classification?

Answer 7

r

Question 8

Example of Clustering

Answer 8

r

Question 9

Example of Classification

Answer 9

f

Question 10

What is a p-value

Answer 10

The p-value for a hypothesis test is the probability of observing a test statistic as extreme, or more extreme(in the direction of the alternative hypothesis), than that which we observed assuming the null hypothesisis true.

Question 11

Suggest a way of finding the ‘best’ number of groups (k) for a data set usingk-means.

Answer 11

Runk-means for all reasonable number of groups we might wish to consider, and record the total within-group sum of squares. View those values graphically, and determine at whichkincreasing the numberof groups further shows little improvement.

Question 12

what is Linear Discriminant Analysis

Answer 12

a method used to find a linear combination of features that characterizes or separates two or more classes of objects or events

Question 13

what is Quadratic Discriminant Analysis

Answer 13

a method used to determine which variables discriminate between two or more naturally occurring groups, it may have a descriptive or a predictive objective.

Question 14

What are LDA and QDA used for?

Answer 14

statistical learning methods used for classifying observations to a class or category

Question 15

what is the response variable to LDA and QDA used for?

Answer 15

categorical

Question 16

Suppose we knew that the Bayes’ classifier was a boundary that was extremely non-linear.If we were usingK-nearest neighbours as a classifier, would you expect a larger or smallervalue ofKto provide a better approximation of the boundary?

Answer 16

As k increases for k-nearest neighbours, we will see increasingly simple (linear looking) boundaries. Therefore , if we have a complicated (extremely non-linear) boundary, then we would expect a relatively small value of k to perform better.