Week 9 DSE

Question 1

What does likelihood mean?

Answer 1

Prob of observing data x if we think that y equals to some number

Question 2

What can bayes rule be used for ?

Answer 2

binary, discrete vairable

Question 3

What does fk (X) mean?

Answer 3

conditional probability mass function of X (if X discrete) for an observation from class k.

Question 4

What is 𝜋𝑘 ?

Answer 4

prior probability that random observation comes from class k

easily estimated under random sampling: just a fraction of the training observations that belong to class k.

Question 5

What is the assumption under naive bayes classifier?

Answer 5

Assumes that within class k, all p predictors are independent.

ELIMINATES THE NEED FOR JOINT DISTRIBUTION

fk(x)= fk1(x1) x fk2(x2) x fk3(x3)

Question 6

What to do when joint distribution is multivariate normal?

Answer 6

Estimate covariance matrix (tough in high dimension)

Question 7

What is another option for quantitative/continuus X besides drawn from normal distribution?

Answer 7

Use non parametric density estimation

make a histogram for observation of jth predictor with each class

or use kernel density estimator

Question 8

How to estimate a qualitative X?
What odes it mean to be qualitative?

Answer 8

qualitative means x discrete

count the proportion of training observations for the j-th predictor

Question 9

What is it called when pi 1 and pi2 both equal 0.5, IF THERE ARE 2 CLASSES

Answer 9

called flat prior

Question 10

What assumption does the naive bayes function in r rely on ?

Answer 10

assumption of Normal distribution for quantitative predictors.

Question 11

What do we ask ourselves when applying statistical methods?

Answer 11

are assumptions valid?

Question 12

What is the purpose of assuming predictors independent conditional on class?

Answer 12

mainly ensures quantitative tractability

Question 13

What is a potential source of high variance? How does naive bayes solve it?

Answer 13

Estimating joint distribution of predictors when P gets larger becomes a very hard problem requiring a lot of data

simplifies this problem, introducing some bias, but reducing variance

Naive bayes is a relatively simple method with little tendency to overfit.

Question 14

When does naive bayes reduce variance drastically?

Answer 14

when N is not large relative to P

Question 15

When does naive bayes work best?(slide 29)

Answer 15

Works best when data has relatively many predictors so that reliable estimation of joint densities of predictors for each class is hard to achieve.