Probability and Stats

Question 1

Law of Large Numbers

Answer 1

As sample size grows, sample mean becomes closer to population mean

Question 2

Bayes Rule

Answer 2

P(A|B) = P(B|A) P(A)/P(B)

Question 3

Probability vs Likelihood

Answer 3

Probability =
P(X > 32 | mu, std_dev)
Likelihood = finding best params of models/best distribution given observation(s)
e.g., L(mu=sth, std=sth|X)

Question 4

Bayes Nets

Answer 4

DAG + variables conditioned on others (local conditional probabilities)
e.g., rain -> cricket -> traffic

Question 5

Bayes Error

Answer 5

e.g.,

Question 6

Markov Decision Process

Question 7

Hidden Markov Models

Answer 7

Hidden State (X) = Markov Process
Observable (Y) = only depends on current state of X
Accounts for temporal relations between hidden states and how those states emit observations
e.g., Language modeling, Y = word, X = part of speech
Tall player fell
Hidden Markov Process: P(adj, noun, verb) = P(adj) P(noun | adj) P(verb | noun)
P(“Tall player fell”) = P(Tall | adj) P(player | noun) P(fell | verb)

Question 8

Full joint distribution from Bayes Net

Answer 8

P(x1, x2, … xn) = prod_i (x_i | parents(x_i))

Question 9

Decoding in HMM

Answer 9

Find the most probable sequence of hidden states, given a sequence of observations

Question 10

Confidence Interval

Answer 10

Collect sample means e.g., using bootstrapping (sampling with replacement)
Define a 95% (typical value) interval i.e., interval covering 95% of those sample means
That’s the CI

Question 11

p-value

Answer 11

H0 = no difference; H1 = different
If p < alpha, then reject H0

alpha = acceptable False Positive Rate
i.e., chances of saying there is a difference, even though there is not in reality

cons: not well calibrated

Question 12

Student’s t-test

Answer 12

t = (x_bar - mu) / (estimate_of_population_std_dev/sqrt(n))

Test: if the calculated t is outside the confidence interval for given confidence level (1-alpha) and sample size, then reject H_0

Cons: assumes equal variances of both populations
Pros: For smaller sample sizes. t-distribution approaches normal distribution for large sample sizes

Question 13

Welch’s t-test

Answer 13

Doesn’t assume equal variances. Assumes normal distribution just like student’s t-test

Question 14

Central Limit Theorem

Question 15

Binomial distribution

Answer 15

Pr(x | n, p) = C(n, x) p^x (1 - p) ^ (n - x)
x = # of successes (e.g., prefers orange fanta) out of n, given success probability p

p^x (1 - p) ^ (n - x) = proba that ‘x’ events are successful for a given configuration,
C(n, x) total configurations in which x successes out of n are observed
e.g., orange vs grape fanta

Question 16

Poisson distribution

Question 17

ANOVA

Answer 17

Analysis of Variance: Use variance to study means for 2 or more populations
- H_0 = all means are the same; H_1 = at least one differs from the others

F = variance between treatments/variance within treatments = MS_treatments/MS_ERRORS

Look up threshold for (# treaments -1) and (# of samples within a group - 1) to see if the differences are significant

Question 18

Expectation Maximization

Question 19

Maximum Likelihood Estimation

Question 20

Law of total variance

Answer 20

EV VE
Var(Y) = E(Variance(Y|X)) + Var(E(Y | X))