Probabilistic Reasoning

Question 1

sample space

Answer 1

set of possible outcomes

Question 2

random variable

Answer 2

result of a random experiment

Question 3

event

Answer 3

any subset of points in a sample space

Question 4

probability density function

Answer 4

for continuous random variables distribution is expressed implicitly though a prob density function that returns the likelihood of an outcome being close to the given value

Question 5

probability distribution function

Answer 5

for discrete random variables distributed expressed explicitly through probability distribution function that returns as prob of an outcome being a given value

Question 6

likelihood

Answer 6

joint density of observed data as function of model parameters

Question 7

joint distribution

Answer 7

distribution function over 2 or more random variables

Question 8

P (a or b)

Answer 8

P(a) + P(b) - P(a,b)

Question 9

P(a and b)

Answer 9

P(a) * P(b)

Question 10

conditional probability and formula

Answer 10

probability of an event occurring given that another event has already occured
P(a|b)= P(a and b)/ P(b)

Question 11

what is bayes rule and derive it

Answer 11

P(a|b)= P(b|a)P(a) / P(b)

start with
P(b|a)= P(alb)/P(a) and P(a|b)= P(alb)/ P(b)

Question 12

what is posterior part

Answer 12

P(cause | effect) it is the probability hypothesis given some evidence

Question 13

what is likelihood part

Answer 13

P(effect | cause) this is the likelihood hat effect will occur if cause if true

Question 14

what is prior belief part

Answer 14

P(cause) in top row prior belief in some cause

Question 15

what is evidence part

Answer 15

P(effect) on bottom is the probability evidence across all possible causes

Question 16

why is bayes rule helpful?

Answer 16

helps convert problem that’s hard to measure to be computed from something that’s easy to measure

Question 17

BR is an _____ to

Answer 17

update to prior belief given new info

Question 18

example of bayes rule classifier

Answer 18

determine if patient has a disease based on + test
P(D|T) = P(T|D)P(D)/ P(T)

Question 19

joint probability distribution

Answer 19

represents probability of different events occurring together

Question 20

curse of dimensionality

Answer 20

as number variables increased size of JPD grows exponentially
if have n variables have to consider 2^n combinations

Question 21

independence

Answer 21

2 events independent if occurrence of one does not affect probability of the other

Question 22

independence rules

Answer 22

P(A,B) = P(A) * P(B)
formally a and b are independent if P(A|B)= P(A)

Question 23

independence in terms of conditional probability

Answer 23

knowing that event B happened doesn’t affect probability of A

Question 24

events are conditionally independent given a third event if…

Answer 24

P(X,Y|Z)= P(X|Z) P(Y|Z)

Question 25

how naive bayes classifier simplifies computation?

Answer 25

by assuming all features were conditionally independent of each other
each feature independently contributes to the likelihood of the class
decreases complexity of computation by transforming computation into a series of independent likelihood calculations for each feature

Question 26

naive bayes classifier bias and variance is

Answer 26

low bias
high variance

Question 27

naive bayes classifier assumption and real life?

Answer 27

assumption not always true in real life as features often correlated/dependent
can lead to suboptimal classifications if there are strong dependencies between features (e.g cough and fever are not independent and may both contribute to the same underlying cause)

Question 28

how to do spam/ham using NBC

Answer 28

1) calculate priors P(spam) and P(not spam)
2) calculate likelihoods
- for each feature (word) calculate P(word|spam) = count of words in spam/ total count of words in spam
P(word|not spam)
3) use bayes to computer posterior
- for a new email calculate prob of being spam/not spam
P(spam | word1, word2) sideways infinit P(spam) * P(w1|spam) * P(w2|spam) .,…
4) decision