Bayes theorem Flashcards
Conditional probability, what is the formula? Pr(E|F)
Probability of E and F dividuded by the probability of F
What do we mean by partition
Something which satisfies the following requirments:
- Events are disjointed – no overlap
- If you combine all events together using the union – that gives us back the full sample space
What is partition used for
Partition is needed for the law of total probability
What is the law of total probability
Used if we have a series of events that form a partition (e1, e2, e3…eN) and an event F that happens across these events. Then we can use the law of total probability to decode the probability of F.
Useful when you dont kmow the Pr(F) but know the conditional probabilities of F given Ei
What does law of total probability help us achieve?
Helps us know the probability of an event, that happens among other events that form a partition.
Uses both conditional probability and partition to achieve this.
-Uses condiitional probability to find the Pr(F|Ei)
-Then uses partition to sum these up –> get the probability of Pr(F)
Explain how bayes theorem is built on conditional probability
We can determine the probability of B given A using conditional probability.
Bayes theorem allows us to determine the reversed probability, A given B.
Exaplain how basyes theorem uses the law of total probability
The denominator for bayes is the probability of the given event occuring as a whole. If the probability for that event is only known via conditional probabilities with other events that form a partition we can determine the probability of the unique event using the the law of total probability.
Formula for partition
Formula for bayes theorem
Do y ou need partition to use bayes theorem
yes
What is the likelihood function
-You have some data (random variables) that depend on a parameter.
- You assume this data follows some probability model—a mathematical rule that
describes how likely different outcomes are.
- You observe some actual data points (real values from this model).
- The likelihood function is just the joint probability (or density) of all these observations, but viewed as a function of the parameter.
Basically its the joint distribution of random variables
The likelihood function if f(X| theta) viewed as a function of theta
What is the joing pdf of random variables X and Y where both are random variables?
f(x, y) = fx(x)fy(y)
because theyre independent you can get the joint pdf by multiplying the each pdf together
Why is it important to have independent observations in a sample
Because the likelihood function, computing the joint pdf of all observations assumes observatiosn are independent. It gives us the best estimate for unknown parameter when we have independence.
What is a probability model
a parametric family of distributions
What is the first step of statistical inference
Specifying the probability model, Need to specify what distribution the data comes from
How does the joint probability change if your data is continuous or discrete
The joint probability model will either be continuous (PDF) or discrete (PMF)
What is the diffference between the joint distribution of the below data and the likelihood function of it
How do you compute the likelihood function of f(X|theta) ?
Depends on the probability model you assume for your data/observation and if your data is independent
