2.1 Probability and entropy Flashcards
What is marginal probability
Marginal probability is the probability of an event irrespective of the outcome of another variable.
What is joint probability?
Joint probability is the probability of two events occurring simultaneously.
What is conditional probability?
Conditional probability is the probability of one event occurring in the presence of a second event.
What is the conditional probability formula?
What is entropy and what does having a high entropy mean?
Entropy is a measure of uncertainty or unpredictability (highest entropy = least predictable).
Suppose you are given a piece of code that is supposed to generate a random number from 1-10. You notice that it seems to output “10” more often than any other value. To better understand what it is doing, you run the code 1 million times and plot the number of times it produces each value, to obtain the probability distribution shown below.
This is an example of what kind of probability distribution?
- Uniform
- Empirical
- Gaussian
- Bernoulli
Emperical
This is an empirical probability distribution because it is obtained by taking many samples from the real world, and it doesn’t correspond to any of these three theoretical distributions (Gaussian, Bernoulli, uniform). (Though if the code had worked as intended, with all values 1-10 equally likely, it should have produced a uniform distribution.)
Which of these has highest entropy?
- A roll of a biased 6-sided die
- A roll of a fair 20-sided die
- A fair coin flip
- A roll of a biased 20-sided die
Entropy is a measure of uncertainty or unpredictability (highest entropy = least predictable). A roll of a fair 20-sided die is least predictable because it can result in any of 20 states, all equally likely. A roll of a biased 20-sided die is more predictable than the fair die because some states are more likely than others. A coin flip or a roll of a 6-sided die is more predictable than a fair 20-sided die because these have fewer possible states.
A university is interested in knowing what proportion of students own cars, to help them plan how to allocate parking spaces. The university surveys 1000 students and asks them (a) Do you live on campus? and (b) Do you own a car? The number of students answering yes/no to these questions is shown in the table below.
What is the marginal probability P(Live on campus = yes) that a student lives on campus?
The marginal probability is the number of students who responded “Lives on campus = yes” divided by the total number of students in the survey.
P(Lives on campus = yes) = 200 / 1000 = 0.2
A university is interested in knowing what proportion of students own cars, to help them plan how to allocate parking spaces. The university surveys 1000 students and asks them
(a) Do you live on campus?
(b) Do you own a car?
The number of students answering yes/no to these questions is shown in the table below.
What is the joint probability P(Live on campus = yes, Own a car = yes) that a student lives on campus and owns a car?
The joint probability is the number of students who meet both criteria (Live on campus = yes AND Own a car = yes), divided by the total number students in the survey.
P(Live on campus = yes, Own a car = yes) = 20/1000 = 0.02
A university is interested in knowing what proportion of students own cars, to help them plan how to allocate parking spaces. The university surveys 1000 students and asks them
(a) Do you live on campus?
(b) Do you own a car?
The number of students answering yes/no to these questions is shown in the table below.
What is the conditional probability P(Own a car = yes | Live on campus = no), the probability that a student owns a car given that they do not live on campus?
The conditional probability is the number of students who own a car and do not live on campus, divided by the number of students who do not live on campus.
P(Own a car = yes | Live on campus = no) = 200 / 800 = 0.25
