Probability in Machine Learning

Question 1

Random Variable

Answer 1

A variable associated with a population. For example, students could have the random variable “grade”.

Question 2

Marginal Probability

Answer 2

P(A)

Question 3

Joint Probability

Answer 3

P(A and B)

Question 4

Conditional Probability

Answer 4

P(A | B)

Question 5

Joint Probability Distribution

Answer 5

A table which demonstrates the probability of the combinations of two events. For example, we could have grade = {A, B, C} and intelligence = {Low, High}. It shows every combination of these two variables.

Question 6

Chain Rule

Answer 6

P(A_4, A_3, A_2, A_1) = P(A_4 | A_3, A_2, A_1) . P(A_3 | A_2, A_1) . P(A_2 | A_1) . P(A_1)

Question 7

Independent Random Variables/ Conditional Independence

Answer 7

If P(A|B) = P(A) then we know these events are independent.
Same with P(A n B) = P(A)P(B).

Question 8

Probability Queries

Answer 8

Given evidence (values of a subset of random variables), compute distribution of another subset of random variables.

Question 9

MAP Queries

Answer 9

Maximum a posteriori probability. Also known as MPE (Most probable Explanation), it asks what is the most likely setting of a subset of random variables?

Question 10

Probability Query Parts

Answer 10

Evidence -> the subset of variables
Query Variable -> another subset of random variables

Question 11

Marginal Probability Estimation

Answer 11

Using the product rule:
P(Y = y_i | E = e) = P(Y = y_i, E = e) / P(E = e).

Question 12

Max

Answer 12

The highest point of a function

Question 13

Argmax

Answer 13

The x values of a function which give the highest y points.

Question 14

MAP Queries Questions

Answer 14

MAP(A,B) = argmax_a,b P(A, B) = argmax_a,b P(B|A)P(A)…