Probability

Question 1

Mutually exclusive/disjoint

Answer 1

Two events cannot occur together.

I.e. coin flip heads or tails

Question 2

Finite Geometric Series

Answer 2

Sn=a1(1−r^n)/(1−r), r≠1

Given the geometric sequence 2,4,8,16,…2,4,8,16,… .

To find the common ratio, find the ratio between a term and the term preceding it.

r=4/2=2

Question 3

Infinite Geometric Series

Answer 3

S=a1/(1−r)

Question 4

Conditional Probability (Prob A occurs, given B is known)

Answer 4

When we know that B has occurred, every outcome that is outside B should be discarded. Thus, our sample space is reduced to the set B. Now the only way that A can happen is when the outcome belongs to the set A ∩ B.

Question 5

Conditional Probability Rules

Answer 5

P(A^c|C) = 1 −P(A|C);
P(∅|C) = 0;
P(A|C) ≤ 1;
P(A −B|C) = P(A|C) − P(A ∩ B|C);
P(A ∪ B|C) = P(A|C) + P(B|C) − P(A ∩ B|C);
if A ⊂ B then P(A|C) ≤ P(B|C)

Question 6

Conditional Probability Chain Rule (when we know the Cond Prob)

Answer 6

P(A ∩ B) = P(A)P(B|A) = P(B)P(A|B)

P(A ∩ B ∩ C) = P(A ∩ (B ∩ C)) = P(A)P(B ∩ C|A)

P(A1 ∩ A2 ∩⋯∩ An) = P(A1)P(A2|A1)P(A3|A2,A1)⋯

…P(An|An−1An−2⋯A1)

Question 7

If A ⊂ B,

then P(B ∩ A) =

Answer 7

P(A)

Question 8

P(A −B)

Answer 8

Events in A minus events also in B

P(A −B) = P(A) −P(A ∩ B).

Question 9

Closure (math)

Answer 9

In mathematics, closure describes the case when the results of a mathematical operation (function) are always defined.

For example, in ordinary arithmetic, addition on real numbers (domain) has closure: whenever one adds two numbers, the answer is a number. The same is true of multiplication.

Division does not have closure, because division by 0 is not defined.

In the natural numbers (domain), subtraction does not have closure, but in the integers (domain), subtraction does have closure. Subtraction of two numbers can produce a negative number, which is not a natural number, but is an integer.

Question 10

Prior

Posterior

Likelihood

Evidence Factor

(DATA603)

Answer 10

P(ωj)-Prior knowledge of how likely we are to achieve ωj state of nature.

P(ωj|x)- the state of nature being ωj given that feature value x has been measured.

p(x|ωj)- the category ωj for which p(x|ωj) is large is more “likely” to be the true category.

p(x)- a scale factor that guarantees that the posterior probabilities sum to one, as all good probabilities must.

Pattern Classification (p. 48). Wiley. Kindle Edition.

Question 11

Class-conditional probability density function,

Answer 11

x to be a continuous random variable whose distribution depends on the state of nature and is expressed as p(x|ω).*

probability density of measuring a particular feature value x given the pattern is in category ωi.

Pattern Classification (p. 47). Wiley. Kindle Edition.

Question 12

(Joint) probability density of finding a pattern that is in category ω_j and has feature value x… p(ω_j, x). (DATA603)

Answer 12

p(ω_j, x) = P(ω_j|x)p(x) = p(x|ω_j)P(ω_j).

Conditional densities p(x|ω_j) for j = 1, 2.

Pattern Classification (p. 47). Wiley. Kindle Edition.

Question 13

Bayes Formula

Answer 13

Pattern Classification (p. 48). Wiley. Kindle Edition.

Question 14

How to formally justify posteriors?

Answer 14

If P(ω2|x) > P(ω1|x), we would be inclined to choose ω2. To justify this decision procedure, let us calculate the probability of error whenever we make a decision.

Whenever we observe a particular x, the probability of error is Equation (4)

we can minimize the probability of error by deciding ω1 if P(ω1|x) > P(ω2|x) and ω2 otherwise. Of course, we may never observe exactly the same value of x twice. Will this rule minimize the average probability of error? Yes, because the average probability of error is given by Equation (5)

Pattern Classification (p. 49). Wiley. Kindle Edition.

Question 15

Bayes Decision Rule for minimizing probabiliy of error:

Answer 15

Under Bayes decision rule Equation (6), P(error|x) becomes equation (7)

Question 16

If for some x we have p(x|ω1) = p(x|ω2), then

Answer 16

that particular observation gives us no information about the state of nature;

in this case, the decision hinges entirely on the Prior probabilities.tion.

Pattern Classification (p. 51). Wiley. Kindle Edi

Question 17

If P(ω1) = P(ω2), then

Answer 17

the states of nature are equally probable; in this case the decision is based entirely on the likelihoods p(x|ωj).

Pattern Classification (p. 51). Wiley. Kindle Edition.

Question 18

Formally, the loss function, λ(α_i|ω_j) . states

Answer 18

exactly how costly each action is, and is used to convert a probability determination into a decision.

describes the loss incurred for taking action α_i when the state of nature is ω_j

Pattern Classification (p. 52). Wiley. Kindle Edition.

Question 19

In decision-theoretic terminology, an expected loss is called a _____, and R(α_i|x) is called the _________.

Pattern Classification (p. 53). Wiley. Kindle Edition.

Answer 19

risk

conditional risk

we can minimize our expected loss by selecting the action that minimizes the conditional risk.

Pattern Classification (p. 53). Wiley. Kindle Edition.

Question 20

A general decision rule is a function α(x) that tells us

Answer 20

which action to take for every possible observation. To be more specific, for every x the decision function α(x) assumes one of the a values α1,…, αa.

The overall risk R is the expected loss associated with a given decision rule.

Pattern Classification (p. 53). Wiley. Kindle Edition.

Question 21

Because R(α_i|x) is the conditional risk associated with action α_i and because the decision rule specifies the action, the overall risk is given by

Pattern Classification (p. 53). Wiley. Kindle Edition.

Answer 21