7.3 Baye's theorem

Question 1

BAYES’ THEOREM

derive it, solve example 1

Answer 1

Suppose that E and F are events from a sample space S such that
p(E) ≠ 0 and p(F) ≠ 0. Then
p(F ∣ E) = p(E ∣ F)p(F) / p(E ∣ F)p(F) + p(E ∣ ~F)p(~F)

check book, page 495.

Question 2

GENERALIZED BAYES’ THEOREM

Answer 2

Note that in the statement of Bayes’ theorem, the
events F and F are mutually exclusive and cover the entire sample space S (that is, F ∪ F = S).
We can extend Bayes’ theorem to any collection of mutually exclusive events that cover the
entire sample space S, in the following way.

Suppose that E is an event from a sample space
S and that F1, F2, … , Fn are mutually exclusive events such that ⋃n
i=1 Fi = S. Assume that
p(E) ≠ 0 and p(Fi) ≠ 0 for i = 1, 2, … , n. Then
p(Fj ∣ E) = p(E ∣ Fj)p(Fj)∑ni=1 p(E ∣ Fi)p(Fi)

Question 3

Bayesian Spam Filter :

Whats spam?
Vocab?
How to find out which is spam and which is not?

priority?

Answer 3

Spam: flood of unwanted and unsolicited messages.

Spam mails might contain words like “offer” and non spam might contain “mom”.

False Negative: spam filters sometimes fail to identify a spam message as spam; this is called a false negative

False positive: And they sometimes identify
a message that is not spam as spam; this is called a false positive.

While filtering it is important to minimize false positives, because filtering out wanted e-mail is much worse than
letting some spam through

Question 4

The probability that the message is spam, how to find it?

Answer 4

B: Set of messages known to be spam.
G: Set of messages known not to be spam.

We count the number of messages in the set containing
each word to find nB(w) and nG(w), the number of messages containing the word w in the sets
B and G, respectively.

Then, the empirical probability that a spam message contains the word
w is p(w) = nB(w)∕|B|, and the empirical probability that a message that is not spam contains
the word w is q(w) = nG(w)∕|G|.

Let S be the event
that the message is spam. Let E be the event that the message contains the word w. The events S,
that the message is spam, and S, that the message is not spam, partition the set of all messages.
Hence, by Bayes’ theorem, the probability that the message is spam, given that it contains the
word w, is
p(S ∣ E) = p(E ∣ S)p(S) / p(E ∣ S)p(S) + p(E ∣ ~S)p(~S)

Without prior knowledge we assume that
p(S) = p(S) = 1∕2.

p(E ∣ S), the conditional probability that the message contains the
word w given that the message is spam, by p(w). Similarly, p(E ∣ ~S), = q(w).
p(S ∣ E) can be estimated by
r(w) = p(w) / p(w) + q(w).

If r(w) is greater than a threshold that we set, such as 0.9, then we classify the message
as spam.

Question 5

Ways to enhance spam filter:

Answer 5

1. Detecting spam based on the presence of a single word can lead to excessive false positives and false negatives. Consequently, spam filters look at the presence of multiple words.
2. The more words we use to estimate the probability that an incoming mail message is spam,
   the better is our chance that we correctly determine whether it is spam.
3. For the most effective spam filter, we choose words for which the probability that each
   of these words appears in spam is either very high or very low.
4. look at the probabilities that particular pairs of words appear in spam and in messages that are not spam. We
   then treat appearances of these pairs of words as appearance of a single block.
5. we can assess the likelihood that a message is spam by examining the structure of a message to determine where words appear in it.
6. spam filters look at appearances of certain types of strings of characters rather than just words. For example, a message with the valid
   e-mail address of one of your friends is less likely to be spam (if not sent by a worm) than one
   containing an e-mail address that came from a country known to originate a lot of spam.

Question 6

the probability that the message is spam given that it contains both w1 and w2, is?

Answer 6

p(S ∣ E1 ∩ E2) = p(E1 ∣ S)p(E2 ∣ S)
p(E1 ∣ S)p(E2 ∣ S) + p(E1 ∣ S)p(E2 ∣ S)
.
We estimate the probability p(S ∣ E1 ∩ E2) by
r(w1, w2) = p(w1)p(w2)
p(w1)p(w2) + q(w1)q(w2)

refer textbook.

Question 7

m the probability that a message containing

all the words w1, w2, … , wk is spam is ?

Answer 7

p(S ∣
⋂
k
i=1
Ei
) =
∏k
i=1 p(Ei ∣ S)
∏k
i=1 p(Ei ∣ S) + ∏k
i=1 p(Ei ∣ S)
.
We can estimate this probability by
r(w1, w2, … , wk) =
∏k
i=1 p(wi)
∏k
i=1 p(wi) + ∏k
i=1 q(wi)

Refer textbook.