Probability Theory and Bayesian Learning

Question 1

What is the Sum rule?

Answer 1

P(A v B) = P(A) + P(B) - P(A ∧ B)
Prob. of A or B is equal to prob. of A plus prob. of B, minus prob. of A and B

Question 2

What is the chain rule?

Answer 2

P(A ∧ B) = P(A|B) * P(B)
Probability of A and B is equal to the probability of A given B multiplied by prob. of B

Question 3

What is the formula for P(A|B)?

Answer 3

P(A|B) = P(A ∧ B) / P(B)

Question 4

How do we make predictions with multiple pieces of data?

Answer 4

• De couple the multiple pieces of data
• Treat each of them as ‘independent’

Question 5

What is the formula for naive bayes?

Answer 5

P(h | multiple data) = P(data(1) | h) *P(data(2) | h) … * P(data(n)) * P(h)

Question 6

What do naive Bayes classifiers assume?

Answer 6

They assume that the value of any particular feature is independent of any other feature

Conditional Independence

Question 7

What is the issue if one of the features has probability of 0, and what do we do about it?

Answer 7

Then the entire function evaluates to 0.

We apply some addition of a small ( < 0.000001) value to the ones with probability of 0

Question 8

What are some other ways to use naive bayes?

Answer 8

• Part of a KNN classifier
  
  • Use the priors in membership calculations

Question 9

What are some applications of naive bayes?

Answer 9

• Used as a standard to compare other algorithms to
• Spam filtering
• Text classification

Question 10

What happens with values in real-valued domains?

Answer 10

Assume that P(X|Y) lies on a normal gaussian distribution

Question 11

How can the gaussian distribution be described?

Answer 11

It can be described with a probability density function

Question 12

What is the gaussian PDF?

Answer 12

p(x) = (1 / ( sq.rt. ( 2pi * std.dev^2) ) )^e^( -0.5 ( ( x - mean ) / variance) ) ^2

Question 13

When and how is naive Bayes flawed?

Answer 13

It is flawed when two datasets overlap to a large extent.

Because this means that the mean and std. dev. would be similar, giving us a similar gaussian distribution.

Question 14

What does a bayesian belief network represent?

Answer 14

The state of some model.

They describe how states are related by their probabilities

Question 15

How can any system be modelled by a BBN?

Answer 15

All the possible states of the model are the possible worlds that can exist, or how all the possible ways that the parts of the system can be configured

Question 16

What are some examples of BBNs?

Answer 16

• Stock market
• House diagnostics
• Car diagnostics
• Ecosystem simulation

Question 17

What does a BBN contain?

Answer 17

A set of interconnected nodes, where each node represents a variable in the dependency model

A set of edges, where each edge represents relationships between variables

Question 18

What are the advantages of Bayes learning?

Answer 18

• No explicit model - easy to modify ‘on the fly’
• Works well on small amounts of data
• Naive Bayes classifiers tend to be robust
  
  • Generally underfit
• Naive Bayes models are efficient
• Easy to implement

Question 19

What are the disadvantages of Bayes learning?

Answer 19

• Handling of continuous data is bad
  
  • Not everything can be represented by a
    Gaussian
• Conditional independence assumption in
  Naive Bayes can be restrictive
• Sparse data can be problematic