Chapter 20 Bayesian Belief Networks

Question 1

What is conditional dependence?

External

Answer 1

In probability theory, conditional dependence is a relationship between two or more events that are dependent when a third event occurs. For example, if A and B are two events that individually increase the probability of a third event C, and do not directly affect each other, then initially (when it has not been observed whether or not the event C occurs:

P(A|B)= P(A) and P(B|A)=
P(B) [A and B are independent]

But suppose that now C is observed to occur. If event B occurs then the probability of occurrence of the event A will decrease because its positive relation to C is less necessary as an explanation for the occurrence of C (similarly, event A occurring will decrease the probability of occurrence of B). Hence, now the two events A and B are conditionally negatively dependent on each other because the probability of occurrence of each is negatively dependent on whether the other occurs. we have:

P(A|C and B)< P(A|C)

Question 2

What is the challenge of designing and using probabilistic models?

P 199

Answer 2

Probabilistic models can be challenging to design and use. Most often, the problem is the lack of information about the domain required to fully specify the conditional dependence between random variables. If available, calculating the full conditional probability for an event can be impractical.

Question 3

What are two approaches to addressing the challenge of not knowing the conditional dependence between all the variables in probabilistic models?

P 199

Answer 3

1. A common approach to addressing this challenge is to add some simplifying assumptions, such as assuming that all random variables in the model are conditionally independent. This is a drastic assumption, although it proves useful in practice, providing the basis or the Naive Bayes classification algorithm.
2. An alternative approach is to develop a probabilistic model of a problem with some conditional independence assumptions.

Question 4

Bayesian belief networks are one example
of a probabilistic model where some variables are ____.

P 199

Answer 4

conditionally independent

Question 5

A Bayesian belief network is a type of ____ model.

P 199

Answer 5

probabilistic graphical

Question 6

What’s a probabilistic graphical model (PGM)?

P 199

Answer 6

A probabilistic graphical model (PGM), or simply graphical model for short, is a way of representing a probabilistic model with a graph structure.

Question 7

What are nodes and edges in a graphical probabilistic model?

P 199

Answer 7

ˆ Nodes/Vertices: Random variables in a graphical model.
ˆ Edges/Links: Relationships between random variables in a graphical model.

Question 8

There are many different types of graphical models, although the two most commonly described are the ____Model and ____.

P 199

Answer 8

Hidden Markov , the Bayesian Network

Question 9

What’s the main difference between Hidden Markov and Bayesian Network graphs?

P 199

Answer 9

The Hidden Markov Model
(HMM) is a graphical model where the edges of the graph are undirected, meaning the graph contains cycles. Bayesian Networks are more restrictive, where the edges of the graph are directed, meaning they can only be navigated in one direction. This means that cycles are not possible,

Question 10

Graph structure that do not allow cycles are referred to as a ____.

P 199

Answer 10

directed acyclic graph (DAG)

Question 11

A Bayesian belief network describes the ____ for a set of
variables.

P 200

Answer 11

joint probability distribution

Question 12

Bayesian probability is the study of
____ or belief in an outcome, compared to the frequentist approach where probabilities are based purely on ____.

P 200

Answer 12

subjective probabilities, the past occurrence of the event

Question 13

What is conditional independence?

External

Answer 13

In conditional independence two events (which may be dependent or not) become independent given the occurrence of a third event.

Ref

Question 14

Central to the Bayesian network is the notion of ____.

P 200

Answer 14

conditional independence

Question 15

A graphical model (GM) is a way to represent a joint distribution by making
[Conditional Independence] CI assumptions. In particular, the nodes in the graph represent random variables, and the (lack of) edges represent ____. A better name for these models would in fact be ____

P 200

Answer 15

CI assumptions, independence diagrams

Question 16

Bayesian networks provide useful benefits as a probabilistic model. Name 3 of them.

P 200

Answer 16

• Visualization. The model provides a direct way to visualize the structure of the model and motivate the design of new models.
• Relationships. Provides insights into the presence and absence of the relationships between random variables.
• -Structured- Computations. Provides a way to structure complex probability calculations.

Question 17

Designing a Bayesian Network requires defining at least three things, Name them

P 201

Answer 17

Random Variables. What are the random variables in the problem?
Conditional Relationships. What are the conditional relationships between the variables?
Probability Distributions. What are the probability distributions for each variable?

In many cases, the architecture or topology of the graphical model can be specified by an expert, but the probability distributions must be estimated from data from the domain.

Question 18

Both the probability distributions and the graph structure itself can be estimated from data, although it can be a challenging process. As such, it is common to use learning algorithms for this purpose. True/False

P 201

Answer 18

True

Question 19

Once a Bayesian Network has been prepared for a domain, it can be used for reasoning, e.g. making decisions. Reasoning (inference) is performed by introducing evidence (data) that sets variables in known states, and subsequently computing probabilities of interest, conditioned on this evidence. What are some use cases for this model?

P 201

Answer 19

Practical examples of using Bayesian Networks in practice include:
* Medicine (symptoms and diseases)
* Bioinformatics (traits and genes)
* Speech recognition (utterances and time)

Question 20

Bayesian Networks can be developed and used for inference in Python. A popular library for this is called ____ and provides a range of tools for Bayesian modeling, including graphical models like Bayesian Networks.

P 202

Answer 20

PyMC3

Question 21

What’s objective probability and subjective probability?

External

Answer 21

Objective probability is the probability an event will occur based on an analysis in which each measure is based on a recorded observation or a long history of collected data. In contrast, subjective probability allows the observer to gain insight by referencing things they’ve learned and their own experience. Subjective probability represents a personal belief about the likelihood of an event. If someone claims that their favorite sports team has an 80% chance of winning the next game, they are expressing a subjective probability.