Bayes

Question 1

What is probability ?

Answer 1

The measure of the likelihood that an event will occur

Question 2

what is a random variable?

Answer 2

usually writtenX, is a variable whose possible values are numerical outcomes of a random phenomenon

Question 3

what two types of random variable are there?

Answer 3

1) discrete

2) continuous

Question 4

what is a discrete random variable?

Answer 4

Adiscrete random variableis one which may take on only a countable number of distinct values such as 0,1,2,3,4,…….. Discrete random variables are usually (but not necessarily) counts. If a random variable can take only a finite number of distinct values, then it must be discrete. Examples of discrete random variables include the number of children in a family, the Friday night attendance at a cinema, the number of patients in a doctor’s surgery, the number of defective light bulbs in a box of ten.

Question 5

what is Theprobability distributionof a discrete random variable?

Answer 5

Theprobability distributionof a discrete random variable is a list of probabilities associated with each of its possible values. It is also sometimes called the probability function or the probability mass function.

Question 6

What probabilities is bayes constituted from?

Answer 6

Conditional probability
Joint
Marginal

Question 7

what is conditional probability?

Answer 7

If data are obtained from two (or more) random variables,
the probabilities for one may depend on the value of the other(s)

You cannot reverse conditional probabilities – not interchangeable

Question 8

what is joint probability?

Answer 8

Is theprobabilityof event Y occurring at the same time event X occurs.

We can multiply conditionals together to make joint probabilities (and they are reversible)

Question 9

what is marginal probability?

Answer 9

The probability of an event occurring (could be thought of as the unconditional probability as it is not conditioned on another event).

You can think of it (as the name suggests) as with a table of results…. the marginal probability is one which is totalled (summed) at the margins, so all values from either the column or row (see here http://bit.ly/2pB0gYi) We just add them up.

Question 10

What is independance?

Answer 10

If talking about conditional probabilities, if they are independent, then the occurrence of one does not affect the probability of occurrence of the other.

Question 11

Example of independance ?

Answer 11

Two independent processes such as the outcome of rolling a die, and outcome of flipping a coin.

The probability of the outcome 5 from a die is completely independent of the outcome of a coin turning out as a coin. We can compute both separately, and then get the joint probability. The fact that these processes are independent makes the calculations much easier – good property to have.

Question 12

If you have many discrete data points, what can you create?

Answer 12

a continuous variable

Many many bins in a histogram of probabilities becomes a continuous / smooth bell curve (of belief).

Question 13

what is Probability density function?

Answer 13

Probability density function is the area under a distribution which is used to specify the probability of the random variable falling within a particular range of values, as opposed to taking on any one value.

or density of a continuous random variable, is a function, whose value at any given sample (or point) in the sample space (the set of possible values taken by the random variable) can be interpreted as providing a relative likelihood that the value of the random variable would equal that sample

Question 14

what is The Cumulative distribution function (CDF)?

Answer 14

The Cumulative distribution function (CDF): The summed amount in a distribution up to a certain point. So from 0 (or -) to a given point – we integrate all these values to a certain point. The integral of the probability function over its range. It grows as we integrate.

(summing up every single value in a continuous way – the integral of the probability density function over its range) for continuous-valued random variables, instead of specifying probabilities, the distribution is described by the CDF (p X < x). Or by its derivative ‘probability density function’.

Question 15

So… what is Bayesian probability about?

Answer 15

Using different and multiple probability terms to quantify our degree of confidence we have for something to be the case based on our current knowledge.

Question 16

Bayes enables us to….

Answer 16

Enable statements to be made about the partial knowledge available (based on data), concerning some situation or ‘state of nature’ (unobservable or as of yet unobserved) in a systematic way – using probability as the measure of uncertainty.

Question 17

Gelman said, The guiding principle is that the state of knowledge about anything unknown is described by a ….

Answer 17

probability distribution

Question 18

say Bayes formula …..

Answer 18

Posterior of the model given the data: P (M | D)
Likelihood of the data given the model: P (D | M);
The prior probability of the model (no data): P (M)
The marginal probability (the data): P (D).

P (M|D) = P (D|M) P(M)
––––––––––––
P(D)

Question 19

what is the

The normalization constant ?

Answer 19

The marginal probability (the data): P (D).

Question 20

name one computational technique to get a normalisation constant?

Answer 20

monte carlo method

Question 21

name one non-computational technique to get a prior?

Answer 21

take it from someone else’s study (the partial eta squared - effect size estimation)

Question 22

what mostly determines the posterior distribution?

Answer 22

The compromise between p(D|M) and the p(M)

Question 23

The more data we have from either p(D|M) or p(M)

Answer 23

the more precise our posterior becomes

Question 24

THE LIKELIHOOD DISTRIBUTION IS NOT….

Answer 24

A PROBABILITY DISTRIBUTION

Set of conditional probabilities of a particular outcome.

Question 25

The denominator P(D) incorporates

Answer 25

the probability of all possible outcomes (all data)

Question 26

P(D | M) is not equal to….

Answer 26

P(M | D) …..

Question 27

P(D | M) is not equal to P(M | D) … why?

Answer 27

sometimes known as the base rate fallacy … these two are certainly not the same …. think hypothesitis from presentation

Question 28

if the frequentist approach uses p values for hypothesis testing, what does bayes theorem use?

Answer 28

bayes factor

Question 29

if The frequentists approach (NHST) uses the maximum likelihood estimate with confidence intervals to estimate with uncertainty, what does bayes use?

Answer 29

Bayes uses the posterior distribution with highest density interval intervals to estimate with uncertainty – P(D|M) and P(M|D)

Question 30

what is the difference between The “frequentist” approaches to statistical inferences which are usually based on maximum likelihood estimation (MLE) and the bayes approach?

Answer 30

MLE chooses the parameters that maximize the likelihood of the data.

In MLE, parameters are assumed to be unknown but fixed, and are estimated with some confidence.

In Bayesian statistics, the uncertainty about the unknown parameters is quantified using probability so that the unknown parameters are regarded as random variables.

Question 31

confidence intervals are calculated from the…

Answer 31

likelihood distribution

distribution of the data given the model θ (fixed).

Question 32

but bayes calculates the highest density interval HDI using

Answer 32

posterior distribution, p(θ|y), and sthg else (nuisance parameters). model θ as random variable.

Question 33

a prior distribution should ….

Answer 33

should be a reflection of the pre-existing knowledge about possible parameter values as far as that is quantifiable.

Question 34

what is an non-informative prior ?

Answer 34

E.g. temperature, every temp I range has equal prob, so we don’t know anything.

Question 35

what is a conjugate prior?

Answer 35

SO conjugate priors lead to the same functional form for prior and posterior.

Question 36

to get the bayes factor we need

Answer 36

THE BAYESIAN INFORMATION CRITERION BIc

Question 37

what is precision ?

Answer 37

the inverse variance

Question 38

how does precision affect the posterior?

Answer 38

The posterior mean is expressed as a weighted average of the prior mean and the observed values, ⃗y, with weights proportional to the precisions.

so

– if original prior had small variance (high precision), that would influence this term, conversely if broad unprecise prior, the term would be very small.

– So what contributes mostly to our estimation of posterior mean is the data

– The more precision in the prior, the more it influences the posterior mean. But if crap, we rely on the data.

The more data we get, the more the data will influence the mean of the posterior

Question 39

so the Posterior distribution can be summed to 2 things?

Answer 39

Ð Compromise between data and prior information.

Ð Modulated by the precision in likelihood and prior distributions. the sample size of the data.

Question 40

summery ….

Answer 40

SUMMERY

1) Probability is a quantification of the degree of confidence.
2) Use prior knowledge or belief and new data to make better predictions (updated beliefs).
3) Posterior is differentially influenced by the data and prior distributions depending on these factors (sample size / precision (inverse variance))

Precision (inverse variance) in each distribution. Sample size (data evidence).

4) Close-forms (analytical expressions) make computations easier. But simulation techniques are required for more complex problems (here she references normal distribution, and other complex ones, and also not knowing and letting the posterior guide this process in an iterative fashion).

Question 41

According to Jeffreys (1961), Bayes factors falling between
> 1 and 3 are considered
> 3 and 10 represents…
> greater than 10 is considered….

Answer 41

“anecdotal” evidence

“moderate” evidence

“strong” evidence.

Question 42

According to Masson (2011), probability values p(H0|D) falling between 0.50 and 0.75 are taken as

Answer 42

weak evidence in favor of the null hypothesis.