Exam 4 Lecture 2

Question 1

Law: Innocent until proven guilty (or not guilty)

Answer 1

‘Burden of proof’ in law is on sufficient evidence
You have been charged with robbing a store
- There is some evidence that you did it (If there was no evidence, you wouldn’t be charged)
- There is some evidence that you did not do it (If the evidence was strong, you would plea.)
Only you know the truth

What does the jury need to decide?
- Is there ENOUGH evidence that you did it? (proof of guilt)
- If yes, you are found guilty (the evidence in favor of guilt was sufficiently strong)
- If no, you are found not guilty (the evidence of guilt was NOT sufficiently strong, aka it was null)
You may BE innocent but you cannot be never FOUND innocent

Question 2

Null definition

Answer 2

Having no value

Question 3

It’s a battle of evidence. Not guilty is the ______ hypothesis. Guilty is the ________ hypothesis.

Answer 3

Not guilty is the null hypothesis (H0)
Guilty is the alternative hypothesis (H1)

H0 is reigning champion. H1 is the challenger.

Not enough evidence? You cannot reject the null (H0 wins = not guilty)
Lots of evidence? You reject the null (H0 loses = guilty)

Why ‘not guilty’ vs ‘innocent’? The question is GUILTY: Yes or No

Not rejecting the null hypothesis= insufficient evidence that you robbed the store
Rejecting the null hypothesis= sufficient evidence that you robbed the store

Question 4

Yeah, you see it, but do you believe it?

Answer 4

“Burden of proof” in stats is on finding a real difference
You do a fancy experiment and you find some fancy results. How do you know that the results are REAl (that is, that if you did the experiment again, you’d get the same results).

It’s not real is the null hypothesis (H0).
It’s real is the alternative hypothesis (H1).
You assume that something is null until you have evidence it is real.

Question 5

Sampling is everything

Answer 5

Statistics are estimates that are derived from a sample that is thought to represent a population.
You are asking a question about a population: What is the most popular car color in the world?
So, you get a sample.
Can we calculate the likelihood that the sample is representative?
- Sample size
- Sample characteristics

Question 6

OK, so to be ‘significantly different’ or ‘significantly changed’ or ‘significantly related’…

Answer 6

You must be reasonably sure that the results are real.

Question 7

Statistical Significance

Answer 7

Statistical significance quantifies the likelihood that your result is due to chance versus ‘real’.
- Sometimes a result is just because of sampling error-> wrong sample/wrong sample size, population characteristics
- Sometimes a result is because your experiment worked, yay!-> You tried to change something, and it changed! You tried to predict something, and it was predictive!
- Your big picture ‘result’; usually denoted as a p-value.

Question 8

What you want to know…
When you perform an experiment, you are asking a specific question:

Answer 8

Does exercise change healthiness of diet?
- The answer you are looking for is YES, exercise changes healthy eating
- The null is NO, exercise does NOT change healthy eating

Are avid exercisers fitter than sedentary people?
- The answer you are looking for is YES, avid exercisers are fitter than non-exercisers.
- The null is NO, avid exercisers are not fitter than non-exercisers.

Question 9

What you want to know vs what you can actually find out:
Does exercise change healthiness of diet?

Answer 9

The answer you are looking for is YES, exercise changes healthy eating. But the best you can say is from a single experiment: EVIDENCE THAT exercise changes healthy eating was PRESENT in THIS SAMPLE.

The null is NO, exercise does NOT change healthy eating. But the best you can say is:
EVIDENCE THAT exercise changes healthy eating is ABSENT in THIS SAMPLE.

If you repeated the experiment in a different sample, would you find the same thing?

Question 10

Statistics is an applied math

Answer 10

We must quantify:
- strength of evidence
- level of confidence in results

Question 11

Statistics uses probability

Answer 11

Statistics are estimates that are derived from a sample that is thought to represent a population.
Since you don’t know everything, you can never be 100% sure!
So, what level of ‘sure’ are you?
And, how sure do you need to be to be confident?
Statistics uses probability to quantify our confidence.

Question 12

What does the “p” in p-value stand for?

Answer 12

Probability

Question 13

Probability, p-value

Answer 13

A value that reflects how likely something is to occur.
A calculation that quantifies what you mean by ‘probably’
A mathematical tool; it is a number (0 to 1)
0 = never happens
1 = always happens
.9 = happens 90% of the time
It reflects our level of uncertainty.

• It’ll probably rain today.
• I doubt you can bench press 100 lbs.
• Chances are that you can run faster than me.

Question 14

The “p” in p-value= probability

Answer 14

To say something is “significant”
- The fitness groups were significantly different from one another
- The sample was significantly changed by the medication
- The class grades were significantly related to study time
- Nicotine vapes significantly increase your odds of getting lung cancer

The p-value reflects our level of uncertainty. But what, exactly, are we certain/uncertain of?
To be significant, you are sating an observed difference in a sample is not just due to chance.

Probably, it cuts both ways
- I found significant differences, and it’s really there (Exercise improves health!)
- I didn’t find significant difference, and it’s really not there (A Burger King diet does not improve health!)

But what if…
- I found significant differences, and it’s NOT really there (A glass of red wine a day improves health?)
- I didn’t find significant differences, and it really IS there (Taking the stairs, not the elevator, improves health?)

Question 15

Probably, it cuts both ways.

Answer 15

A probability of .9 means 90% confidence- in what?
- I found significant different differences, and it’s really there-> True Positive
- I didn’t find significant differences, and it’s really not there-> True Negative

And what’s the other 10%?
- I found significant differences, and it’s NOT really there- Type 1 error= False-Positive
- I didn’t find significant differences, and it really IS there- Type 2 error= False-Negative

Question 16

False-positive vs. False-negative, which is worse?

Answer 16

You take a statistics test, would you rather:
- See F and then find out you actually got an A?
- See A and then find out you actually got an F?

You take a pregnancy test, would you rather:
- See positive and then find out you are not pregnant?
- See negative and then find out you are pregnant?

A pharmaceutical company is testing a new drug, would you rather:
- They report that the drug works when it actually doesn’t
- They report that the drug doesn’t work when it actually does

Turns out, it’s a trade-off.

Question 17

Significantly different… is a juggling act

Answer 17

I found significant differences, and it’s NOT really there
- You rejected a null hypothesis that is actually true
- Type 1 error= false-positive

I didn’t find significant differences, and it really IS there
- You didn’t reject a null hypothesis that you should have
- Type 2 error = false-negative

Question 18

Balancing act with nulls

Answer 18

You can reject the null. You can FAIL to reject the null. But you can never accept a null hypothesis.

Question 19

The a sets the bar for false-positives

Answer 19

Typically considered the ‘significance level’ or ‘significance cutoff’
- The pre-set, ‘acceptable’ probability of making the wrong decision to reject the null when it is true
- It is conservative! We don’t like these kinds of errors! (You’ve got cancer. JK.)
- Aka saying there is an effect when there is not
- Aka finding a person guilty, when the person is not guilty
- Aka Type 1 error
Almost always set at .05 (5 times out of 100, you’ll be wrong)
- Except in certain fields/under certain conditions
- Even though highly criticized as a cut point!
Your airplane is safe, p<.05

Question 20

Being less than a is significant!

Answer 20

Alpha = a = .05
- So, when we say p<.05, we’re saying…
- The probability of saying it is real when it isn’t is less than 5 in 100
- We are reasonably confident
- We report “significant differences”

a is what determines ‘significance’
a-level is the ‘cutoff’ for the p-value

Question 21

Why can’t you just test something directly?

Answer 21

Because you only have a sample, but you are asking about a population
You are only estimating the likelihood.
A statistic is actually measuring the difference between a) the observed data and b) what is expected from the null hypothesis

The real question is not whether this sample of 50 Rutgers students is smarter than this sample of 50 random Americans, but whether RU students are smarter than average Americans. So there is the OBSERVED (sample results) and there is the EXPECTED (null)/

How different are the samples results from the null?

Question 22

Why can’t you just test something directly?

Answer 22

Are Rutgers students smarter than the average American?
What we know:
Average US IQ= 100 +/- 15
Null hypothesis (EXPECTED):
- Rutgers IQ= 100 +/- 15

What is OBSERVED?
- The greater the observed difference, the more likely the difference is real.
- The probability of a false positive (saying it is a real difference when it isn’t is getting less and less)
- a is getting lower
- You are more confident that what you see is a reproducible finding and it becomes significant.

Question 23

Does vaping affect respiratory function?

Answer 23

H1: Yes, vaping affects it. In fact, yes, vaping negatively affects it.
H0: No, vaping does not affect it.

Question 24

B is the other bar for false-negatives

Answer 24

The pre-set, “acceptable” probability of making the wrong decision to accept a null hypothesis when it is false.
- Aka saying there is no effect when there actually is an effect
- Aka- finding a person not guilty when the person is guilty
- Aka Type 2 error

Almost always set a .20 (20 times out of 100, you’ll be wrong)
- A more lenient cut point
- The world has decided false-negative better than false-positive… but it is based on what you are studying.

Question 25

The other side of B is POWER

Answer 25

Is B= .20, then 1-B= .80 (80 times out of 100, you’ll be right)
- This is the probability of making the correct decision to reject the null
- Aka saying there are differences and there really are differences

Question 26

Even more important than B is 1-B

Answer 26

1-B= Power
Power= the likelihood that your test can identify an effect when there is one (reject null when null is false)= True positive (vs. false-negative)
Lower B= greater power
- But there is a trade-ff, because lower B also means higher a

AGAIN- we are talking about the difference between sample and population, but here, the problem is different.

Question 27

What affects power?

Answer 27

1. a (we don’t want to mess with this!)
2. Central tendencies of groups, conditions (e.g., expected mean differences)
3. Amount of variability in each group, condition
4. Sample size

Sample size (#4) is easiest way to increase power without affecting a
- Another strategy to increase power is to make your groups more different from one another (#2) o homogeneous (#3)

Question 28

Does cannabis use reduce pain after exercise?
Design = test DV before and after a workout
IV= cannabis use, DV = pain

Answer 28

Change sample size
- Sample size = 5 (Do these 5 represent all humans? Harder to differentiate real from chance.)
- Sample size = 5,000 (More likely to be representative. Easier to differentiate a real effect from chance.)

Change means
- Testing condition = 5mg THC oral (Is this enough to cause change?)
- Testing condition = 50mg THC oral (More likely to cause a physiological/perceptual change)

Change variance
- Sample= 18-25yr olds (Are these young adults similar to each other or very different?)
- Sample= 18-25yr olds
- Report frequent cannabis use
- Report typical pain of ‘8-10’
- Report 1-3x/wk exercise

Question 29

When all you know is alpha…

Answer 29

10 men, 0x week resistance training for 1 year
SMM = 105lbs +/- 5 lbs
Same men, 5x week resistance training for 1 year
SMM = 120lbs +/- 8 lbs

We’d run a statistical test (paired t-test), and let’s say:
p= .0442

We’d report:
Resistance training caused a significant increase in muscle mass.

Question 30

When you know alpha and beta…

Answer 30

3 women, 0x week cannabis use for 1 year
rHR = 67 +/- 9 bpm
3 women, 5x week cannabis use for 1 year
rHR = 75 bpm +/- 6 bpm
We’d un a statistical test (t-test), and let’s say:
p= .002, but B= .50 (50 times out of 100, we’d be wrong)

We’d report:
Cannabis use caused a significant increase in rHR (because p refers to alpha), but our beta is too high to be confident.

THIS IS A REAL DANGER OF SMALL SAMPLES!

Question 31

In sum…

Answer 31

When you perform a statistical (inferential) test, you are asking: Are any observed differences real or due to chance?
Accepting the null hypothesis = an observed difference is probably due to chance; no real relationship probably exists.
Rejecting the null hypothesis = an observed difference is probably more than chance; a real relationship may exist.

You cannot PROVE a real relationship exists; you can only say it is more likely than chance to exist.

Rejecting/accepting a null hypothesis is based on probability of making a TYPE 1 error (False POSITIVE). If you find something interesting, testing for ALPHA tells you how likely it would be to find it again in a different sample.