12/16 MEETING

Question 1

TRUE OR FALSE: Null hypothesis significance testing (NHST) has been pointed out to be flawed.

Answer 1

TRUE.

People say Bayesian stat is a good alternative for it, but not a lot of people know it.

Question 2

Sir says, we’re gonna focus significantly on test of significance and acceptance. I will introduce to you the T-value in a more correct way (wat)—given that science relies heavily on this

Answer 2

Most of the statistical procedures are anchored on T-values and will always present the T-values. So we might as well interpret it and describe it correctly.

Question 3

Researcher’s hypothesis has a positive look.

Answer 3

TRUE.

In the sense that there is something

Question 4

What are the three ways in where the researcher’s hypothesis can be stated?

Answer 4

Non-directional — M1 =/M2 OR (M1-M2) =/= 0
Right-tailed — M1 > M2 OR (M1-M2)>0
Left-tailed — M1 < M2 OR (M1-M2)<0

*This all assumes that the second group is the anchor group or the point of reference

Question 5

TRUE OR FALSE: There will be that tendency to look for data that ONLY supports your reseacher’s hypothesis.

Answer 5

TRUE.

This is what you call confirmation bias. Because of this, we are more likely to say that our hypothesis is right.

Question 6

So now we’re addressing the issue of confirmation bias, so now we go to Karl Popper and his falsifiability criterion.

Answer 6

Essentially, think of the opposite of your hypothesis, and then if you can look for evidence that could falsify your hypothesis, then you will be given the ticket to assert your researcher’s hypothesis.

BUT if you cannot falsify it, then you cannot assert your hypothesis.

Because of Mr Popper, we have the null hypothesis, that negates the researcher’s hypothesis.

Question 7

Is it null hypothesis vs researcher’s hypothesis?

Answer 7

No. Instead, think of it as:

There was no null hypothesis in the beginning. So you’re free to assert your researcher’s hypothesis. But to avoid the possibility of confirmation bias, come up with a hindrance, or null hypothesis to combat or blockade your research bias. Now, if your data can falsify the null hypothesis, meaning your evidence can negate the null hypothesis, you can open the door. And in effect, you are free to assert your research hypothesis.

IT IS NOT AN EITHER/OR IN THIS CONTEXT. The null hypothesis is a blockage.

Question 8

Is it null hypothesis vs researcher’s hypothesis?

Answer 8

No. Instead, think of it as:

There was no null hypothesis in the beginning. So you’re free to assert your researcher’s hypothesis. But to avoid the possibility of confirmation bias, come up with a hindrance, or null hypothesis to combat or blockade your research bias. Now, if your data can falsify the null hypothesis, meaning your evidence can negate the null hypothesis, you can open the door. And in effect, you are free to assert your research hypothesis.

IT IS NOT AN EITHER/OR IN THIS CONTEXT. The null hypothesis is a blockage.

“Statistically significant results or observed evidence” means or indicates that the observed evidence are sufficient to falsify the null, and we are therefore free to assert our researcher’s hypothesis.

Question 9

Does being able to falsify the null hypothesis make the researcher’s hypothesis true?

Answer 9

No.

It only makes you free to assert it.

Question 10

TRUE OR FALSE: Sir wants you to reject the null hypothesis testing.

Answer 10

NO.

The null hypothesis is so embedded in the idea that it is difficult to change paradigms. But Sir wants us to go away from dichotomous thinking.

Question 11

So now, how do we state the null hypothesis?

Answer 11

You state the opposite of your researcher’s hypothesis.

Question 12

Test of significance by Fisher

Answer 12

A test of evaluation against the null hypothesis

How strong is your evidence against the null hypothesis?

Question 13

Test of significance by Fisher

Answer 13

A test of evaluation against the null hypothesis

How strong is your evidence against the null hypothesis?

Question 14

FISHER’S TEST OF STATISTICAL SIGNIFICANCE

STEPS

Answer 14

1. Researcher’s hypothesis: Filipinos are more intelligent than any typical person in the world.
2. Identify number of groups and number of samples in the study. (In this case, just 1) (associate the anchor with a population) (this is a one-sample case, where you compare one sample against a population)

Comparison of sample mean to a population mean
In this case, it is right-tailed.

We establish an anchor, and we remember the anchor is always the second element in the hypothesis.

We have to know what the population mean is. We need to look for a theory that will lead to a value for the population. Ideally, the population mean should be given to you in the problem set. But if it is not, find a logical argument or a theory so you can assign a value to the population mean.

ex. We are talking about IQ, and the population mean is not given to us. Let’s say we will follow the theory of IQ being mental age over chronological age multiplied by 100 percent. By this, we say the population mean is 100. We could say, therefore, the statistical mean of Filipino IQ is greater than 100. (Remember we only theorize when the population mean isn’t given)
3. Identify which statistical test to use.

Identify number of samples to use

• –If 1 or 2 samples, T-TEST or Z-TEST
• –If 3 or more samples, ANOVA (it’s an extension of T-test, there’s nothing special about it) (sometimes the ANOVA is called an F-test)

Now, in the case of 1 sample, when do we use a T-test and when do we use a Z-test?

• –Use the Z-test when n is large and when population variance and SD is given (if n is small, it will give you faulty estimates, so it’s not recommended for this case)
• –Use t-test when population variance and standard dev’n is not given and you have smaller n (if you run t-test and sample size is large, it will be the same as the z-test)

QUESTION: So what’s the advantage of the Z-test over T-test? Sabi ni Sir be wary of different textbooks that explain hypothesis testing of T-test over Z-test.

Question 15

TRUE OR FALSE: The test of statistical significance will anchor its analysis on Ho.

Answer 15

TRUE.

What happens to your researcher’s hypothesis? Goodbye!

Question 16

CONDITIONAL PROBABILITY

Answer 16

We will observe our sample in the context of this population.

Question 17

We also represent the anchor as a sampling distribution

Answer 17

X-axis is all the possible sample means that we could possibly observe. In theory, this will be normally distributed thus rendering the center of the distribution to be trhe mean of the average IQs, or the mean of the population

IMPORTANT: We will assume that what is true in this sampling distribution is the null hypothesis. In theory, the true state of the world in this anchor group is the null hypothesis. Why? Falsifiability criterion.

What you observed with your sample Filipinos is empirical, and is therefore your evidence.

We will now observe our evidence in the context of our theory (the null hypothesis).

Question 18

Evidence

Answer 18

Our evidence is 110 (mean of sample) and something more extreme than 110 (if we are dealing with right-tailed hypothesis, something higher than 110)

This area (shaded in yellow by Sir) under the null hypothesis becomes your p-value. If the p-value is very small, the overlap between your evidence and null hypothesis, therefore your evidence is strong evidence If p-value is .6, there is large similarity between your evidence and the null hypothesis, therefore it’s weak evidence against the null.

Question 19

Now how do you get the p-value?

Answer 19

Remember X-M

Standard error of the mean (standard deviation of the sample means)

Question 20

T-score is actually how many standard steps our null hypothesis is away from the evidence.

YOU NEED EXCEL TO GET THE T-SCORE.

Answer 20

X - M / SEM

SEM = S / square root of n

[refer to vid for example]

Question 21

in excel, = t.dist.rt (1.43,35)

35 is df = (n-1)

you have your p-value.

Answer 21

Therefore your p-value is p=0.081

How often will you find this result for something more extreme in a world where the hypothesis is true? 8 times out of 100? How similar is your evidence to the null hypothesis? 8%. In the long run, how often will you observe these findings in a world where the null hypothesis is true? 8 out of 100 times.

Do you have a strong evidence against the null hypothesis? If 8 times out of 100…. yes or no?

Question 22

in excel, = t.dist.rt (1.43,35)

35 is df = (n-1)

you have your p-value.

Answer 22

Therefore your p-value is p=0.081

How often will you find this result for something more extreme in a world where the hypothesis is true? 8 times out of 100? How similar is your evidence to the null hypothesis? 8%. In the long run, how often will you observe these findings in a world where the null hypothesis is true? 8 out of 100 times.

Do you have a strong evidence against the null hypothesis? If 8 times out of 100…. yes or no?

Question 23

[CHECK 3.02—THE P-VALUE]

TRUE OR FALSE: If the p-value is low, the findings serve as strong evidence against the Ho.

Answer 23

TRUE.

When the p-value is low.

Question 24

TRUE OR FALSE: If the p-value is high, the findings serve as a weak evidence against the Ho.

Answer 24

TRUE.

NOTE: Anchor it in your study, not in assuming the true state of the world.

Question 25

How low is low? How high is high?

Answer 25

FISHER: “Let’s come up with significance levels! Demaracation levels and lines (significance level) that separate low from high!”

This is not the p-value that you saw, this is just a boundary that separates the low from the high

Question 26

Theoretical cut-off that separates low from high

Answer 26

Significance level is a hypothetical p-value to separate a low from a high p-value

.01, .001, .05

These ain’t alpha levels (alpha levels are used in test of acceptance, measuring probability of level of error you can tolerate)

Question 27

T OR F: Alpha levels and significant levels are the same.

Answer 27

FALSE.

They are different.

Question 28

SENSITIVITY AND SAMPLE SIZE can be said to be used in lieu of the p-value bc it’s problematic (bc p-value is affected by sample size)

Answer 28

When we say sensitivity, it talks about ek ek to defend something. It all boils down to the ability of your statistical procedure to detect differences between the two groups. This is affected by sample size.

If you have large sample size, your statistical procedure could be oversensitive.

Question 29

p-hacking

Answer 29

when you change the sample size to edit the p-value

Question 30

Is statistical significance everything?

Answer 30

What happens to those non-statistically significant papers? File and Drawer problem.

Question 31

WHAT TO AVOID (TRANSGRESSIONS):

P-hacking — increasing sample size
Harking — publishing after the results have been known

Answer 31

These seem to revolve around the belief that statistical significance is everything.