Chapter 13 - Inferential Statistics

Question 1

Parameters

Answer 1

Corresponding values in the population

Question 2

Sampling error

Answer 2

The random variability in a statistic from sample to sample

Note that the term error here refers to random variability and does not imply that anyone has made a mistake. No one “commits a sampling error.”)

Question 3

Interpreting Possibilities of statistical relationships in a sample

Answer 3

In fact, any statistical relationship in a sample can be interpreted in two ways:

There is a relationship in the population, and the relationship in the sample reflects this.

There is no relationship in the population, and the relationship in the sample reflects only sampling error.

The purpose of null hypothesis testing is simply to help researchers decide between these two interpretations.

Question 4

NHST

Answer 4

A formal approach to deciding between two interpretations of a statistical relationship in a sample.

Question 5

Null Hypothesis

Answer 5

The idea that there is no relationship in the population and that the relationship in the sample reflects only sampling error (often symbolized H0 and read as “H-zero”).

Question 6

Alternative Hypothesis

Answer 6

An alternative to the null hypothesis (often symbolized as H1), this hypothesis proposes that thereisa relationship in the population and that the relationship in the sample reflects this relationship in the population.

Question 7

Steps of NHST

Answer 7

Although there are many specific null hypothesis testing techniques, they are all based on the same general logic. The steps are as follows:

Assume for the moment that the null hypothesis is true. There is no relationship between the variables in the population.

Determine how likely the sample relationship would be if the null hypothesis were true.

If the sample relationship would be extremely unlikely, thenrejectthenullhypothesisin favor of the alternative hypothesis. If it would not be extremely unlikely, thenretainthenullhypothesis.

Question 8

P value

Answer 8

A crucial step in null hypothesis testing is finding the probability of the sample result or a more extreme result if the null hypothesis were true (Lakens, 2017).[1]This probability is called thepvalue. A lowpvalue means that the sample or more extreme result would be unlikely if the null hypothesis were true and leads to the rejection of the null hypothesis. Apvalue that is not low means that the sample or more extreme result would be likely if the null hypothesis were true and leads to the retention of the null hypothesis.

Question 9

How low does a p value have to be before sample result is considered unlikely enough to reject null hypothesis?

Answer 9

But how low must thepvalue criterion be before the sample result is considered unlikely enough to reject the null hypothesis? In null hypothesis testing, this criterion is calledα(alpha)and is almost always set to .05. If there is a 5% chance or less of a result at least as extreme as the sample result if the null hypothesis were true, then the null hypothesis is rejected. When this happens, the result is said to bestatistically significant. If there is greater than a 5% chance of a result as extreme as the sample result when the null hypothesis is true, then the null hypothesis is retained.

Question 10

Misunderstanding the P value

Answer 10

Thepvalue is one of the most misunderstood quantities in psychological research (Cohen, 1994)[2]. Even professional researchers misinterpret it, and it is not unusual for such misinterpretations to appear in statistics textbooks!

The most common misinterpretation is that thepvalue is the probability that the null hypothesis is true—that the sample result occurred by chance. For example, a misguided researcher might say that because thepvalue is .02, there is only a 2% chance that the result is due to chance and a 98% chance that it reflects a real relationship in the population. But this isincorrect. Thepvalue is really the probability of a result at least as extreme as the sample resultifthe null hypothesisweretrue. So apvalue of .02 means that if the null hypothesis were true, a sample result this extreme would occur only 2% of the time.

You can avoid this misunderstanding by remembering that thepvalue is not the probability that any particularhypothesisis true or false. Instead, it is the probability of obtaining thesample resultif the null hypothesis were true.

Question 11

clarifying what it means to test the null hypothesis

Answer 11

Recall that null hypothesis testing involves answering the question, “If the null hypothesis were true, what is the probability of a sample result as extreme as this one?” In other words, “What is thepvalue?”

Question 12

Interpretive skill

Answer 12

weak relationships based on medium or small samples are never statistically significant and that strong relationships based on medium or larger samples are always statistically significant. If you keep this lesson in mind, you will often know whether a result is statistically significant based on the descriptive statistics alone. It is extremely useful to be able to develop this kind of intuitive judgment. One reason is that it allows you to develop expectations about how your formal null hypothesis tests are going to come out, which in turn allows you to detect problems in your analyses. For example, if your sample relationship is strong and your sample is medium, then you would expect to reject the null hypothesis.

Question 13

A statistically _____ result is not necessarily a ____ one.

Answer 13

significant, strong

Question 14

practical significance

Answer 14

Practicalsignificancerefers to the importance or usefulness of the result in some real-world context. Many sex differences are statistically significant—and may even be interesting for purely scientific reasons—but they are not practically significant. In clinical practice, this same concept is often referred to as “clinical significance.”

Question 15

T test

Answer 15

The most common null hypothesis test for this type of statistical relationship is thet-test.

The t-test is a test that involves looking at the difference between two means.

Question 16

One Sample t-test

Answer 16

Used to compare a sample mean (M) with a hypothetical population mean (μ0) that provides some interesting standard of comparison

Question 17

Test Statistic

Answer 17

A statistic (e.g.,F,t, etc.) that is computed to compare against what is expected in the null hypothesis, and thus helps find the p value.

Question 18

Critical Value

Answer 18

The absolute value that a test statistic (e.g.,F,t, etc.) must exceed to be considered statistically significant.

Question 19

Two-tailed test

Answer 19

Where we reject the null hypothesis if the test statistic for the sample is extreme in either direction (+/-).

Question 20

One-tailed test

Answer 20

Where we reject the null hypothesis only if thetscore for the sample is extreme in one direction that we specify before collecting the data.

Question 21

Dependent-samples t-test

Answer 21

Used to compare two means for the same sample tested at two different times or under two different conditions (sometimes called the paired-samplest-test).

Question 22

Difference score

Answer 22

A method to reduce pairs of scores (e.g., pre- and post-test) to a single score by calculating the difference between them.

Question 23

Independent-samples t-test

Answer 23

Used to compare the means of two separate samples (M1andM2).

Question 24

ANOVA stands for?

Answer 24

Analysis Of VAriance
A statistical test used when there are more than two groups or condition means to be compared.

Question 25

one-way ANOVA

Answer 25

Used for between-subjects designs with a single independent variable

Question 26

meansquaresbetweengroups(MSB)

Answer 26

An estimate of the population variance and is based on the differences among the sample means.

Question 27

mean squares within groups(MSW)

Answer 27

An estimate of the population variance and is based on the differences among the scores within each group.

Question 28

post hoc comparisons

Answer 28

An unplanned (not hypothesized) test of which pairs of group mean scores are different from which others.

Question 29

repeated-measures ANOVA

Answer 29

Compares the means from the same participants tested under different conditions or at different times in which the dependent variable is measured multiple times for each participant.

Question 30

Factorial ANOVA

Answer 30

A statistical method to detect differences in the means between conditions when there are two or more independent variables in a factorial design. It allows the detection of main effects and interaction effects.

Question 31

Rejecting the null hypothesis when it is true is called a ____ __ _____

Answer 31

Type I Error - This error means that we have concluded that there i a relationship in the population when in fact there is not. This is a false positive.

Question 32

Retaining the null hypothesis when it is false is called a ____ ___ ___

Answer 32

Type II Error - This error means that we have concluded that there is no relationship i the population when in fact there is a relationship. This is a false negative.

Question 33

setting a to .05 is ideal why?

Answer 33

It keeps the rates of both type I and II errors at acceptable levels; accoring to some researchers.

Question 34

What is the file-drawer problem?

Answer 34

The problem of research results not being published that fail to find a statistically significant result. As a consequence, the published literature fails to contain a full representation of the positive and negative findings about a research question.

Question 35

What is the result of the file-drawer problem?

Answer 35

One effect of this tendency is that the published literature probably contains a higher proportion of Type I errors than we might expect on the basis of statistical considerations alone. Even when there is a relationship between two variables in the population, the published research literature is likely to overstate the strength of that relationship.

Question 36

What is p-hacking?

Answer 36

When researchers make various decisions in the research process to increase their chance of a statistically significant result (and type I error) by arbitrarily removing outliers, selectively choosing to report dependent variables, only presenting significant results, etc. until their results yield a desirablepvalue.

Question 37

Statistical Power

Answer 37

In research design, it means the probability of rejecting the null hypothesis given the sample size and expected relationship strength.

Question 38

How to increase statistical power

Answer 38

Given that statistical power depends primarily on relationship strength and sample size, there are essentially two steps you can take to increase statistical power: increase the strength of the relationship or increase the sample size. Increasing the strength of the relationship can sometimes be accomplished by using a stronger manipulation or by more carefully controlling extraneous variables to reduce the amount of noise in the data (e.g., by using a within-subjects design rather than a between-subjects design). The usual strategy, however, is to increase the sample size. For any expected relationship strength, there will always be some sample large enough to achieve adequate power.

Question 39

Confidence Intervals

Answer 39

A range of values that is computed in such a way that some percentage of the time (usually 95%) the population parameter will lie within that range.

Question 40

Bayesian Statistics

Answer 40

An approach in which the researcher specifies the probability that the null hypothesis and any important alternative hypotheses are true before conducting the study, conducts the study, and then updates the probabilities based on the data.

Question 41

Replicability crisis

Answer 41

A phrase that refers to the inability of researchers to replicate earlier research findings.

Question 42

5 questionable research practices:

Answer 42

1. Deleting outliers to influence statistical results.
2. Cherry picking results.
3. HARKing - Hypothesizing After Results are Known.
4. p-hacking - determining whether a statistic is significant before recruiting more participants.
5. Outright fabrication of data (fraud).

Question 43

Important things to do now

Answer 43

Designing and conducting studies that have sufficient statistical power, in order to increase the reliability of findings.

Publishing both null and significant findings (thereby counteracting the publication bias and reducing the file drawer problem).

Describing one’s research designs in sufficient detail to enable other researchers to replicate your study using an identical or at least very similar procedure.

Conducting high-quality replications and publishing these results (Brandt et al., 2014)[10].

Question 44

Open science practices

Answer 44

A practice in which researchers openly share their research materials with other researchers in hopes of Increasing the transparency and openness of the scientific enterprise.

Question 45

Criticisms and Defenses of Null Hypothesis Testing

Answer 45

Null hypothesis testing has been criticized on the grounds that researchers misunderstand it, that it is illogical, and that it is uninformative. Others argue that it serves an important purpose—especially when used with effect size measures, confidence intervals, and other techniques. It remains the dominant approach to inferential statistics in psychology.