Chapter 13 - Inferential Statistics

Question 1

inferential stats

Answer 1

• Use data collected on a sample to infer what is happening in the population
• Is the effect we found in our sample due to random chance, or due to a true effect in the population?

Question 2

why are we often wrong about inferential stats?

Answer 2

• We focus on random (rare) events
• Fail to use probabilistic information when making judgments
• Focus on specific rather than general information (Eg. base rate fallacy/”person-who” statistics - we know that smoking is bad, but someone will say “well my grandpa smokes a pack a day and he’s still healthy”)
• See patterns in randomness (Ex. Seeing pattern between mass shootings and violent video games -> although the shooters had a bunch of things in common (ie. Both brush teeth), we pay attention to commonality that we feel makes a pattern)

Question 3

random sample

Answer 3

• Taking random sample from population to estimate true effect
• As sample size decreases, estimate is less accurate of what really happens in the population
• Is the sample representative?
• samples are imperfect assessments of overall probabilities
• ex. In any fun-sized packet of Smarties, how many of each colour are there?

Question 4

population level

Answer 4

• ex. in the Smarties factory, how many of each colour are made?

Question 5

probabilistic trend

Answer 5

• What is the true overall effect?
• Not likely to be reflected in every sample and case
• The smaller the sample, the less likely you’ll see a probabilistic trend that reflects the overall effect
• ex. The expected proportion of each colour of Smarties in the population of Smarties at the factory

Question 6

large vs. small samples

Answer 6

• Small samples subject to more error in estimating population value
• Even with random assignment, problem still remains
• Random assignment works best with large sample sizes (chance plays a major role in statistical analysis and research methods)

Question 7

random sampling vs. random assignment

Answer 7

• Random sampling: when everyone in the population has an equal chance of being chosen as a participant (if you don’t do random sampling, bias can be introduced)
• Random assignment: when all participants have equal chance of being chosen to be in experiment or control condition

Question 8

directional null hypothesis

Answer 8

• assumes there is no effect on population and any difference is due to error -> has to account for everything your research hypothesis doesn’t account for
• Scientific notation: H0
• ex. Mean 1 is less than or equal to mean 2
• True effect: California label will not make wine taste better in population
• Says that random chance caused Mean 1 > Mean 2 in our sample

Question 9

directional research hypothesis

Answer 9

• assumes that the means are not equal in the population
• ex. Mean 1 > mean 2
• Scientific notation: H1 or HA
• True effect: California label makes wine taste better
• Random chance is very unlikely explanation that mean 1 > mean 2 in our sample

Question 10

non-directional research hypothesis

Answer 10

• assumes that the means are not equal in the population
• ex. Mean 1 doesn’t equal mean 2
• Scientific notation: H1 or HA
• True effect: label affects taste perception in the population
• Random chance is a very unlikely explanation of the difference between groups in our sample

Question 11

non-directional null hypothesis

Answer 11

• assumes there is no effect on population and any difference is due to error -> has to account for everything your research hypothesis doesn’t account for
• ex. Mean 1 equals mean 2
• Scientific notation: H0
• True effect: label doesn’t affect taste perception in the population
• Random chance caused any difference between groups in our sample

Question 12

when analyzing data, start by assuming that the ____ is true

Answer 12

• null hypothesis (meaning your results are due to random chance)
• as you go, figure out if you can reject it
• Whether something is due to chance is always the most parsimonious alternative explanation for any research finding

Question 13

basics of sampling distribution

Answer 13

• aka: binomial distribution or null hypothesis sampling distribution
• distribution of probability that a statistic would emerge in pop based on many previous samples -> what result would you get from data if null hypothesis is true and results are only due to random chance?
• Helps establish critical values -> threshold for determining significance
• ex. If you claim that you have magical powers to sense and pull out purple smarties from a box, what you need to know first is how many purple smarties other people would randomly pull? -> sampling distribution
• from there, you can use your sampling distribution to establish how many purple smarties one would have to pull out of a box to be magical -> critical value
• ex. if people randomly pick 4 purple smarties on average, you might say your critical value is 8 smarties

Question 14

how to calculate statistical significance

Answer 14

• Calculate a statistic that captures the effect (eg. chi square, t or f value, correlation, etc.) -> aka find obtained t value
• Refer to sampling distribution for comparison (what’s the expected statistic value for this sample size?) -> aka find critical t value
• Decide if our statistic value is rare enough to consider it significant -> if yes, reject null hypothesis and publish, if not, retain null hypothesis

Question 15

degrees of freedom (df)

Answer 15

• used to locate appropriate sampling distribution
• formula: N – 2 (total sample minus number of groups)
• ex. If N = 6, df = 4
• df is dependent on sample size, and more = better

Question 16

t-test

Answer 16

• allows us to determine whether mean difference between 2 groups is significant
• mean/group difference (difference between means for each group) divided by error (within-group variability) = t
• ex. 2/4 = 0.5 = t
• the bigger the t-value, the better
• there is a different sampling distribution of t for each sample size
• If groups are farther apart (larger effect size), the size of the t-statistic will increase

Question 17

how to locate appropriate tcrit?

Answer 17

• alpha level (how likely are we to incorrectly reject the null hypothesis?)
• If |tobt| > |trcrit|, then we reject the null hypothesis
• Alpha is usually set to 0.05

Question 18

Two-tailed vs. one-tailed tests

Answer 18

• 2 tailed: look at alpha level (usually 5%) on either the left or right side of distribution -> if tobt falls outside of that 5%, it’s considered rare enough
• 1 tailed: look at alpha level on both sides of distribution (2.5% + 2.5%) -> if tobt falls between these values, it’s considered rare enough

Question 19

how do we reduce error so we get larger t-values?

Answer 19

• avoiding poorly worded questions (eg. Double-barreled question, double-negative)
• Minimizing effect of uncontrolled variables (eg. Environmental variables, distractors)
• Having a larger sample size
• Between- vs. Within-subjects design

Question 20

t-obtained vs. f-obtained (ANOVA)

Answer 20

• both use interval or ratio DV and nominal IV
• both calculate a signal to noise ratio
• DIFFERENCE: t-test can compare between 2 levels of IV, while f-test can compare between MORE than 2 levels of IV
• One condition where f and t are directly related to each other: when you have two groups in your study (In this case: F = t^2)

Question 21

if tobt > tcrit, ___ null hypothesis

Answer 21

reject null hypothesis (your results aren’t likely due to chance!)

Question 22

if tobt < tcrit, ___ null hypothesis

Answer 22

retain null hypothesis (your results are likely due to chance)

Question 23

what does statistically significant mean?

Answer 23

results aren’t likely to be due to chance (unlikely that difference between 2 groups has a t of 0)

Question 24

errors in judgment

Answer 24

• Type 1 error: H0 is true in population, but you reject H0 (Similar to a false positive)
• Type 2 error: H0 is not true, but you retain it (Similar to a false negative)
• Research community takes Type 1 errors more seriously than Type 2 errors due to publication bias
• However Type 2 errors are also problematic as unreported “file drawer” studies with the type 2 error reduce long-term accuracy of the field by being excluded from meta-analyses

Question 25

error rates

Answer 25

• type 1 error rate: alpha
• type 2 error rate: beta
• correct decision to retain true null hypothesis: 1-alpha
• correct decision to reject untrue null hypothesis: 1-beta (aka: POWER)

Question 26

power

Answer 26

• Probability of making the correct decision to reject a false null hypothesis (ability to detect an effect if one truly exists)
• power = 1-p(type 2 error) where p(type 2 error) means “probability of making a type 2 error”

Question 27

how does effect size influence type 1 and type 2 error?

Answer 27

• As effect size increases, type 1 error doesn’t change -> it only changes if you change whatever the alpha value is that you set
• As effect size increases, type 2 error rate decreases because you have higher power

Question 28

p-hacking

Answer 28

• A set of ethically questionable practices researchers to get significant results
• ex. Data peeking: Run 20 participants, look for significance. No significance? Run 10 more. Significance? Stop and publish. -> Capitalizing on chance
• ex. Tossing out participants’ data that disagree with your hypothesis
• ex. Selective reporting: use multiple measures of the same construct; only 1 measure shows significance; you only report the results of the one that shows significance
• ex. No significance? Find, and report, random interactions as though you predicted them, whether you predicted them or not

Question 29

statistical significance

Answer 29

• are differences between groups due to random error, or do they represent a real effect?
• there will almost always be a difference between your groups, but it may just be caused by random error -> that’s why statistical significance is important
• if something is statistically significant, it’s unlikely that the results are due to random error

Question 30

F-test

Answer 30

• test of statistical significance
• ratio: systematic variance / error variance
• systematic variance: score of all participants across conditions
• error variance: how much individual scores in each group deviate from their respective group means
• the larger the F ratio, the more likely that the difference between groups is statistically significant

Question 31

confidence intervals

Answer 31

• interval of values that will capture the true population value a certain proportion of the time (ex. 95%)
• ex. 95% confidence interval indicates that 95% of confidence intervals we calculate based on samples of that size will capture the population value
• if the confidence levels for 2 means don’t overlap, this is a clue that the difference is statistically significant
• as sample size increases, confidence interval narrows and estimates become more precise

Question 32

conclusion validity

Answer 32

the extent to which conclusions about the relationships among variables reached are correct or reasonable

Question 33

3 factors that power (and type 2 error rates) rely on

Answer 33

1. Sample size: Greater the sample size, greater the power, less error in data to detect effect
2. Magnitude of effect size: The larger the difference is in the population, the easier it is to detect, thus greater power
3. Alpha level: The larger the alpha level, this easier it is to find the data consistent with research hypothesis (reject null hypothesis), thus greater power