Statistics Theory L8 = Drawing Statistical Conclusions Flashcards
Drawing statistical conclusions?
= involves the consideration of the scope of inference.
Scope of inference?
= includes 2 aspects.
Aspects of the Scope of inference? (2)
- Making conclusions about cause and effect.
- Making inferences beyond the observed sample (to the statistical population).
Two case studies to help us understand the aspects of the scope of inference?
- Motivation & Creativity (Randomized experiment).
- Sex discrimination in employment (Observational study).
Description of Case study 1: Randomized experiment? (6)
- 2 groups of students.
- Randomly assigned to “intrinsic” & “extrinsic” groups.
- Intrinsic = prompted by a questionnaire with questions about intrinsic reasons for writing.
- Extrinsic = same but with extrinsic reasons for writing.
- Each student wrote a Haiku style poem.
- Writing judges judged the work (/40 points).
Results of Case study 1? (6)
Treatment Count AvScore SDScore
Extrinsic 23 15.7 5.25
Intrinsic 24 19.9 4.44
Statistical conclusion of Case Study 1?
There is strong evidence that the intrinsic treatment scored higher than the extrinsic treatment (differences: 4.1; 95% CI: 1.3, 7.0).
Can you infer cause and effect from Case study 1? Why or why not?
Yes, you can infer cause and effect because there was randomisation, as students were randomly assigned to the different groups.
Can you make inferences to the statistical population using the results of Case study 1? Why or why not?
No, you cannot make inferences about the statistical population, as there was no random sampling. Therefore, the findings are only applicable to the students in the study.
Randomised experiment VS Random sampling?
- Randomised experiment
= random allocation of experimental units to treatments. - Random sampling
= random selection of experimental units from the statistical population.
Description of Case study 2? (4)
- Data from 1960s - 1970s.
- 32 males & 61 females.
- Skilled, entry-level employees at a bank.
- Compared starting salary between male & female.
Results of Case study 2? (6)
Sex Count AvSal SDSal
Female 61 5139 540
Male 32 5957 691
Statistical conclusion of Case study 2?
Strong evidence of a difference; M > F: USD 818 (95% CI: 560, 1076).
Can we infer cause and effect from Case study 2? Why or why not?
No, we cannot infer cause and effect, as there was no random assignment of salaries (it wasn’t a sex-blind experiment/randomised experiment).
Can we make inferences to the statistical population using the results of Case study 2?
No, we cannot make inferences to the statistical population, as there was no random sampling. Therefore, the results apply to the specific bank in which the study was conducted.
Term used to refer to making conclusions about the cause & effect?
Causal inference.
Term used to refer to making inferences beyond the observed sample & to the statistical population?
Inference to the population.
Causal inference categories? (2)
- Randomised experiment.
- Observational study.
Randomised experiment?
= we use a chance mechanism to assign groups to treatments.
Observational study?
= group status is beyond the control of the investigators (can’t be assigned).
Causal inference in a Randomised experiment attributes? (3)
- Allows us to make conclusions about causation.
- Subjects with different relevant features are mixed up between the two groups.
- Statistical tools incorporate this uncertainty & help us assess whether observed differences could have arisen by chance.
Causal inference in Observational study attributes? (2)
- Conclusions are limited to correlations & associations.
- Can’t conclude causation because of the influence of confounding variables (variables that are related to both the group membership & the outcome).
Inference to the population attributes? (2)
- Only possible if a study uses random sampling.
- Common approach = Simple random sampling.
Values of Observational studies? (3)
- Establishing causation might not be the goal.
- Establish causation in other ways.
- Might be an early step leading to future experiments.
Statistical inference in relation to the chance mechanism? (2)
- Always some level of uncertainty in statistical conclusions.
- Investigators job is to assess & communicate that uncertainty when presenting our findings.
Inference?
= a conclusion that patterns in the data are present in some broader context.
Statistical inference?
= inference that is justified by a probability model linking data to the broader context.
Summary? (2)
Without random sampling:
- Can only conclude about the units in the sample (no inference to statistical population).
- For randomised experiments, can still conclude case-effect, but no broader inference.
Random assignment + Random selection of units?
- Causal inferences can be drawn.
- Inferences to the populations can be drawn.
Random assignment + Non random selection of units?
Only Causal inferences can be drawn.
Non random assignment + Random selection of units?
Only Inferences to the populations can be drawn.
Non random assignment + Non random selection of units.
- No causal inferences can be drawn.
- No inferences to the populations can be drawn.
Therefore, random selection (random sampling) = …?
Inferences to the statistical population can be drawn.
Therefore, random assignment (randomized experiment) = …?
Causal inferences (cause & effect) can be drawn.
Statistic?
= number calculated from data.
Test statistic?
= a measure of the plausibility of the alternative hypothesis relative to the null hypothesis.
What do we use to measure uncertainty in Randomized experiments?
Randomization tests.
What do we use to measure uncertainty in Observational studies?
Permutation tests.
Randomization tests in randomized experiments?
=
Permutation tests in observational studies?
=
p-value?
= proportion of times that the random difference is larger than the observed difference.
Things to note concerning measuring uncertainty in both these studies? (2)
- The larger the difference, the lower the probability that the effect observed could have arisen by chance. Therefore, we conclude that there is an effect of the treatment.
- To judge whether difference occurred by chance, observe the tails of the distribution.
NB thing to note when talking about scope of inference? (2)
It’s important to talk about:
- Causal inferences (whether cause & effect can be inferred).
- Inferences to the statistical population being drawn.
Solution to the Randomization & Permutation tests to approximate the sampling distribution?
Tools like the t-test!!
