Wk 6 - Analytic Strategy Flashcards
What are 3 steps that motivate hypothesis testing?
Desire to experimentally investigate some phenomenon
Measure and/or manipulate a behaviour
Analyse data to determine any effect
What are 3 possible sources of variation in data?
and we can… (x1)
All is systematic
All is random
Both systematic and random factors involved
Dismiss the first in psych…
What is the current standard for evaluating data in psych? (x1)
Which… (x2)
NHST
Distinguishes random chance alone vs chance+systematic effects
Establishes statistical hypothesis of no systematic effects
Give 3 egs of assumptions made by the null hypothesis
But it should be remembered that it… (x1)
No difference in mean scores between groups
No differences in variances across groups
No systematic relationship between variables
Allow for diffs in data - just assumes not meaningful
What does NHST evaluate in practice? (x2)
Probability of observing the data (or more extreme data), if the null hypothesis were true
This probability, p(d|H0), is expressed by p values
If we adopt a conventional p < .05 criterion, significant results imply…(x3)
Ulikely to obtain the data if H0 were true
We reject H0 as viable account - effects not just chance
And endorse account of theoretically interesting/systematic factors
What are the limitations of the inductive leap from data to hypotheses in NHST? (x2)
We evaluate p of obtaining DATA
Then use this to make inferences about HYPOTHESES
What mathematical logic demonstrates the limitations of NHST? (x4)
NHST dosn’t evaluate probability of the null being true
Assumes that it is
The probability of obtaining the data if the null is true,
Is not equal to the probability of obtaining the null given the data
If an AIDS test has a hit rate of 99.99%, and a correct rejection rate of 99.99%,
What is the probability of a positive result if you actually have the disease?
99.9%
If an AIDS test has a hit rate of 99.99%, and a correct rejection rate of 99.99%,
What is the probability of having the disease given a positive test result?
49.98%
What explains the difference between the probability of a positive result if the hypothesis is true,
And the probability of the hypothesis being true given a positive result? (x4)
The overall probability of the hypothesis being true, ie:
1/10 000 men have AIDS
Which gives 1 true, positive test
And 1 false positive (given 9 998 correct rejections
So out of 2 positive results, only 1 actually has AIDS
Why is it critical to remember p(d|H0) ≠ p(H0|d) ? (x3)
p tells us nothing about experimental hypotheses,
Or the probability of replicability
Only about the data we currently have
Why is NHST still relevant, despite legitimate criticisms? (x2, x3)
Not generally just testing for AN effect,
*But contrasting theoretical predictions of where/when they’ll occur
We seldom hypothesise a null
* Predict an event WILL occur, rather than not * Null being true is 1 in a zillion weays of not finding an effect
What are 2 theoretical goals of research?
To contrast different theories of psychological phenomena
Find out what is true about the world and explain it
What are 2 communicative goals of research?
Inform others about how well different theories account for psych phenomena
Convince others about what is true in the world and how it ought to be explained
How do statistics facilitate the theoretical goals of research? (x2)
Quantitatively assess mis/match between theory and data
Allow distinguishing random from systematic variation
How do statistics facilitate the communication goals of research? (x2)
Strengthen evidence supporting theoretical claims
*Objective reference for interpreting implications of data
How do communication goals influence research design? (x2)
Readers/publishers have expectations re design/analysis
Need to think on how research question influences study design and how variables measured, analysed, interpreted
When should you be deciding on data analysis plan? (x3)
Well before collecting data:
* Can constrain method, and therefore viability question * Protect against collecting unanalysable data
What are the practical benefits of planning analysis before collecting data? (x2)
Organise thinking about your data set a priori
And about project/how to answer research questions
What are the 4 steps in developing data analysis plan/deciding on key statistical tests to address research questions?
Review:
*Research Questions
*Study Design and Measurement
*Hypotheses
Choose appropriate statistical testsWhat are the considerations when reviewing research questions for data analysis planning? (x1, x2)
What constructs are you interested in and how are they related?
If there are multiple IVs, are some more important than others?
What is your contribution to existing research?
What are the considerations when reviewing study design and measurement for data analysis planning? (x3, x3, x3)
Were your IVs just measured:
* Single/multiple time points? * Continuous/categorical scales?
Or manipulated?
* Number of levels of IV? * Between/within Ps?
Relevant DVs:
* 1 or more time points? * Continuous/categorical?
What are the considerations when reviewing hypotheses for data analysis planning? (x1, x1, x2, x2)
What are operationalised IV and DVs?
Theoretically driven main effects of IVs on 1+ DVs?
Expected diffs across conditions?
*Interactions?
Follow-up comparisons req'd?
*Planned vs post hoc
