Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

PSYC4050 > Wk 6 - Analytic Strategy > Flashcards

Wk 6 - Analytic Strategy Flashcards

1
Q

What are 3 steps that motivate hypothesis testing?

A

Desire to experimentally investigate some phenomenon
Measure and/or manipulate a behaviour
Analyse data to determine any effect

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What are 3 possible sources of variation in data?

and we can… (x1)

A

All is systematic
All is random
Both systematic and random factors involved

Dismiss the first in psych…

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

What is the current standard for evaluating data in psych? (x1)
Which… (x2)

A

NHST

Distinguishes random chance alone vs chance+systematic effects
Establishes statistical hypothesis of no systematic effects

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Give 3 egs of assumptions made by the null hypothesis

But it should be remembered that it… (x1)

A

No difference in mean scores between groups
No differences in variances across groups
No systematic relationship between variables

Allow for diffs in data - just assumes not meaningful

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

What does NHST evaluate in practice? (x2)

A

Probability of observing the data (or more extreme data), if the null hypothesis were true
This probability, p(d|H0), is expressed by p values

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

If we adopt a conventional p < .05 criterion, significant results imply…(x3)

A

Ulikely to obtain the data if H0 were true
We reject H0 as viable account - effects not just chance
And endorse account of theoretically interesting/systematic factors

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

What are the limitations of the inductive leap from data to hypotheses in NHST? (x2)

A

We evaluate p of obtaining DATA

Then use this to make inferences about HYPOTHESES

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

What mathematical logic demonstrates the limitations of NHST? (x4)

A

NHST dosn’t evaluate probability of the null being true
Assumes that it is
The probability of obtaining the data if the null is true,
Is not equal to the probability of obtaining the null given the data

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

If an AIDS test has a hit rate of 99.99%, and a correct rejection rate of 99.99%,
What is the probability of a positive result if you actually have the disease?

A

99.9%

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

If an AIDS test has a hit rate of 99.99%, and a correct rejection rate of 99.99%,
What is the probability of having the disease given a positive test result?

A

49.98%

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

What explains the difference between the probability of a positive result if the hypothesis is true,
And the probability of the hypothesis being true given a positive result? (x4)

A

The overall probability of the hypothesis being true, ie:
1/10 000 men have AIDS
Which gives 1 true, positive test
And 1 false positive (given 9 998 correct rejections
So out of 2 positive results, only 1 actually has AIDS

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Why is it critical to remember p(d|H0) ≠ p(H0|d) ? (x3)

A

p tells us nothing about experimental hypotheses,
Or the probability of replicability
Only about the data we currently have

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

Why is NHST still relevant, despite legitimate criticisms? (x2, x3)

A

Not generally just testing for AN effect,
*But contrasting theoretical predictions of where/when they’ll occur

We seldom hypothesise a null

* Predict an event WILL occur, rather than not
* Null being true is 1 in a zillion weays of not finding an effect
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

What are 2 theoretical goals of research?

A

To contrast different theories of psychological phenomena

Find out what is true about the world and explain it

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

What are 2 communicative goals of research?

A

Inform others about how well different theories account for psych phenomena
Convince others about what is true in the world and how it ought to be explained

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

How do statistics facilitate the theoretical goals of research? (x2)

A

Quantitatively assess mis/match between theory and data

Allow distinguishing random from systematic variation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

How do statistics facilitate the communication goals of research? (x2)

A

Strengthen evidence supporting theoretical claims

*Objective reference for interpreting implications of data

18
Q

How do communication goals influence research design? (x2)

A

Readers/publishers have expectations re design/analysis

Need to think on how research question influences study design and how variables measured, analysed, interpreted

19
Q

When should you be deciding on data analysis plan? (x3)

A

Well before collecting data:

* Can constrain method, and therefore viability question
* Protect against collecting unanalysable data
20
Q

What are the practical benefits of planning analysis before collecting data? (x2)

A

Organise thinking about your data set a priori

And about project/how to answer research questions

21
Q

What are the 4 steps in developing data analysis plan/deciding on key statistical tests to address research questions?

A 
Review:
    *Research Questions
    *Study Design and Measurement
    *Hypotheses
Choose appropriate statistical tests
22
Q

What are the considerations when reviewing research questions for data analysis planning? (x1, x2)

A

What constructs are you interested in and how are they related?
If there are multiple IVs, are some more important than others?
What is your contribution to existing research?

23
Q

What are the considerations when reviewing study design and measurement for data analysis planning? (x3, x3, x3)

A

Were your IVs just measured:

* Single/multiple time points?
* Continuous/categorical scales?

Or manipulated?

* Number of levels of IV?
* Between/within Ps?

Relevant DVs:

* 1 or more time points?
* Continuous/categorical?
24
Q

What are the considerations when reviewing hypotheses for data analysis planning? (x1, x1, x2, x2)

A 
What are operationalised IV and DVs?
Theoretically driven main effects of IVs on 1+ DVs?
Expected diffs across conditions? 
    *Interactions?
Follow-up comparisons req'd?
    *Planned vs post hoc
25
Q

What are the considerations whenchoosing appropriate statistical tests during data analysis planning? (x2, x1, x1, x1)

A

Diff methods apply to diff designs
*SPSS will allow anything!
Between-/within-Ps, cross-sectional or longitudinal?
How many IVs/DVs? Continuous/categorical? Covariates?
Mediation/moderation hypotheses?

26
Q

What are the diffs between practical and theoretical implications of experimental findings? (x2)
So it’s good to… (x1)

A

Reporting p values is expected,
But can be poor indicator of importance
Report effect size too (eg r-squared)

27
Q

Why does effect size matter?

A

Any effect is significant with big enough sample

Need sufficient power to find effects of expected size

28
Q

What 3 factors influence statistical power?

A 
Significance level (alpha)
Sample size
Error variance
29
Q

What can we do to ensure adequate sample size/power? (x3)

A

Bigger enables detection of smaller effects
Can use Power Analysis to calculate
Decide BEFORE commencing study

30
Q

What design/analytic techniques can be used to maximise power? (x2)

A

Use blocking designs

Include control variables (eg ANCOVA)

31
Q

Simpler is better when communicating research results, so you should… (x3, x3)

A

Limit number of IVs/DVs

  • Easier for you/others to make sense of data
    * And to write compelling narrative

Use simplest statistical analyses you can

* Don't over-complicate results
* Doing so creates suspicion
32
Q

Wneh seeking simplicity in communicating results, its important to remember that…(x4)

A

You can’t conceal inconvenient info!

* Note any missing data issues
* Report any preliminary analyses, eg EFA
* Be clear on details of analyses
33
Q

What is HARKing? (x1)

Which entails? (x2)

A

Hypothesising after results are known
Revising hypotheses post-analysis
Presenting post-hoc hyps as if a priori

34
Q

What is the general hypothetico-deductive scientific method? (x4)

A

Develop theory-driven, testable hypotheses
Design experiment
Report results
Draw conclusions based on evidence

35
Q

Why is post hoc HARKing not science? (x2)

A

By definition, unfalsifiable

*Hypotheses tailored to fit data

36
Q

Explain why are HARKed hypotheses bad? (x3)

A

Often begins with no EXPECTED effects in data, but something UNexpected observed
No theoretical motivation to explain this - likely a false positive
Buckets of replication failures needed to discredit initial ‘finding’

37
Q

If HARKing is so bad, why does it happen? (x2, x3)

A

Researchers want to tell compelling story
*Often post-hoc theorising plays out subconsciously

Meta-scientific reasons

* Publish or perish
* Journals don't publish null findings
38
Q

How to recognise HARKING: It probably is if… (x2)

A

You acknowledge lack of a priori hypotheses, but draw post-hoc conclusions from data
Fail to report unsupported a priori hypotheses

39
Q

How to recognise HARKing: It’s not if… (x1)

A

Competing a priori hypotheses presented, then conclusions drawn from data

40
Q

What are 3 ways to spot harking? (x2, x2, x1)

A

Mismatched prior theory and a priori hypotheses
*Incoherent, overly complex, implausible

Mismatched research questions and methodology
*Odd study setup

Unmotivated/expected testing new condition/qualifier that conveniently fits the data

41
Q

What are 4 ways to stop yourself from HARKing?

A

Pre-register studies
Report what you do
Be honest about hypotheses
Use multiple studies to examine unexpected results

PSYC4050 (10 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions