Reliability & replicability

Question 1

RELIABILITY

Answer 1

extent to which procedure/task/measure is consistent

Question 2

REPLICABILITY

Answer 2

extent to which the study can be repeated in the same/very similar way to test for consistent results

Question 3

TEST-RETEST RELIABILITY

Answer 3

everything must be kept the same - doing the study once and then repeating it the exact same way

Question 4

INTER-RATER RELIABILITY

Answer 4

extent to which two researchers interpreting Ps response will provide the same records from raw data

Question 5

INTER-OBSERVER RELIABILITY

Answer 5

consistency of observational data gathered between two researchers watching the same event

Question 6

BIASES

Answer 6

differences in behaviour responses, emotions, cognition caused by extraneous factors rather than the IV

Question 7

DEMAND CHARACTERISTICS

Answer 7

cues in procedure that provide info about aims - cause Ps to change behaviour - decrease validity
to avoid:
- be vague about aims
- collect DV in multiple ways
- avoid asking Ps qs. directly

Question 8

SOCIAL DESIRABILITY

Answer 8

when Ps respond to tasks in a way that makes them look good
to avoid:
- keep anonymous
ask same qs. in different ways
- obscure study aims
- avoid asking qs. directly

Question 9

ORDER EFFECTS

Answer 9

fatigue effects & practice effects
to avoid:
- counterbalancing
- use independent/matched pairs design

Question 10

RESEARCHER BIAS

Answer 10

unconscious tendency for researcher to act in a way that alters results - often in expected direction of hypothesis
to avoid:
- double blind studies
- external raters to assess DV
- compare ratings with peers

Question 11

OBSERVER BIAS

Answer 11

errors in recording experiment data due to looking for specific behaviours/ misinterpreting other behaviours happening
to avoid:
- check inter rater reliability
- standardising recording of observations (behaviour checklist)
- pilot studies

Question 12

RESPONSE BIAS

Answer 12

tendency of Ps to prefer one choice (e.g always picking 3 in 1-5 scale)
to avoid:
- mixing up question format
- randomise order of tasks
- keep qs. short and clear

Question 13

SAMPLING BIAS

Answer 13

some members of population are statistically more likely to be selected for sample - any finding will be less generalisable to target population
to avoid:
- use more representative sampling methods
- follow up on non responders (volunteer)

Question 14

SELECTION BIAS

Answer 14

when Ps in sample are not equally or randomly allocated to both experimental and control groups - renders results meaningless cuz effect on DV is likely to be caused by participant variable
to avoid:
- randomisation
- random allocation
- use repeated measures/matched pair design