Reliability Flashcards
If we take multiple measurements, will they be consistent?
This is known as?
Reliability
You measure once and you measure again at a later point in time is called?
test-retest reliability
test-retest reliability can be seen on which plot?
Bland-Altman plot
Internal consistency: different forms of questions
2 versions of the form can pass test re-test reliability
If we have a larger set of questions and administer them (only use half) to each group, then you measure how correlated the are.
This is known as which correlation?
Split-half correlation
look at stuff that’s administered at the same time is known as testing?
Testing Internal consistency
What requires experimenter or observer to evaluate and give ratings to an object or phenomenon?
interrater reliability
-you give instructions on how to rate the thing
-sentences are coded to see how much they relate to the research question
-then see how correlated they are
-the different people coding it should be consistent
Test-retest reliability (consistent across times) use what to measure consistancy?
Test-retest correlation, Bland-Altman plot
Internal consistency (consistent across items) uses what to measure consistancy?
Split-half correlation, Cronbach’s α (alpha)
Interrater reliability (consistent across researchers) uses what to measure consistancy?
Intraclass correlation, Cohen’s κ (kappa)
An important feature of experiments is the causal relationship between variables is known as?
An experiment
If an independent variable is manipulated, how are those changes reflected in a dependent variable?
CAUSAL (A→B) relationships
Experiment only if something is manipulated
(otherwise can still be a study, but may not be an experiment)
How well does a measure relate all facets of a given construct is known as?
content validity
-eg. both positive and negative symptoms of schizophrenia
How well does a measure relate to the theoretical concept you are studying is known as?
construct validity
- we want to asses this
What are the 3 broad components of construct validity?
1- describing theoretical concepts and relationships
2- developing measures of constructs proposed based on theory
3- empirically test the relationships
How well does a measure account for the manipulated change is known as?
E.g., strength of causal relationship, potential influence of confounding factors
internal validity
-can be affected by confounding variables which is a threat/ anything that can create a confound
How well do findings relate to conceptually similar circumstances and the generalisation of findings is known as?
external validity
-High external validity (such as a field experiment) may lead to less control for confounds
Think about would the results generalise to the population (making inferences)
Name examples of factors that can lead to a decrease in internal validity (threats):
Participant selection
Participant motivation (fatigue, time of day)
Maturation (development, aging)
Experimenter training
Equipment decay/use
Lack of random assignment
Name 3 types of Population validity:
Population validity
Experimental sample to defined population
Many studies are based on undergraduate students which does not generalise to everyone
Ecological validity
Experimental setting to real world/other settings
Multiple-treatment interference
Sequence/carry-over effects
Population validity is based on WEIRD, what does this stand for?
Western
Educated
Industrialized
Rich
Democratic
Population validity is also based on STRANGE for animal studies, what does this stand for?
Considering population representability also applies to animal species
Social background
Trappability and self-selection
Rearing history
Acclimation and habituation
Natural changes in responsiveness
Genetic make-up
Experience
How well do psychology findings generalise to the real world is known as which type of external validity?
E.g., older adults remember less words in a memory study than young adults, but how closely does this correspond to memory use in daily life?
ecological validity
