reliability and validity Flashcards
What is another word for reliability?
consistency
When can a test be considered reliable?
if a test is performed with the same person they should produce the same results
What does replicable mean?
easy to repeat
What is internal reliability?
if a test or measure is made up of different parts should consistently measure the same thing
What are key components of internal reliability?
- controls
- standardisation
- same or similar results
- consistency between measures
What is external reliability?
the extent to which results of a procedure can be replicated from one time to another
What is the key component of external reliability?
generalisability
What are the tests for reliability?
1- test re-test
2- split-half
3- inter-rater
What is test re-test reliability?
when you conduct a test and then at a later date conduct again
What is split-half reliability?
split test in a half and participants complete both halves (as 2 wholes)
What is inter-rater reliability?
comparison of ratings between 2 or more researchers to check results are similar and agreement
What factors reduce reliability for experiments?
extraneous variables - make it impossible to replicate
What factors increase reliability for experiments?
control any extraneous variables to make study replicable
What factors decrease reliability for self-reports?
- behaviour of the interviewer (researcher effect)
- ambiguous questions
What are ambiguous questions?
a question that can be interpreted differently
What factors can increase reliability for self-reports?
- train interviewers
- structured interview
- pilot studies to remove ambiguous questions
What factors can decrease the reliability for observations?
- lack of control
- influence of extraneous variables
- observer bias
What factors can increase the reliability for observations?
- pre-determined coding scheme
- train observers
- inter-rater reliability
- video recording (human error)
What is another word for validity?
accuracy
What is the main thing that validity looks for?
causality
What is internal validity?
how behaviour is defined and measured within a study
What are the types of internal validity?
- face validity
- construct validity
- concurrent validity
- criterion validity
What is face validity?
the extent to which a study appears to measure what it intended to measure
What is construct validity?
the extent to which a study actually measures the construct it was designed to measure
What is a construct?
a psychological concept
What are examples of a construct?
depression, extraversion, IQ, memory
What is important about construct validity?
to operationalise variables
What is construct validity most likely to be present for?
scientific measures
What is concurrent validity?
refers to the extent to which the findings from a study agree with previously validated studies measuring the same construct
What is criterion validity?
the extent to which a study can predict the performance or behaviour of a measured construct in the future, based on performance in the present
What is external validity?
the extent to which the conclusions from a study can be generalised to people outside of the study
What is the purpose of external validity?
to assess external factors to see if findings are representative of the population
What are the types of external validity?
- population validity
- ecological validity
What is population validity?
the extent to which results from the sample relate to the population
Why is population validity important?
it is impossible to test everyone so the sample must be representative so results can be generalised
What is ecological validity?
the extent to which results from a study are reflective of what would be found in a natural environment
Why is ecological validity not common?
most studies take place in a controlled environment as it is the best way to control for extraneous variables, establish causality and obtain internal validity
What is mundane realism?
the extent to which a task is representative of one in a natural environment
What factors would decrease validity in an experiment?
- extraneous variables - cannot establish causality
- more chance of demand characteristics
- more likely to have order effects
What factors would increase validity for experiments?
- control extraneous variables (causality)
- natural setting (reduce demand characteristics)
- counterbalancing (reduce order effects)
What factors would decrease validity for self-reports?
- leading questions
- ambiguous questions
- closed questions
- social desirability bias
- demand characteristics
What factors would increase validity for self-reports?
- pilot study
- open questions
- anonymous responses
What factors would decrease validity for an observation?
- demand characteristics
- observer bias
- subjectivity
What factors would increase validity for an observation?
- pre-determined coding scheme
- structured observation
- training - objectivity
