Reliability and Validity Flashcards
What is meant by reliability?
The extent to which results or a measure are consistent
What is meant by internal reliability?
Whether the first half of a tests results are consistent with the the second half’s.
What is meant by external reliability?
The ability to reproduce the same results each time the test is carried out.
How to improve reliability in an experiment?
1)Standardisation-Making all the procedure exactly the same for each time the test is carried out through a script etc
2)Take multiple measures and find an average
3)Pilot studies beforehand check the ambiguity of questions and allows for rewording of such questions and timings etc
How do you assess reliability of an experiment?
Test-retest involving the ppts repeating the same questionnaire or assessment multiple times with gap in between (e.g a week) and finding whether correlation of the results is strongly positive
How do you improve reliability of an observation?
1) Behavioural categories must be operationalised and used consistently across the study
2)Pilot studies allow for timings to be checked, behavioural categories to be made less ambiguous and other aspects of the procedure to be checked
3) Standardised procedure so that every repetition is identical
How do you assess reliability of an observation?
Inter-rater reliability where two researchers observe the same activity and use the same pre-agreed behavioural categories and compare the results. a correlation coefficient of +0.8 indicates reliable results
How do you improve the reliability of self report methods?
1)reduce ambiguity of questions
2)pilot studies to make sure questions are less ambiguous
3)standardise the questions
How do you assess the reliability of self report methods?
1)test-retest repeating tests with time inbetween
2) splithalf
3) inter-interviewer reliability
What is meant by validity?
Validity is the extent to which a measure accurately measures what it claims to and how much it can be generalised outside of the lab setting
What is internal validity?
This is the extent to which a test measures what is claims to measure
What is meant by demand characteristics and how do they impact internal validity?
Demand characteristics indicate the aims and hypothesis of a study to the ppts causing them to change there behaviour meaning the test no longer tests what is claims to
What meant by experimenter bias and how does it impact internal validity?
Experimenter bias refers to the preconceptions and interpretations effect the results displaying the researchers bias
What are ppt variables and how do they impact internal validity?
These refer to characteristics of the ppts that impact the dependent variable and mean the test no longer measures what it claims to
What is external validity?
This refers to the extent to which results can be generalised outside of the lab setting
What is temporal validity?
This refers to how the period that the study effects the internal validity as certain cultural norms have changed
what is population validity?
This is how representative of the target population a sample is and how it affects the generalisability of the results
What is ecological validity?
This is the extent to a which findings can be generalised to situations outside of the research setting
How do we assess internal validity?
1) Face validity - a common sense judgment
2) Concurrent validity - two tests of the same characteristic display the same/similar results
3) Content validity - other experts assess the degree to which a test measures what it claims to
How do we improve the internal validity of a questionnaire?
By revising and consequently removing irrelevant questions
How do we improve the internal validity of an experiment?
Control for:
Investigator effects - standardise etc
Demand characteristics
Confounding variables
Social desirability
How do we improve the validity of an experiment?
1)Single blind method - reduces demand characteristics as ppts do not know what conditions mean
2)double blind method - reduces both demand characteristics and investigator bias by having neither ppts or researcher know what each condition means
Other methods of improving validity?
1) Standardisation of procedure and questions
2) Operationalised variables
3) Control of extraneous variables such as ppt variables
