Reliability and Validity Flashcards
What is the scientific definition of Validity?
The soundness or appropriateness of a test or instrument in measuring what it is designed to measure
What is the scientific definition of Reliability?
The degree to which a test or measure produces the same scores if applied in the same circumstances
What is the scientific definition of Objectivity
The degree to which different observers agree on measurements
What is internal validity?
Whether the experimenter is measuring the effect of the independent variable on the dependent variable
What is external validity?
Whether the results of a test can be generalised to the wider population
What are the types of validity?
- Internal
- External
- Logical
- Statistical
- Construct
What are the two kinds of Logical Validity?
- Face
- Content
What is face validity?
It is clear that the test measures what it is supposed to
What is Content Validity?
Infers that the test measures all aspects contributing to the variable of interest
What is a logically valid test?
One which appears to measure the right variable in its entirety
What are the two types of statistical validity?
- Concurrent Validity
- Predictive Validity
What is Concurrent Validity?
Infers that the test produces similar results to a previously validated test
What is Predictive Validity?
Infers that the test provides a valid reflection of future performance using a similar test
What is a statistically valid test?
One which produces results that agree with other similar tests
what is Construct Validity?
The test is capable of measuring what should exist, whether it exists or not
What are general threats to Validity?
Maturation (change in DV over time irrespective of IV) and History (extraneous variables affecting DV)
What are the threats to internal and external validity?
- Pre-testing
- Experimental Mortality
- Statistical Regression
- Instrumentation
- Selection Bias
What are the threats to external validity?
- Inadequate description
- Biased Sampling
- Hawthorne Effect
- Demand Characteristics
- Operationalisation
Why is pre-testing a threat to validity?
the interactive effects due to pre-testing such as learning and sensitisation may affect the results beyond that of the effect of the IV
How does experimental mortality reduce external validity?
Less data means lower quality results which are less able to be generalised to the wider population
How does experimental mortality reduce internal validity?
Subject dropout means less data, which means reduced statistical power and the quality of data is reduced
How does statistical regression reduce internal validity?
An initial extreme score is likely to be followed by less extreme scores because extreme scores are likely to be outliers/based on chance (e.g. getting a bad test score, then getting a less bad test score next time because the first time was just a bad day)
How does instrumentation threaten internal validity?
Difference in the way two comparable variables are measured due to uncalibrated equipment leads to invalid results
How does biased sampling threaten external validity?
sample group doesnt represent larger population
How do demand characteristics affect the validity of a test?
Participants will act differently if they are aware of the purpose of the test which will affect the results
What can be done if a test is not valid but is reliable?
Use a correction for all results
What are the types of Reliability?
- Relative
- Absolute
- Rater Reliability
- Intra-rater Reliability
- Inter-rater Reliability
What is Relative Reliability?
- When individuals maintain their position or rank in group between tests
What is absolute reliability?
when the scores for an individual are same every test
What is Intra-rater reliability?
The consistency of a given observer or measurement tool on more than one occasion
What is Inter-rater reliability?
The consistency of a given measurement from more than one observer or measurement tool
What are the threats to reliability?
- Fatigue
- Habituation
- Standardisation of Procedures
- Precision of Measurements
How is fatigue a threat to reliability?
- If re-tests are too close together results can vary due to awake/tiredness of participants
How is Habituation a threat to reliability?
- Individuals can get better at the test every re-test
What is a solution to fatigue to improve reliabillity?
Increase time between re-tests
What is a solution to habituation to improve reliability?
- Familiarise the participants fully before the first test
