Reliability and Validity Flashcards
What is reliability?
> Scientists need their results to be reliable in order for them to be considered valid
Results are reliable if they are consistent; this means that a test gives the same results each time
It is the best way to test reliability of an experiment is to repeat it.
- If the same results are obtained then the experiment is reliable
Name two methods of testing reliability
Test-Retest method
Split-half method
What is a limitation of the Test-Retest method of testing reliability
> Open questions and unstructured interviews lack reliability because they are difficult to replicate
What is Split-half method of testing reliability
Taking half of a sample of questionnaire answers and comparing them to the other half to check reliability – (or, even numbered vs. odd numbered questions)
A significant positive correlation between the two responses would indicate reliability
What is inner-observer reliability?
> The problem with making observations, especially of behaviour, is that observers can interpret the same event differently
How can we avoid issues with inner-observer reliability?
More than one observer often necessary
- However, rates of reliability between
them (should be at least 85%) can differ
> Ensuring consistency between observers findings
How can inner-observer reliability be assessed?
> Observer reliability can be assessed by measuring the extent to which different observers achieve similar results when observing and scoring the same participants
How can Inner-observer reliability be improved? (2 methods)
- All observers should be trained
thoroughly in the techniques they’re
required to used prior to the data collection. - Operational definitions should be fully
understood
How do we know a test is valid?
> A test is valid if it measures what it has been designed to measure
Give an example of validity, (using manufacturing rulers)
> For example: if a manufacturer had printed the scale on a ruler incorrectly we would think we were measuring validity, but we’re not – therefore the test is not valid
What are the two types of validity?
Internal and External
What is Internal Validity?
Shows there is a casual relationship between the independent variable and the dependent variable in an experiment
What is External Validity?
Shows the results of an experiment can be generalised to other times and places
What are the two types of external validity?
Population and Ecological Validity
What is population validity?
do the findings apply to target population (inc. appropriate sample)
What is ecological validity?
Do findings generalise to other situations:
o E.g. lab studies = low ecological validity (false situations) but high reliability
What are the 3 ways of measuring validity?
Face, concurrent, predictive validity
What is Face validity?
measure of validity
simplest method of testing validity. It involves looking at the test and seeing if it looks as if it’s measuring what it’s supposed to be
What is Concurrent validity?
measure of validity
uses comparisons to existing results. If a past experiment that measures the same variable is known to be valid, a new test can be compared to it
What is predictive validity?
measure of validity
requires that a test can be used to make accurate predictions about future results, before it can be considered valid
Define reliability
This refers to how consistent a study or measuring device is. A measurement is said to be reliable or consistent if the measurement can produce similar results if used again in similar circumstances.
What is context validity?
External validity
This refers to the extent to which research finding can be generalised to settings other than that of the original research. Psychologists often call this ‘ecological validity’. It is sometimes assumed to beam applicable to everyday life, implying that this means ‘naturalistic.’ However, ecological validity more accurately means more applicable applicable across different contexts whatever these may be.
What is temporal validity?
external validity
This is concerned with the ‘shelf life’ of research findings and whether they are able to endure over time
What is Population validity?
external validity
This refers to the extent to which research findings can be generalised to people other than those actually involved in the actual research.
