Reliability Flashcards
Reliability
The term reliability in psychological research refers to the consistency of a research study or a measuring test. For example, if a person weighs themselves during the course of a day they would expect to see a similar reading, scales which measured weight differently each time would be useless.
Internal reliability
Internal reliability refers to the extent to which a measure is consistent within itself. For example, whether the different questions in a questionnaire are all measuring the same behaviour, attitudes etc.
Split-half method
The split-half method assesses the internal reliability of a test, such as questionnaires. It measures the extent to which all parts of the test contribute equally to what is being measured. This is done by comparing the results of one half of a test with the results from the other. A test can be split in half in several ways, e.g. first half and second half, or by odd and even numbers. If the two halves of the test provide similar results this would suggest that the test has internal reliability
External reliability
External reliability refers to the extent to which a measure varies from one use to another.
Two types of reliability
Internal and external
Test-retest
Test-retest involves giving participants the same test on two separate occasions. If the same or similar results are obtained then external reliability is established. For example, if researchers wanted to check if a sleep questionnaire was a reliable measure of sleep quality, the same participants would complete the sleep questionnaire on more than one occasion. Each participant’s scores from the first occasion should be correlated with their results from the later occasion, to be shown on a scattergraph to describe the correlation with scores from the first test being plotted on one axis and the scores from the second test plotted on the other axis. The strength of the correlation should then be assessed using either a Spearman’s rho test or a Pearson’s r test. The degree of reliability is then determined by comparing the correlation with the statistical table to determine the extent of the correlation – there should be a strong positive correlation between the two sets of data. Researchers generally accept 0.8 correlation between the test and retest.
Inter-rater reliability
With inter-rater reliability researchers observe the same behavior independently (to avoided bias) and compare their data. If the data is similar then it is has external reliability. Researchers generally accept 0.8 correlation between the test and re-test.
For example, if a researcher was doing an observation of how people spend their time at the gym, they could use two observers (inter-observer reliability) who discuss and agree beforehand their interpretation of the behavioural categories. Then a statistical comparison of data from both observers will be carried out and compare their separate recordings. The researchers could also check for intra-observer reliability. This is when an observation is video recorded so that it can be watched several times
Intra researcher reliability
This examines the consistency of the individual researchers behaviour during the research. Intra means “within” and intra researcher reliability is achieved if the researcher behaves consistently during research. This can be assessed by measuring the extent to which the researcher gains similar results when observing or measuring the same/similar situations/behaviour on more than one occasion
Inter observer reliability
Make sure the observation is not biased. Use more than one observer/psychologist to observe and record the behaviours separately. The results from both psychologists could be compared and should gain a positive correlation coefficient of +0.8 or more if the observation is reliable
2) If Inter observer reliability is low, then the reliability could be improved by ensuring that the behavioural categories have been operationalised properly and clearly so that each observer understands the categories properly when recording data. The observers might need further training about which behaviours to observe and how they should be measured.
Questions used in the interview
We must make sure that the interview questions are not ambiguous. They should be very clear so that participants understand them and can give the same answers if the questions were asked again in the future. Ambiguous questions might need to be rewritten if necessary or removed
Inter researcher reliability
Make sure the self-report is not biased. If conducting an interview, it is possible to use more than one psychologist to interview participants separately or together and record answers separately. The researchers need to act in similar and consistent ways, and each researcher needs to carry out the procedure and design in exactly the same way so as to make the research consistent. The results from both psychologists could be compared and should gain a positive correlation coefficient of +0.8 or more if the interview is reliable
Standardisation of instructions
If an experiment is conducted twice, then the procedures are repeated twice. Procedures should be exactly the same for each participant that takes part in the experiment, this will help ensure that reliable results are gained. Standardised instructions should be used and key concepts and variables should be operationalised.
