L19 - Assessing and Improving Reliability Flashcards
what is RELIABILITY?
RELIABILITY is a MEASURE of CONSISTENCY
E.g if researchers are using a QUESTIONNAIRE to measure LEVELS OF DEPRESSION, they want to ensure that the MEASURE IS CONSISTENT BETWEEN PPs and OVER TIME
what is TEST-RETEST RELIABILITY? - PT 1
One way of testing reliability is the TEST-RETEST METHOD
Quite simply, the SAME PERSON or GROUP OF PEOPLE are asked to UNDERTAKE THE RESEARCH MEASURE, e.g a QUESTIONNAIRE, on DIFFERENT OCCASIONS
When using the TEST RETEST METHOD, its important to remember that the SAME GROUP of pps are being STUDIED TWICE, so researchers need to be AWARE OF ANY DEMAND CHARACTERISTICS
E.g if the questionnaire is given TWICE IN ONE DAY, there is a STRONG CHANCE that pps will be able to RECALL EARLIER ANSWERS and if this is the case, the psychologists are then testing their MEMORY rather than the RELIABILITY OF THEIR MEASURE
what is TEST-RETEST RELIABILITY? - PT 2
Its also important to make sure there is NOT TOO MUCH TIME BETWEEN EACH TEST
If psychologists are testing a MEASURE OF DEPRESSION, and question the pps A YEAR APART, its possible that they MAY HAVE RECOVERED in that time, and so they give DIFFERENT ANSWERS but NOT BECAUSE THE QUESTIONNAIRE IS NOT RELIABLE
After the measure has been COMPLETED ON TWO SEPARATE OCCASIONS, the TWO SCORES ARE CORRELATED. If the CORRELATION is shown to be SIGNIFICANT, then the measure is DEEMED to have GOOD RELIABILITY - a PERFECT CORRELATION is 1, therefore the CLOSER the CORRELATION COEFFICIENT IS TO 1, the GREATER THE RELIABILITY
what is INTER OBSERVER RELIABILITY? - PT 1
INTER OBSERVER RELIABILITY refers to the extent to which TWO OR MORE OBSERVERS are OBSERVING and RECORDING BEHAVIOUR in a CONSISTENT WAY
This is a USEFUL WAY of ENSURING RELIABILITY in situations where there is a RISK OF SUBJECTIVITY.
In PSYCHOLOGY STUDIES where BEHAVIOURAL CATEGORIES are being applied, INTER OBSERVER RELIABILITY is important to make sure that the categories are being USED IN THE CORRECT MANNER.
what is INTER OBSERVER RELIABILITY? - PT 2
The INDEPENDENT OBSERVERS would observe the SAME SITUATION or EVENT SEPARATELY, and then their OBSERVATIONS would be CORRELATED to see whether they are SUITABLY SIMILAR
If the observers achieve a KAPPA SCORE of at least +0.8 then we can say that the FINDINGS OF THE STUDY HAVE INTER OBSERVER RELIABILITY
how to IMPROVE THE RELIABILITY of QUESTIONNAIRES?
For questionnaires, its possible to identify WHICH QUESTIONS are having the BIGGEST IMPACT ON RELIABILITY, and ADJUST THEM AS NECESSARY
If its deemed that they are IMPORTANT QUESTIONS that MUST REMAIN, then they can be REWRITTEN in a MANNER THAT REDUCES THE POTENTIAL for them to be INCORRECTLY INTERPRETED
If they aren’t that important to the PURPOSE OF THE RESEARCH then they can be taken out altogether
how to IMPROVE the RELIABILITY of INTERVIEWS?
if RELIABILITY needs IMPROVING IN AN INTERVIEW there are SEVERAL FACTORS that can be ADJUSTED
Firstly, ensuring that the SAME INTERVIEWER is CONDUCTING ALL INTERVIEWS will HELP REDUCE RESEARCHER BIAS; there is the POTENTIAL FOR VARIATION in the WAY THAT QUESTIONS ARE ASKED which can then LEAD TO DIFFERENT RESPONSES
In addition, SOME RESEARCHERS MAY ASK QUESTIONS THAT ARE LEADING or OPEN TO INTERPRETATION which is likely to REDUCE RELIABILITY.
If the SAME INTERVIEWER CANNOT BE USED THROUGHOUT, then TRAINING SHOULD BE PROVIDED in order to LIMIT THE POTENTIAL BIAS; further to this, CHANGING THE INTERVIEW from UNSTRUCTURED TO STRUCTURED will LIMIT RESEARCHER BIAS
how to IMPROVE THE RELIABILITY of EXPERIMENTS?
In EXPERIMENTS, the LEVEL OF CONTROL that the researcher has over VARIABLES is one way that RELIABILITY CAN BE INFLUENCED
LAB EXPERIMENTS are often referred to as having HIGH RELIABILITY due to the HIGH LEVEL OF CONTROL OVER THE INDEPENDENT VARIABLE, which in turn makes them EASIER TO REPLICATE by FOLLOWING THE STANDARDISED PROCEDURES
To IMPROVE RELIABILITY WITHIN EXPERIMENTS researchers might try to take MORE CONTROL OVER EXTRANEOUS VARIABLES, helping to further REDUCE THE POTENTIAL FOR THEM TO BECOME CONFOUNDING
how to IMPROVE THE RELIABILITY of OBSERVATIONS?
OBSERVATIONS can LACK OBJECTIVITY, since they are RELYING on the RESEARCHER’S INTERPRETATIONS of a SITUATION
If BEHAVIOURAL CATEGORIES are being used, its important that the researcher is APPLYING THEM ACCURATELY and NOT BEING SUBJECTIVE in their INTERPRETATIONS
One way to IMPROVE RELIABILITY would be to OPERATIONALISE the BEHAVIOURAL CATEGORIES - this means that the CATEGORIES NEED TO BE CLEAR and SPECIFIC on what CONSTITUTES the BEHAVIOUR IN QUESTION.
There should be NO OVERLAP BETWEEN CATEGORIES leaving NO NEED for PERSONAL INTERPRETATION OF THE MEANING
