A2 RESEARCH METHODS - RELIABILITY AND VALIDITY Flashcards
Define reliability
Consistency of findings, methods and measuring tools
Describe the ways of assessing reliability
Test-retest reliability: the same ppts do the same test using the same materials about 2 weeks after the first test; 2 weeks so that practice is minimised but the thing you’re measuring (e.g. IQ or memory) can’t have changed that much. If the results for both tests are similar then good reliability
Inter-observer/inter-rater: Having 2 or more observers/raters work independently on the same study and then have their findings correlated; if the correlation coefficient is bigger than or equal to +0.80, then there’s good reliability
Describe the ways of improving reliability
In…
Experiments: keep all conditions the same by std.ising procedures; measuring tools must be kept the same; must do things like check the batteries on any stopclocks before you run the exp, etc
Questionnaires: if you complete the test-retest and there’s poor validity, you may need to remove/change some of the questions to make them less ambiguous (e.g. change some open ended q’s into closed ones)
Observations: clearly operationalise all the beh.al categories to remove ambiguity and all observers are recording the same beh. Also ensure that beh.al categories do not overlap, as this may become unclear on which category a beh falls under
Interviews: use a structured interview w/ the same interviewer and q’s for all ppts; if the same interviewer isn’t practical, then make sure you train all the interviewers to deliver the same wording/tone etc
Define validity
Validity in general is the accuracy of the research; are you actually measuring what you intended to measure? Results can be reliable and also invalid e.g. if you ask a student to hit a 20 on a dartboard and they consistently score 5s, they’re reliable as they score 5 each time, but they’re invalid as they’re not getting 20s. However, findings cannot be valid but unreliable.
Internal validity is the accuracy inside of the study i.e. did the IV cause the DV or was it an EV?
External validity is the accuracy outside of the study i.e. can the findings be generalised to outside of this study?
Describe the 4 types of validity
Ecological validity: can we generalise the findings to everyday life? The setting of the study and the mundane realism of the tasks themselves can affect the ecological validity of a study.
Temporal validity: do the findings hold up over time; Asch’s research has been criticised as a “child of the times” in very conformist 1950s America, so it may have low temporal validity.
Face validity: do the findings look right/seem to be accurate?
Concurrent: do the results from this new test match the results from an already existing test?
Describe the 2 ways of assessing validity
To assess…
Face validity: “eyeball” the results/ give to an expert to check if it seems to be true
Concurrent validity: correlate the results from the new test with an already existing test; if the correlation coefficient is +0.80 or bigger, then the concurrent validity is good
Describe how validity can be improved
In…
Experiments: use of control roups, std.ised procedures, double blind technique so minimise researcher bias etc. Field experiments tend to have a high ecological validity and lab experiments tend to have good internal validity due to control of EVs, but the materials and tasks obviously also affect this
Questionnaires: ensure ppts know that their info is being kept strictly confidential; they’ll tend to be more honest. Also sometimes reverse the q’s/ add in “lie q’s” to make sure they’re not just randomly circling things
Observations: use a natural observation (so that ppts behave more naturally), non-ppt (to keep objective and minimise the effects of researcher bias), covert (so that ppts behave more naturally if they’re unaware they’re being studied) and ensure categories are operationalised and do not overlap
Qualitiative research (e.g. content analysis): ensure reporting is clear and coherent, use direct quotes, use triangulation (use of lots of different sources for the same ppt e.g. interviews, questionnaires, diaries etc)