RESEARCH METHODS - reliability Flashcards
its a measure of
consistency
how do we measure reliability in psychology
if a test or measurement in psychology assessed x on a certain day
then we would expect the same result on a different day
unless thing itself had changed
how can we asses reliability
test retest
interobserver reliabiliity
what is a test retest
administering same test or questionnaire to same person or people on different occasions
in a test retest if the test or questionnaire is reliable results obtained should be
same or very similar each time theyre administered
test retest commonly used for
but can be applied to
quesytionnaires and psychological tests - such as IQ
interviews
in a test retest theri must be sufficient time between the test and retest why
ensure particpant/respondents cant recall their answers to questions to a survey
their must be sufficient time between a test retest but not too long why
as attitudes/opinions/abilities may have changed
in case of questionnaires/ test
what would happen to two sets of scores in test retest reliability - x to make sure x
and then if correlation turns out to be x and x
then the reliability of x x assumed to be x
correlated to make sure thier similar
if coorelation turns out to be significant and positive(+0.8 or greater )
then the reliability of the measuring instrument is assumed to be good
when is reliabliity assumed to be good
correlation is at least +0.8
how do we remeber correlation has to be +0.8
relaibility aint great unless its +.8
inter observer reliabiltiy - everyone has their own way of seeing
the world
explain why everyone seeing the world in their own way an issue in observational research
one obsververs interpreation of event may differ widely from someone else’s - this intriduced subjectivity bias and unreliabilty into the data collection process
how do we get over observational research issue of subjectivity bias
observers should conduct their research in teams of two (minimum)
watch same event and record their obsv independently
results should then be correlated @ end to assess for reliability
how can we estbalish intro observer reliability
2 ways
- pilot study of the observation in order to check behavioural categories being applied in same way by observers
- reporting a comparison at the end of sthe study
as with test retest method , data collected by two osbervers should be what to do what
correlated to assess its relaiability
in content analysis how would introobserver rewliabilty be refferd to
inter rater relaiblity
in interviews how would introobserver relaibiltiy be referred to
inter interviewer reliability
in interobserver reliabilty if at the end both observes come out and they dont find got strong +ve correlation in terms of what theyve observed maybe theyve done soemthign imporoperoly give an exmaple
behavioural categories not operationalised properly e.g too vague therfore researchers misinterpreting what tgetve seen
how do we measure reliability
correlational analysis
in terms of measuring reliabiliry in a test retest and inter observer relaiblity the two sets of scores are
correlated
the correlaito coffiecicne thsould exceed what for reliability
+.80
IMPROVING RELIABILTY - If a questionanire produces a low test relaiability it may require some questinos to be
deselcted
rewritten
IMPROVING RELIABILTY - in a questionnire if some questions too copmlex or ambiguis they may be interpeted
differntly by same person on differnet occasios
IMPROVING RELIABILTY - how do we reduce ambigiouty and coplexity inn questionanires
replace some open questions wher emay be room for misinerpeearion with closed , fixed choice alt
which may be less ambigious
IMPROVING RELIABILTY - interviews , wahts the best way of improving reliablity
use same interviewer each time
IMPROVING RELIABILTY - if cant use same intervewre each time as not possible or practical what cna we do
properly train all interviewres
prevent sone from asking leading or ambiguous questions questions
IMPROVING RELIABILTY - how can we easily avoid researchers asking leading or ambigious questions
using structured interviews
IMPROVING RELIABILTY - how can using structured interviews improve reliability
interviewres behaviour is mroe contorlled by fixed questions
whereas unstructured interviews ar emore free flowing and less likely to be reliable - as more likely ot ask leading or ambigous quetions
IMPROVING RELIABILTY - how can we imporved reliabilty of observatinos
making sure behaviorual categories are properly operatinoalised
theyre measurbale and self evident
give example of self evident (obvious)
pushing is much less open to interpreatation than aggressionn
IMPROVING RELIABILTY - behaviorual categories such not and should include
overlap
shoudl include all possible behaviours on the checklist
e,g hugging and ucddling
IMPROVING RELIABILTY - behaviorual categories should not overlap and should include
overlap
all possible behaviours shoudl be covered on the checklist
e,g hugging and ucddling
IMPROVING RELIABILTY - what ahppens if categories arent operationalised well , overlappig or absnet
diff observers got to make own judgement on what to record where
and may end up with differing and inconsistent records which reduces reliability
IMPROVING RELIABILTY - if reliablity is lo within observations what may researcehrs need and wish to do
may need further training in using behaviorual categories
may wish to discuss descision with eahc otehr so can apply categories more consistently
IMPROVING RELIABILTY - explain how we can improve it in terms of type of experiment
use lab studies
researchers got high level of control over procedures and instructions given which increases standardisation
IMPROVING RELIABILTY - in experiemtns what is the focus of relaibility
procedures
IMPROVING RELIABILTY - in epxeriments to coapmre performane of differnt particpants ( as well as results from differnt stydies procedure must be
the same /consistent everytime
IMPROVING RELIABILTY - in experietns in terms of reliablity experimenter is concerned about
stadardised procedures
why ar elab experiemtns more reliable than fiield experimetns
researcehrs can exert strict controls over many aspects of teh procedure
reliabiliy in experiemtns can be imporved by standardisation of procedures why
experimetn can be replicated and get consisten results
intro observer reliabilty helps overcome the issue of
researcher bias
