Reliability Flashcards
Reliability definition?
the extent to which a measurement of behav is consistent
internal reliability definition?
consistency within a study or measure (tool and/or researcher)
across researchers def?
more than 1 researcher - is there consistency in how they are interpreting behav or carrying out the study
inter-rater reliability def ?
- internal reliability
- 2+ observers watching same behaviour should have same data
- both can agree on
- mainly used in observations
across measures reliability def?
- MORE THAN 1 MEASURE - if there is more than 1 test (such as used in a repeated measures design) then is there consistency of challenge in each test?
split-half reliability def ?
- used to test IR
- answers to test are halved and compared - should be similar
- questionnaire/ test of behav
- ppts/ measures ect..
External reliability definition def?
consistency across time (different occassions)
across time or different occasions def?
- Behaviour is only measured once - the test may not measure behav consistenly at other times e.g. a personality test may only indicate ppt’s personality traits at the time the study was carried out
test - retest reliability def?
- assess ER
- test scorers compared, should be same
- retested again (after a while)
Dealing w issues of IR - more than one researcher?
More than 1 researcher:
- clearly operationalise variable so less room for
interpretation
- standardise procedures
- detailed training of researchersDealing w issues of IR - multiple measures?
multiple measures:
- standardize measures - make sure they are the same
Dealing w issues of IR - within a measure?
within a measure:
- standardise each question or item
method: Lab environment, reliability:?
High R - high control of the envir - control over EVs
Method: field experiment, Reliability:?
Low R - experiment in real life conditions - low control over EVs
Method: quasi experiment - Natural, R:?
Low R - naturally occurring IV -low control over EVs
Questionnaires - Reliability?
High R - ppts can repeat questions ro check for consistent results
Interview - unstructured: R?
Low reliability as each ppt is asked diiff questions - diff to repeat
Interview structured: R?
High R - each ppt is asked the same questions so it can be repeated to check for consistent results
Observation - controlled: R?
High R - behav is observed in a controlled envir - control over EVs
Observation - naturalistic: R?
Low R - behav is observed in real life envir - low control over EVs
Case study: R?
Low R - unique cases - impossible to repeat in same conditions
Content analysis: R?
High R - behaviour categories - can be repeated/ checked by other researchers
Assessing IR: Inter- rater reliability?
- all of the researchers carry out a pilot on the same ppts, if they are 80% or higher in agreement then they are reliable
- Pilot the different measures on the same ppts w/o changing the conditions and then compare their scores to ensure consistency
Assessing IR: Split half?
Measure behaviour using 1st half of the test and then the second half of the test using same ppts, if scores are similar, then the test is consistent throughout
Assessing ER?
- Test - retest
- The same ppts repeat the same test at another time e.g. 4 weeks later, to show a consistency over time
How to deal with ER?
Ensure tests ect. Are designed in relation to measuring behaviours, attitudes ect beyond one setting or time
What is assessing reliability?
Measures used to check the consistency of a set of results
DEFINITION FOR EXAM - inter rater reliability?
- Method of assessing IR
- Where 2 or more psychologists in the same study produce consistent results by using a standardised procedure , agreed coding system, or correlation of their data.
- This can be checked for by comparing each psychologists results if there is 80% + agreement then = IR
DEFINITION FOR EXAM - split half?
- Method of assessing IR
- Involves splitting a pp’s test answers in half and seeing whether s/he got the same or similar scores on the 2 halves. If so, IR is high
DEFINITION FOR EXAM - test re-test?
- Method of assessing external R
- Involves testing and retesting the same ppts over time, with the same test, and comparing their scorers. If the scores are the same, the test has ER
What are the issues of internal reliability?
1- more than 1 researcher
2 - more than 1 measure
3 - within a measure
What are the issues of external reliability?
Behaviour is only measured once
How to deal with issues of IR?
1- inter rater reliability
2 - split half
How to deal w issues of ER?
1- test retest
