reliability - A-level

Question 1

test-retest definition

Answer 1

a method of assessing the reliability of a questionnaire or psychological test by assessing the same person on two separate occasions. this shows to what extent the test (or other measure) produces the same answers - is constant and reliable

Question 1

reliability definition

Answer 1

refers to how consistent a measuring device is- this includes psychological tests or observations which assess behaviour

Question 2

inter-observer reliability definition

Answer 2

extent to which their is agreement between two or more observers involves in observations of a behaviour. this is measured by correlating the observers of two or more observers. a general rule is that if…
(total number of agreements)/(total number of observations) > 80
the data has high inter-observer reliability

Question 3

what is reliability a measure of

Answer 3

measure of consistency

Question 4

how is a measurement seen as being reliable

Answer 4

if it is made the same result then that measurement is described as being reliable

Question 5

what happens if there is a change in a measurement over time

Answer 5

attribute that change to the object being measured

Question 6

when we would we expect the same results

Answer 6

is we assed a thing on a particular day we would expect the same results on a different day unless the thing itself has changed

Question 7

why is reliability important

Answer 7

psychologists research things that aren’t concrete such as attitudes, aggression and memory. it is important researchers can have confidence in their psychological tests, observations and questionnaires

Question 8

what are ways of assessing reliability

Answer 8

-test retest
-inter observer reliability

Question 9

what does test-retest involve

Answer 9

administering the same test or questionnaire to the same person on different occasions

Question 10

what happens in test-retest if results are reliable

Answer 10

results obtained form test or questionnaire (can also be applied to interviews) are the same or very similar every time

Question 11

how does test-retest need to be used

Answer 11

sufficient time between test and retest is needed to ensure all participants cannot recall their answers to questions to a survey, but also not so long that attitudes, opinions or abilities have changed

Question 12

what happens at the end of test-retest

Answer 12

two sets of scores are correlated to make sure they are similar and if the correlations turn out to be significant (and positive) the reliability of the measuring instrument is assumed to be good

Question 13

why is inter-observer reliability used

Answer 13

everyone has their own unique way of seeing the world

Question 14

why is inter-observer reliability important in observational research

Answer 14

one observers opinion may differ from another’s which introduces subjectivity and bias into the data collection

Question 15

how many observers need to observe

Answer 15

at least 2

Question 16

how can inter-observer reliability be achieved in a pilot study

Answer 16

check observers are applying behaviour categories in the same way

Question 17

how is inter-observer reliability carried out at the end of a study

Answer 17

comparison

Question 18

how is inter-observer reliability carried out

Answer 18

observers need to watch the same event but record their data independently. the data needs to be correlated

Question 19

What are the names of 2 methods (similar to inter-observer reilabilty) that are used in content analysis and interview to improve reliability

Answer 19

-in content analysis where is it inter-rater reliability
-in an interview where is is called inter-interviewer reliability

Question 20

how is reliability measured

Answer 20

correlation analysis between the 2 sets of scores

Question 21

what should the correlation coefficient be for tests to be reliable

Answer 21

+0.80

Question 22

how do you improve reliability in a questionnaire

Answer 22

-some items need to be removed or rewritten
-if questions are too ambiguous they can be interpreted differently so may need to replace open questions with closed, fixed choice questions that are less ambiguous

Question 23

how is reliability improved in interviews

Answer 23

-use the same interviewer each time
-if the same interviewer cannot be used, all the interviewers must be properly trained
-questions should not be leading or ambiguous which is more easily avoided by fixed choice questions
-this is because unstructured interviews are more free flowing and less likely to be reliable

Question 24

how can the reliability of observations be improved

Answer 24

-behaviour categories should be properly operationalised and be measurable and self-evident –> this is because there is much less interpreation
-behaviour categories should not overlap and all possible behaviours should be covered on the checklist
-all categories should be operationalised and not be overlapping or missing as observers would have to make their own judgements which could end up differning and being inconsistent
-if the reliability is low observers may need further training in using behavioural categories and/ or may wish to discuss their decisions with each other so they apply categories more consistently

Question 25

how can the reliability of experiments be improved

Answer 25

-procedures are the focus of reliability
-in order to compare the performance of different participants (and different studies) the procedures must be consistent every time
-standardised procedures