Assessing/ Improving Reliability

Question 1

what does reliability mean?

Answer 1

reliability is the extent which results can be repeated overtime.

Question 2

if something is reliable, there must also be …. CORRELATION.

Answer 2

HIGH

Question 3

the s…. of a correlation will be measured using the p…… or spearman coefficient.

Answer 3

STRENGTH / PEARSON

Question 4

what is high correlation coefficient? (the number)

Answer 4

+0.8 of higher

Question 5

a high correlation indicates what?

Answer 5

indicates that a test is reliable

Question 6

a low correlation coefficient indicates what?

Answer 6

poor reliability

Question 7

what are the two ways of assessing reliability for of questionnaires?

Answer 7

split half method and test-retest

Question 8

how do we test the reliability of an observation?

Answer 8

inter-rater method

Question 9

what is the split-half method?

Answer 9

Measures internal reliability. Split test into 2 and get the PPTs to do both halves. They should produce similar results. Used with questionnaires. Any areas where there is a low correlation should be removed from the questionnaires

Question 10

what is inter-rater/ observer method?

Answer 10

Using more than one researcher/observer to conduct the study, compare results.
Conduct a correlation of all researcher/observer scores. High correlation (+0.8 or more)=consistent research results.

Question 11

what is the test-retest method ?

Answer 11

Measures external reliability. Give the same test to the PPTs on 2 different occasions. Should get a similar result/ strong correlation.

Question 12

what is split-half method?

Answer 12

measure internal reliability. Split into 2 and get the ppts to do both halves. They should produce similar results. Used with questionnaires. Any areas where there is low correlation should be removed from the questionnaires

Question 13

how do we assess reliability?

Answer 13

if there is a high correlation coefficient ( r >0.8) which indicates the test is reliable, if low correlation coefficient indicates poor reliability.

Question 14

what does internal reliability mean?

Answer 14

does all of the same questionnaire measure the same thing

Question 15

what does external reliability mean?

Answer 15

are scores on questionnaire consistent overtime

Question 16

what are the 3 ways to improve reliability?

Answer 16

• where possible take more than one measurement from each ppt
• pilot study
• standardise the way in which researchers collect data, inter-rater reliability, such as training observers in use of the checklist

Question 17

what is a pilot study?

Answer 17

pilot studies are small, trail versions of purposed studies to test their effectiveness and make improvements. They can be helpful for identifying potential issues in a study early, which can then be rectified before commiting to the length and expense of a full investigation.

Question 18

give three potential problems that might be picked up by a pilot study.

Answer 18

• The procedure might take to long.
• The instructions to participants might not be clear enough.
• There might be vital steps or measurements that have been missed out.

Question 19

how would you improve the inter-rater reliability of an observational study?

Answer 19

1) standardise how the observers record results - use a checklist of clearly defined objective behaviours.
2) train the observers in the use of the checklist
3) carry out a pilot study to ensure that the checklist is suitable, and to allow the observers to compare their interpretations of the same behaviour.

Question 20

how would you improve reliability with the split-half method?

Answer 20

any areas where there is a low correlation should be removed from the questionnaire

Question 21

how would you improve reliability in test-retest method?

Answer 21

any items where there is a low correlation should be removed from the questionnaire