Lecture 3A: Measurement Reliability

Question 1

What is measurement reliability?

Answer 1

Reliable if it measures in the same manner and results in the same answers when measuring the same characteristic, time after time.

This definition is based on Bailey (1997), p.103.

Question 2

Define systematic error.

Answer 2

Predictable, one-sided, constant error that can usually be corrected.

Example: An electronic scale that always yields 2kg more than the actual weight.

Question 3

Define random error.

Answer 3

Error that occurs by chance, difficult to detect or control.

Example: An electronic scale that gives varying readings over time.

Question 4

What is regression to the mean?

Answer 4

• Instrument used is unreliable
• Extreme scores on an initial test tend to be less extreme on subsequent tests.

Question 5

What assumptions are made when establishing measurement reliability?

Answer 5

• Stable characteristics of subjects
• Stable testing conditions

For example, using an electronic scale or blood pressure monitor.

Question 6

What is test-retest reliability?

Answer 6

Assess reliability of an instrument by testing the same subjects on two or more occasions.

Question 7

What factors may influence test-retest reliability ratings?

Answer 7

• Testing effect (practice/training effect)
  
  • e.g. 6 minute walk test
• Memory effect
  
  • e.g. questionnaires
• Real change in patients’ condition
  
  • e.g. blood pressure monitor
• Fatigue
  
  • e.g. hand grip dynamometer

Question 8

What is intra-rater reliability?

Answer 8

Involves a single tester/assessor over two or more trials.

Question 9

What is inter-rater reliability?

Answer 9

Involves two or more testers/assessors on the same group of subjects, usually in one testing session.

Question 10

Fill in the blank: A researcher has 4 OTs rate the same 20 patients using the Wolf Motor Function Test in the same session. This form of measurement reliability is called _______.

Answer 10

Inter-rater reliability.

Question 11

Fill in the blank: To assess the intra-rater reliability of the Barthel Index, you videotaped the patient’s performance and viewed the video in the first week and again in the second week. One factor that may influence the reliability rating is the _______.

Answer 11

Memory effect of the therapist.

Question 12

True or False: Systematic errors are always unpredictable.

Answer 12

False.

Question 13

True or False: Random errors can usually be corrected easily.

Answer 13

False.