Exam One - measurement reliability

Question 1

reliability

Answer 1

consistency with which an instrument or rater measures a variable
extent to which a measurement is free from error

Question 2

validity

Answer 2

concerns the closeness of the measurement to the “true” value
is the test measuring what it is intended to measure?

Question 3

Measurement =

Answer 3

true score + error

Question 4

The _________ is the recorded data point and the _______ is the real or actual value under investigation

Answer 4

measurement, true score

true score is fixed and exists independently of the measurement

Question 5

Measurement error

Answer 5

variability between measurements is an estimate of the extent of measurement error

Question 6

The greater the _____ between measurements the __________ the reliability is presumed to be

Answer 6

variability, lower

Question 7

What are the primary contributors to measurement error?

Answer 7

1 - subject: mood, fatigue, injury type, practice, motivation, knowledge
2 - testing: test instruments, multiple testers, tester skill, environment
3 - instrumentation: calibration, measurement drift

Question 8

________ reliability increases you confidence that DV differences between observation are ______

Answer 8

greater, real

Question 9

formula for estimating reliability

Answer 9

(sample variability between subjects at test one)/(sample variability between subjects at test one + (test2-test1))

Question 10

ratios closer to 1 are considered

Answer 10

reliable

Question 11

ratios closer to 0 are considered

Answer 11

unreliable

Question 12

intrarater reliability

Answer 12

extent that scores of a tester agree with a second set of scores by the same tester

Question 13

inter-rater reliability

Answer 13

extent that scores of one tester agree with scores for a different tester

Question 14

what are some important reliability considerations?

Answer 14

• would other raters achieve similar results?
• timing of retest?
• measurements performed on separate occasions?

Question 15

How would the reliability estimate be affected if the true score DID change between test 2 and test 1?

Answer 15

differences will always be interpreted as error!

Question 16

ICC stands for?

Answer 16

intraclass correlation coefficient

Question 17

ICC

Answer 17

comparison of two or more repeated measurements
- continuous data
- provides estimate of association and agreement in absolute magnitude

Question 18

What is the cutoff categories for ICC?

Answer 18

<0.75 = unacceptable
0.75-0.9 = good but consider other options
>0.9 = excellent and acceptable

Question 19

Kappa stat

Answer 19

categorical data only
- often used to measure agreement between raters

Question 20

What are the ranges for kappa statistc?

Answer 20

<0.2 = no different than expected by chance alone
.2-.4 = minimal agreement
.4-.6 = moderate
.6-.8 = substantial
>0.8 = near perfect (suitable for clinical practice)

Question 21

________ statistic corrects for chance agreement

Answer 21

kappa

Question 22

correlation coefficient

Answer 22

describes the extend that one measurement varies with another
- continuous data

Question 23

R value meanings

Answer 23

positive = direct relation
negative = inverse relation
range: -1 to 1
|R|
0.25-.5 = fair
.5-.75 = moderate
>0.75 = good

Question 24

Pearson correlation coefficient _________ represent the extent of the agreement of the magnitude for the two measures

Answer 24

does not
- is NOT a TRUE measure of reliability