Measurement Issues

Question 1

What is measurement reliability?

Answer 1

extent to which a measurement is consistent & free from error, indicates ability of an instrument to produce similar scores, any test is situation specific

Question 2

What are the 6 ways measurement reliability is assessed by?

Answer 2

1. accuracy
2. consistency
3. stability
4. precision
5. reproducibility
6. dependability

Question 3

What 2 characteristics is measurement validity dependent on?

Answer 3

reliability & relevance

Question 4

All valid tests are ____, however all ____ tests are not valid.

Answer 4

reliable, reliable

Question 5

What are 3 sources of measurement error?

Answer 5

• the individual taking the measurement
• variability of characteristics being measured
• the measuring instrument itself

Question 6

What is predictable, consistent, and usually correctable?

Answer 6

systematic error

Question 7

What is due to chance, unpredictable, and effect scores are unprectable?

Answer 7

random error

Question 8

What is the term for the difference between T (true score) and X?

Answer 8

measurement error

Question 9

systematic error:

Answer 9

A form of measurement error, where error is constant across trials.

Question 10

random error:

Answer 10

measurement are due to chance and can affect a subjects score in an unpredictable way from trial to trial

Question 11

What are 6 ways to avoid measurement error?

Answer 11

• clear operational definitions
• careful planning
• using detailed procedures
• being trained in procedures
• practice of specific procedures
• inspection of equipment

Question 12

regression toward the mean:

Answer 12

A statistical phenomenon in which scores on a pretest are likely to move toward the group mean on a posttest because of inherent positive or negative measurement error; also called statistical regression.

Question 13

variance:

Answer 13

a measure of the variability of differences among scores within a sample

Question 14

The larger the variance the greater the _____.

Answer 14

dispersion of scores

Question 15

The smaller the variance the more ____.

Answer 15

homogeneous the scores

Question 16

You have measured dogs for height the heights (at the shoulders) are 600mm, 470mm, 170mm, 430mm, and 300mm. Find the mean, variance, and standard deviation.

Answer 16

mean: 394
variance: 21,704 (all answers squared / # of answers)
standard deviation: 147 (square root of variance)

Question 17

with zero error the ratio will produce what coefficient?

Answer 17

1.00

Question 18

coefficient:

Answer 18

a number used to multiply a variable

Question 19

reliability coefficient of <.50 is:

Answer 19

poor

Question 20

reliability coefficient of .50-.75 is:

Answer 20

moderate

Question 21

reliability coefficient of >.75 is:

Answer 21

good

Question 22

test-retest reliability:

Answer 22

• stability of the instrument
• repeated administration of test
• administered by one rater
• one sample tested repeatedly
• time intervals are important
• depends on stability of what’s being measured
• assumption is that what is being measured remains the same
• any changes seen is then due to random error

Question 23

intrarater reliability:

Answer 23

1 individual over 2 or more trials. Usually over short interval of time
- Statistics used ICC model 2 or 3

Question 24

interrater reliability:

Answer 24

The degree to which two or more raters can obtain the same ratings for a given variable.

Greater than or equal to 2 raters measuring same subject group. Best done during a single trial. Important for generalizability.
- Statistics used ICC model 2 or 3

Question 25

internal consistency:

Answer 25

-

This is the homogeneity of items on a test. Reflexts extent to which items measure various aspects of a characteristic. Important when developing physical performance measures. Looks at correlation of all items on the test.

Possible statistics used are Cronbach’s Coefficient Alpha, Spearman-Brown Prophecy Statistic, Item-to-Total Correlation.

Question 26

What are the carryover effects of test-retest?

Answer 26

training & motivation

Question 27

What are the testing effects of test-retest

Answer 27

pain ensues & soft tissue stretch

Question 28

What are the statistics used for test-retest?

Answer 28

intraclass correlation coefficient (ICC) model 3 & kappa statistics (nominal data)

Question 29

What are examples of alternate forms of reliability? What are the statistics used?

Answer 29

• exists in 2 or more versions of a test
• equivalent, parallel, or alternate forms
• 2 alternate forms administered to the same group in 1 setting
• will look at correlation between paired observations
• needed to generalize findings from one situation to another

statistics used are limits of agreement & correlation coefficients

Question 30

What are 6 factors that affect reliability?

Answer 30

• things internal to participant
• time between testing
• testing period circumstances
• appropriate level of difficulty
• precision of measurement
• environmental conditions

Question 31

What is the standard error of measurement (SEM)?

Answer 31

Relates to the reliability of the measurement. The degree to which an observed score fluctuates due to errors of measurement. Is the standard deviation of the errors of measurement around the observed score.

Question 32

true score + random error = actual score

Answer 32

classical reliability theory

Question 33

true score + random error + test-retest error + rater error + other error sources = actual score

Answer 33

generalizability theory (Cronbach)

Question 34

What is the minimal detectable difference?

Answer 34

The amount of change in a variable that must be achieved to reflect a true difference. The greater the reliability the smaller the MDD.

ex) pain scale- change of 2 is MDD. MMT- a full grade, not plus and minus.

Question 35

pilot testing:

Answer 35

a smaller sample to test reliability of measurement