Validity and Reliability

Question 1

what is validity?

what is it dependant on?

Answer 1

the ability of a test to measure accurately, the degree to which a test measures what it purports to measure
–> dependant on reliability, relevance and appropriateness of scores

Question 2

what is reliability

Answer 2

the consistency or repeatability of an observation, the degree to which repeated measurements of a trait are reproducible under the same conditions

Question 3

4 common types of validity evidence

Answer 3

1. construct validity
2. logical validity
3. criterion validity
4. convergent validity

Question 4

all types of validity can be estimated either _______ or _______

Answer 4

logically, statistically

Question 5

the test effectively measures the desired construct

Answer 5

construct validity

Question 6

the measures obviously involves the performance being measured
- no statistical evidence is required

Answer 6

logical/face validity

Question 7

degree to which scores on a test are related to recognize standard or criterion (gold standard)

Answer 7

criterion validity

Question 8

AKA statistical or correlation validity

Answer 8

criterion validity

Question 9

how is criterion validity obtained?

Answer 9

by determining the correlation/ validity coefficient (r) between scores for a test and the criterion measure

Question 10

The criterion is measured at approximately the same time as the alternate measure and the scores are compared
- ex?

Answer 10

concurrent validity

ex, skin folds and hydrostatic weighing

Question 11

the criterion is measured in the future (week, months, years later)
- ex?

Answer 11

predictive validity

ex, the pre selection test battery score and the selection success

Question 12

2 or more measurements are conducted to collect data and establish that a test (battery) is measuring what is purports to measure

Answer 12

convergent validity

Question 13

the consistency of repeatability of an observation

- the degree to which repeated measurements of a trait are reproducible under the same conditions

Answer 13

reliability

Question 14

how do calculate reliability

Answer 14

rest retest scores to calculate reliability coefficient

eg. r= 0.99 to sit and reach (very high reliability)

Question 15

3 types of reliability

Answer 15

1. stability reliability
2. internal - consistency reliability
3. objectivity

Question 16

what is stability reliability ?

Answer 16

when scores do not change across days

–> look at the relationship b/w multiple trials across multiple days

Question 17

3 factors that contribute to low stability

Answer 17

1. the people tested may perform differently
2. the measuring instruments may operate or be applied differently
3. the person administering the measurement may change

Question 18

what is internal consistency reliability ?

Answer 18

evaluator gives at least 2 trials of the test within a single day

Question 19

the internal consistency reliability coefficient is not comparable to stability reliability coefficient, the I-C coefficient is almost always?

Answer 19

higher

Question 20

what is objectivity reliability

Answer 20

rater/judge reliability

- inter-tester reliability

Question 21

2 factors affecting objectivity

Answer 21

1. the clarity of the score system

2. the degree to which the ‘judge’ can assign a score accurately

Question 22

9 considerations for reducing measurement error

Answer 22

1. validity and reliable test
2. instructions
3. test complexity
4. warm up and test trials (learning effect - may need 5 trials)
5. equipment quality and preparation (e.g calibration)
6. testing environment
7. scoring accuracy
8. experience and state of mind of person conducting the test
9. state of mind of person being tested

Question 23

Why calibrate?

How calibrate?

Answer 23

important to confirm accuracy of what equipment is telling you

• requires comparison between measurements (one of known magnitude and one of unknown magnitude that need sot be confirmed)
• check equipment is up to date and proper functioning

Question 24

how often do you calibrate?

Answer 24

at least every 6 months, or according to manufacturer’s guidelines

Question 25

reliability can be expected when ? (3)

Answer 25

1. the testing environment is favorable to good performance
2. ppl are motivated , ready to be tested and familiar
3. the person administering the test is trained and competent