Reliability & Validity

Question 1

What is test-retest reliability?

Answer 1

• testing/measuring things again to make sure its the same

Question 2

What question does test-retest reliability answer?

Answer 2

does the test produce the same results every time?

Question 3

What is internal consistency?

Answer 3

• do the items in the study correlate highly with one another

Question 4

What are the two ways to measure internal consistency?

Answer 4

• cronbachs alpha
• split-half reliability

Question 5

What question does internal consistency answer?

Answer 5

are different questions on the same test measuring the same thing?

Question 6

What is inter-rater reliability?

Answer 6

• the agreement of observations made by two or more raters

Question 7

How to measure inter-rater reliability?

Answer 7

cohen’s kappa

Question 8

What does inter-rater reliability answer?

Answer 8

Are different researchers measuring the same thing

Question 9

What is the relationship between reliability and validity?

Answer 9

reliability is necessary but not sufficient condition for validity

Question 10

What are the 3 questions about concerns of validity?

Answer 10

• is there a real relation between IV and DV?
• are the findings generalizable?
• are we measuring what we intended to measure?

Question 11

What is the question of internal validity?

Answer 11

Can I infer that the IV is really related to the DV?

Question 12

What type of validity is the answer to the question: Do the findings generalize beyond this particular study?

Answer 12

External validity and ecological validity

Question 13

What is ecological validity?

Answer 13

real-world meaningfulness

Question 14

What is construct validity?

Answer 14

the degree to which a test or measurement tool accurately assesses the theoretical concept or “construct” it is intended to measure

Question 15

What are the 8 sources of threat to validity?

Answer 15

• selection bias
• selection drop-out
• history
• maturation
• testing
• instrumentation
• regression toward the mean

Question 16

What is selection bias?

Answer 16

 Assignment of non-equivalent participants to groups being compared
* Ex. Ses, gender, age

Question 17

What is the term for when there is a systematic loss of participants?

Answer 17

selection drop-out or attrition

Question 18

What is the term for when questioning if there are any events contributing to the results beside the IV?

Answer 18

history

Question 19

What is maturation ?

Answer 19

changes in behavior/ results due to maturation

Question 20

What is the testing threat to validity?

Answer 20

effects of taking a test on performance at a later test

Question 21

what is the instrumentation threat to validity?

Answer 21

unintended changes in experimenters, observers, or measuring instruments