Week Four - Reliability & Validity

Question 1

In order for measures to be reliable..

Answer 1

Tools used for measurement must be consistent and accurate (precise and unbiased)

Question 2

In order for measures to be valid..

Answer 2

Descriptions, relations and explanations (causation) must be truthful

Question 3

Reliability =

Answer 3

Consistency

Question 4

What are the three broad types of

consistency?

Answer 4

Consistency over time (test-retest reliability)
Consistency across items (internal consistency)
Consistency across researchers (inter-rater reliability)

Question 5

What is consistency measured by?

Answer 5

Variability

Question 6

Explain the inverse relationship between variability and reliability.

Answer 6

High variability = low reliability

Low variability = high reliability

Question 7

Ameasure that is reliable is not automatically..

Answer 7

Valid

Question 8

Variability occurs for what 2 reasons?

Answer 8

Causal relationship between an IV and DV (effect).

.Variability due to random factors (noise).

Question 9

Short-term change may indicate..

Answer 9

Problems with reliability (and therefore validity)

Question 10

Long-term change may reflect

Answer 10

More likely to reflect real changes/effects in

variable, not merely error

Question 11

What is Test-retest Reliability?

Answer 11

Test-retest reliability is a measure of reliability obtained by administering the same test twice over a period of time to a group of individuals. The scores from Time 1 and Time 2 can then be correlated in order to evaluate the test for stability over time.

Question 12

What is Internal consistency?

Answer 12

Consistency of responses across test items.

If participants are consistent in their responding to the questions scores on different items should be correlated.

Question 13

How do we test internal consistency? (2)

Answer 13

Split-half reliability.

Cronbach’s alpha (α)

Question 14

What is Interrater reliability?

Answer 14

Inter-Rater Reliability refers to statistical measurements that determine how similar the data collected by different raters are. A rater is someone who is scoring or measuring a performance, behavior, or skill in a human or animal.

Question 15

Inter-rater reliability is tested using what?

Answer 15

Cohen’s kappa (κ)

Question 16

What is Validity?

Answer 16

Validity is the extent to which measures represent what they are intended to represent

Question 17

Three major forms of validity

Answer 17

Construct validity
Internal validity
External validity

Question 18

What is Construct validity?

Answer 18

Extent to which manipulated variables (independent variables, IVs) and measured variables (dependent variables, DVs) represent what they are intended to.
- Are IQ tests a measure of intelligence?

Question 19

Construct validity can be tested through what 2 forms of validity?

Answer 19

Convergent and Discriminant validity

Convergent Validity would be demonstrated by strong, positive correlations (≥ .8)

Discriminant Validity would be demonstrated by weak correlations among assumed unrelated measures (≈0).

Question 20

What is Face Validity?

Answer 20

Do IVs and DVs appear to reflect or measure what they are intended to measure.

Question 21

What is Internal Validity?

Answer 21

Extent to which causal statements about the relations among variables can be made.

Question 22

What is External validity?

Answer 22

The extent to which the findings of study can be generalised beyond the study itself.

Question 23

External validity can only be confirmed by

Answer 23

Replication

Question 24

Internal Validity is dependent upon what?

Answer 24

Study design

Question 25

Designs that effectively control extraneous variables have high what?

Answer 25

Internal validity

Question 26

When does Externally invalid research occur?

Answer 26

When research conclusions are limited to original setting, sample and/or situation.

Question 27

What does Convergent validity test?

Answer 27

whether constructs/measures that should be related are related.

Question 28

What does discriminant validity test?

Answer 28

Tests whether constructs/measures that should be unrelated are unrelated.

Question 29

What does predictive validity test?

Answer 29

Tests whether current performance on some measure predicts future performance on a different.

Question 30

What does Concurrent validity test?

Answer 30

Test whether performance on a validated test correlates with performance on a new test.

Question 31

What are the 7 threats to Internal validity?

Answer 31

• History
• Maturation
• Instrumentation decay
• Statistical regression
• Selection
• Testing
• Mortality