Reliability

Question 1

What is reliability a measure of?

Answer 1

Consistency in measurement over time. (If your observation of the behavior will give you the same concept of the behavior at different points in time.)

Question 2

True Score Theory

Answer 2

The observed score = True Ability + Random Error

Question 3

What are the two types of error?

Answer 3

• Random

* Systematic

Question 4

How do the two types of error differ?

Answer 4

Random errors in experimental measurements are caused by unknown and unpredictable changes in the experiment. These changes may occur in the measuring instruments or in the environmental conditions.
Examples of causes of random errors are: electronic noise in the circuit of an electrical instrument, irregular changes in the heat loss rate from a solar collector due to changes in the wind.

Systematic errors in experimental observations usually come from the measuring instruments. They may occur because: there is something wrong with the instrument or its data handling system, or because the instrument is wrongly used by the experimenter.

Question 5

Random Error

Answer 5

*Increases variability, not the mean

Question 6

Why is there no change in mean with random error?

Answer 6

We assume random inflated scores will cancel out random deflated scores

Question 7

Good Variability

Answer 7

Variability in data that is truly based on differences in the construct (like if one person has more of a trait than somebody else).

Question 8

Noise

Answer 8

Variability around mean

Question 9

Systematic Error

Answer 9

Affects the average (shifts it)–this is called bias

*Can happen due to testing bias or history x selection, etc.

Question 10

How can you reduce Measurement Error (from systematic error)

Answer 10

• Pilot Test
• Train-standardization
• Double-check Data
• Stat Correction
• Multiple Measures

Question 11

What is a downside of Multiple Measures?

Answer 11

Although you get high reliability you get low efficiency.

Question 12

Types of Reliability

Answer 12

• Inter-Rater or Inter-Observer Reliability
• Test-Retest Reliability
• Parallel-Forms Reliability
• Internal Consistency Reliability

Question 13

Inter-Rater Reliability

Answer 13

Consistency across raters

Question 14

Test-Retest Reliability

Answer 14

Consistency of a single measurement at two points in time

Question 15

Parallel Forms Relaibility

Answer 15

Consistency across multiple versions of the same measure

Question 16

Internal Consistency Reliability

Answer 16

Consistency across items within the same measure at the same time