Reliability Chapter 4

Question 1

The Domain sampling model definition of reliability

Answer 1

Reliability is the relation of the variance of the observed score on a shorter test in the variance of the long run true score

Question 2

Domain sampling model how is error introduced:

Problem and solution

Answer 2

By using a sample of items rather than the entire Domain of knowledge. The greater the items the higher the reliability

Question 3

Classical test score Theory

Formula

Answer 3

Observed score equals true score plus error

Question 4

Classical test Theory assumption

Answer 4

Error is random true score of a person doesn’t change with repeated applications of a test

Question 5

Item response Theory explanation

Answer 5

Computer chooses a the questions to give an individual based on how many correct questions they have already got question difficulty goes up and down based on the individuals getting items of similar difficulty right or wrong

Question 6

Item response Theory reliability

Answer 6

A shorter test has higher reliability

Question 7

Sources of error

2 things.

Answer 7

Situational factors EG loud noises and temperature

Items not representative of the Domain

Question 8

Three ways to estimate test reliability

Answer 8

Test-retest method
Parallel forms
Internal consistency

Question 9

Time sampling the test retest method

Answer 9

Administer the test on two occasions correlate the scores

Question 10

Problems with test-retest

2 things.

Answer 10

Carry over effects
practice effects
Maturation effects
Length of interval between test is important

Question 11

Parallel forms methods

Answer 11

2 equivalent forms that measure the same atribute using different items
Sometimes given on the same day with counterbalanced order
One of the most rigorous assessments of reliability commonly in use

Question 11

Parallel forms methods

Answer 11

2 equivalent forms that measure the same atribute using different items
Sometimes given on the same day with counterbalanced order
One of the most rigorous assessments of reliability commonly in use

Question 11

Parallel forms methods

Answer 11

2 equivalent forms that measure the same atribute using different items
Sometimes given on the same day with counterbalanced order
One of the most rigorous assessments of reliability commonly in use

Question 13

Split half method of reliability:
Definition
Problem
Solution

Answer 13

Two halves of a test
Correlatio

The spearman-brown formula corrects for using a shorter test
Estimates what the correlation between the two halves would have been if each half had been the length of the whole test

Question 14

Internal consistency reliability split half how to overcome the problem that all halves are slightly different in difficulty

Answer 14

Kuder Richardson formula considers always of splitting a test in half

Question 15

Split half reliability when to use cronbachs Alpha:

2 reasons

Answer 15

When variance is between the two halves is unequal

Scores have more options than right or wrong

Question 16

Reliability of a difference score:

Definition and how to find it?

Answer 16

If you want to know the difference between performances at 2 points in time
Convert scores to Z units
Reliability for the difference score is lower than the reliability of the first and second measures

Question 17

Reliability and behaviour observation studies:
Names for it
Best formula and why

Answer 17

Inter-rater reliability also called interscorer interobserver or into judge
Best method Kappa statistic
Kappa for chance agreement the servers are rating whether a behaviour occurred or did not occur there is a 50% chance of agreement

Question 18

Standard errors of measurement and the rubber yardstick: problem and solution

Answer 18

The larger the standard error of measurement the less certainty about the accuracy with which the atribute was measured.
Researchers use confidence intervals to show the probability that the true score Falls within a certain interval around the observed score

Question 19

How reLiable is reliable for basic research and for making decisions about people?

Answer 19

Basic research.70-.80

Instruments used to make important decisions .95

Question 20

Increasing reliability of a test

Definition and formula

Answer 20

Increase the number of items. The prophecy formula can estimate how many more items a test needs to gain an acceptable level of reliability
Prophecty Formula assumptions the probability of error for new items = the probability of error for original items tiredness and boredom is not going to increase error

Question 21

Discriminabilty analysis

Answer 21

Correlation between performance on single item and total test score if low item probably measures something different from the others.
Item may be too easy

Question 21

Attenuation of correlation

Answer 21

Correlations are diminished by measurement error

Question 22

Relibility needs Factor and item analysis because….

Answer 22

Tests are reliable if they are uni-dimensional one Factor should account for more variance than any other Factor

Question 23

Correcting for attenuation

Answer 23

To Estimate what the correlation would have been between two measures without error
Find the reliability of both measures and the correlation between them
Use the correction for attenuation reliability formula

Question 24

Use Krs20

Answer 24

Dichotomous

Question 25

Use alpha

Answer 25

Likert skewed batiance

No right or wrong answer

Question 26

Use pearson

Answer 26

Two continuous variables