Reliability Flashcards
What is meant by reliability ?
The desired consistency or reproducibility of test scores. It shows the amount of error in a measurement tool
How do we calculate error in tests ?
x = T + e
x = observed score
T= true score
e= error
What are the assumptions made by die Classical Test Theory ?
1) each person has a true score we could obtain if there was no error
2) measurement error is random
3) true scores doesn’t change, but observed score does
4) distribution of errors and observed scores will be same for all
What does the domain sampling model tell us ?
We cannot ask all possible questions and fewer items can lead to introduction of error.
Reliability = the variance of observed score on short test devided by the variance of the true score
What is meant by die standard error of measurement (SEM)?
Work out the measurement error - i.e we work out how much on average an observed score on test differs from true scores.
When the SEM is known, we can create confidence intervals
What is the formula for SEM ?
Formula ?
What are the different types of reliability ?
1) Test-retest
2) Parallel forms
3) Inter-rater
4) Internal consistency
- split-half
- coefficient / Cronbach’s Alpha
Explain what is meant by test-retest reliability ?
When you give someone a test at one point in time and then give them the same test at a later point of time
If the scores are highly correlated (co-efficient of stability) then the reliability is good
The source of error measured is time sampling
What are the issues of test-retest reliability ? (3)
1) carry-over effects : test-takers attitude/performance at the second test could be influenced by the first test
2) Practise effects: being exposed to some information more than once could influence a test
3)Time between testing: too litte or too long time can influence second test
Explain Parallel forms reliability
When two forms of the same test is given to the same person
If scores are highly correlated (co-efficient of equivalence) , then there is good reliability
The source of error management is item sampling
What is meant by Inter-rater reliability ?
It measures how consistently raters (2 or more) agree - it correlates the raters scores
Source of error = variance across raters
Correlation = intra-class correlation coefficient
Ranges from 1 to -1
>.75 = exellent
.50 to .75 = satisfactory
<.40 = poor
Explain Internal consistency reliability
Do different items in one test measure the same thing to the same extent ? Internal consistency refers to how inter-related the items are
Use split-half and Cronbach’s Alpha
Source of error measured is internal consistency
How do we measure split-half reliability ?
What are the advantages and disadvantages?
Split a test in half and score each half seperately. The total scores of each half is correlated
Advantages - you only need one test
Disadvantages - we do not always know how to devide a test into equivalent halves
Why do we use the Spearman-Brown Correction formula ?
It is used to adjust for numbers of items in each half of test when calculating split-half reliability
What is the Coefficient/Cronbach’s Alpha used for ?
It estimates consistency of responses to different scale items. It measures the error associated with each item as well as the error associated with how well the test items fit together
Reflected with number between 0 and 1 :
>= .70 is exploratory research
>= .80 is basic research
>= .90 is applied research
