Reliability Flashcards
What is reliability?
Reliability is the stability of a test. Meaning that the score you get on a test Tuesday will be the same score you get on the test Wednesday.
What is the formula for reliability in CTT?
Reliability = 1 - [VAR(E)/VAR(X)]. Which is 1 minus the ratio of random error variance to total score variance.
What is Test-Retest Reliability?
Test-restest reliability tests the stability of a test over time. That means that the same test is given to the same sample at time point 1 and then at time point 2.
How do you analyze data for Test-Restest Reliability?
You run a Pearson correlation to see if there is a high correlation between the two time points.
What are drawbacks of Test-Retest Reliability?
Attrition. It is expensive to administer two tests. carryover effects from T1 to T2. The time interval may be inappropriate.
What is Alternative Forms Reliability?
It is much like Test-Retest Reliability, however, instead of the same test at two different times you can use two different tests that are equivalent. Uses the same analyzes as Test-retest, and has the same drawbacks except for no carryover effects & concern of the two different tests being the same.
What is internal consistency?
Assess the stability or consistency of responses across items and are primarily based on intercorrelations between items.
What is being analyzed with internal consistency?
Stability indices are not being measured across total test scores but instead across items.
What is an advantage of internal consistency measures?
The stability indices can be measured on a single sample as they are not measured between subjects’ total scores but instead across items.
What is Split-Half Reliability?
Split-Halves is when a test is split in half into two equal parts and the scores on the two halves are correlated.
What is a problem with Split-Half Reliability?
A test that is split in half shortens a test. The problem with a short test is that under CTT longer tests lead to higher reliability. Therefore, a split-half test will underestimate the reliability.
How do you analyze a Split-Half Reliability?
You can use a Pearson Correlation, however, it will underestimate the reliability. Therefore, the correct test would be the Spearman-Brown.
What do you use the Spearman-Brown correlation for?
You can use the Spearman-Brown for calculating the reliability of a split-half test. You can also use the Spearman-Brown if you need to shorten a reliable test. For example, if you have a 100 item test but you shortened it to 75 for an experiment you would want to use the Spearman-Brown to know what the new reliability would be. You can also use the Spearman-Brown to test how many items to add to a test to increase the reliability of a test.
What is Cronbach’s alpha used for?
It is a measure of internal consistency. It is the average of all possible spilt-half reliabilities and it is the lower bound for a test of parallel items.
What is Coefficient Theta used for?
It is based on a principal component analysis of test items.
