Reliability Flashcards
What is reliability?
It refers to the consistency with which a test measures what it purports to measure in any given set of circumstances
What does Domain-sampling Model refers to?
It refers to sampling items from the population of possible items that could be used in a test.
The 4 races in singapore example
What is Standard Error of Measurement (SEM)?
An expression of the precision of an individual test score as an estimate of the trait it purports to measure
Explain Reliability Coefficient.
An index – often a Pearson product moment correlation coefficient – of the ratio of true score to error score variance in a test as used in a given set
What does a reliability coefficient of 0.5 means?
50% of the variance in the scores obtained with the test is due to true score variance, and the remaining 50% is to errors of measurement
(measurements are moderately consistent but not highly reliable)
Explain split-half reliability.
It is the estimate of reliability obtained by correlating scores on the two halves of a test formed in some systematic way (e.g., odd vs even items)
What is the disadvantages of split-half reliability?
- Potential bias
- Limited sample size – Splitting a measurement instrument into two halves reduces the sample size available for each half, which can result in reduced statistical power. Smaller sample sizes can lead to less precise estimates of reliability.
- Dependency on Splitting method
What is Cronbach’s alpha?
It is an estimate of reliability that is based on the average intercorrelation of the items in a test
How do we interpret Cronbach’s alpha?
Cronbach’s alpha ranges from 0 to 1.
A higher value indicates greater internal consistency reliability, with values closer to 1 indicating stronger consistency among the items.
What is a G Theory?
G theory is known as generalizability theory.
It is a statistical framework that extends the concept of reliability in psychometrics.
It provides a method to estimate and understand the various sources of measurement error and assess the generalizability or reliability of test scores across different conditions or contexts.
What is the purpose of G Theory?
It is to determine the consistency and dependability of measurements when there are multiple sources of variation.
It goes beyond traditional reliability estimation methods, such as Cronbach’s alpha, by considering multiple sources of error and their interactions.
Explain Inter-rater Reliability.
The extent to which different raters agree in their assessments of the same sample of ratees
What is the ideal ranges of reliability for a test?
> 0.5 for test development
0.7 for using a test in research
0.9 for use in individual assessment
Explain Equivalent Forms Reliability
It is a way to check if different versions of a test that measure the same thing produce consistent results.
What is one advantages of Equivalent Forms Reliability?
- Avoid problems such as practice effects and biases that may arise from using a test repeatedly
- Have different versions of a test that can be used in different situations, while still ensuring that we are measuring the same thing consistently.
