Week two: Psychological Assessment (HON403)-Reliability & Validity Flashcards
What is Test-Retest Reliability?
It measures the stability of a test over time. If the test is given to the same group at different times, it should give similar results.
What is Internal Consistency?
It refers to how well the items on a test measure the same construct. Common methods include Cronbach’s alpha and split-half reliability.
What is Inter-Rater Reliability?
It is the degree to which different raters agree on their assessments. High inter-rater reliability means raters give similar scores.
What is Parallel-Forms Reliability?
It assesses if two different forms of the same test yield similar results, ensuring they measure the same construct.
What factors affect Reliability?
Test Length: Longer tests tend to have higher reliability.
Test-Retest Interval: The time between test administrations can affect reliability.
Variability in Scores: Greater variability can improve reliability.
How is Reliability expressed?
Reliability is expressed as a coefficient from 0 (no reliability) to 1 (perfect reliability). A coefficient of 0.70 is considered acceptable.
What is the Cronbach’s Alpha (α)?
it is a measure of internal consistency.
Interpretation:
α ≥ 0.70: Acceptable reliability
α ≥ 0.90: Very good reliability
α < 0.70: May indicate low reliability
What is Standard Error of Measurement (SEM)?
It estimates the standard deviation of measurement error. It shows how much an individual’s observed score might vary from their true score.
What is the Domain Sampling Model?
It assumes a test measures only a sample of the domain of a construct. The test’s reliability reflects how well it represents the broader domain of possible items.
What is Generalizability Theory?
It extends reliability by considering multiple sources of error, such as raters, occasions, or items, affecting the test score.
How can Reliability be improved?
Increase the number of test items.
Standardize test administration.
Provide clear instructions.
Train raters well.
Why is Reliability important in practice?
Reliable tests ensure consistent, trustworthy results. Unreliable tests can lead to poor decisions, misdiagnoses, and ineffective interventions.
What is Inter-Rater Reliability?
It refers to the degree of agreement or consistency between different raters or observers. High inter-rater reliability means raters give similar scores for the same individual or item.
What is a Reliability Coefficient?
It quantifies the degree of consistency or stability in a test. It ranges from 0 (no reliability) to 1 (perfect reliability), and can be estimated using methods like test-retest, internal consistency, and inter-rater reliability.
What is Test-Retest Reliability?
It measures the consistency of test scores over time. The test is given to the same group at two different times, and the correlation between the scores estimates the reliability.
What is Content Validity?
It refers to the degree to which a test represents all aspects of the construct it aims to measure. For example, an intelligence test should cover all major facets of intelligence.
How is Content Validity established?
Content validity is often established through expert judgment, where experts review the test items to ensure they represent the full domain of the construct.
What is Construct Validity?
It refers to the degree to which a test measures the theoretical construct it is intended to measure. It is assessed through empirical research.
What is Convergent Validity?
t occurs when a test strongly correlates with other measures that assess the same or similar constructs, showing that the test is measuring what it’s supposed to.
What is Discriminant Validity?
It occurs when a test does not correlate highly with measures of different constructs, ensuring that the test is not measuring something unrelated.
How is Construct Validity assessed?
It is typically assessed using statistical methods such as factor analysis or correlation coefficients between the test and other relevant variables.
What is Concurrent Validity?
It refers to the degree to which test scores correspond with an external criterion measured at the same time. It helps to validate a test when a well-established measure is available for comparison.
How is Concurrent Validity assessed?
It is assessed using correlation coefficients between the test score and the criterion score (external measure).
What is Predictive Validity?
It refers to how well a test predicts future performance on a related criterion, such as predicting academic success or job performance.
How is Predictive Validity assessed?
It is assessed using a correlation coefficient between test scores and future performance measures.
What is Incremental Validity?
It refers to the added value of a new test in predicting a criterion, beyond what is already predicted by existing measures.
How is Incremental Validity assessed?
It is assessed using regression analysis to evaluate the contribution of the new test while controlling for existing predictors.
What is Decision Theory in validity?
In decision theory, validity focuses on the consequences of decisions made based on test results, such as the accuracy of diagnosing a condition or hiring someone for a job.
What are False Positive and False Negative?
False Positive (Type I error): Incorrectly deciding a person has a condition when they do not.
False Negative (Type II error): Incorrectly deciding a person does not have a condition when they do.
What is the Multitrait-Multimethod Matrix?
It is a tool used to assess convergent and discriminant validity by showing the correlations between multiple traits (constructs) measured by multiple methods.
What is Standard Error of Estimate (SEE)?
It measures the accuracy of predictions made by a regression equation, quantifying the average distance between predicted and observed values.
