reliablity & validity ppt notes Flashcards
Variables
A category or type of change that can occur among objects, events or situations
Scales of measurement
1.Nominal
Categories or names
2.Ordinal
Categories or names
Rank-order data
3.Internal
Categories or names
Rank-order data
Equal intervals
4.Ratio Categories or names Rank-order data Equal intervals Absolute zero point
Factors distinguishing C&E research
- Third variable
- Control
- Cause & effect
- Artificiality of setting
- Subject variables
- Ethics
- Sometimes prediction is good enough
- Temporal precedent: if one is the result of another
Types of relationships
- Positive
- Negative
- Curvilinear
- Zero
Reliability
- Classic true score model
- Types of reliability
- Considerations regarding the nature of the test itself
Validity
- defined
- validation strategies
reliability - classic score model
- The reliability coefficient expresses the ratio between the true score & the total variance
- X=T+E
- True score is the hypothetical average of all the observed test scores that would be obtained were an individual to take the test an infinite number of times
1. reliability - consistency
2. Validity- accuracy - Rxx
Reliability- types
1.Test- retest reliability
Coeffifincet of stability- if time interval is 6 months or more
2.Alternate or parallel forms
Coefficient of equivalence
3.split - half/ internal consistency
4.Inter-rater reliability/ inter-judge/ interscorer
Error types
- test development
- Test admin
- test scoring
Validity- defined
- The degree to which a test measures what it purports to measure
- The degree to which inferences from test scores are appropriate
- Validation (messick 1989)- the scientific inquiry into test score meaning
Validity- types
- Face: does not require any systematic research, informal / casual, comes from test takers perspective
- Content:
- Criterion- related validity:
a. Predictive
b. Concurrent
c. Expenctany tables - Construct: most comprehensive, only one not based on one study
Content validity
-How adequately does a test sample behavior representative of the universe of -behavior the test was designed to sample
-Content validity ratio ne-N/2/ N/2
–N=number of experts/ panelists
ne= number of panelists suggesting this is an essential item
–If > ½ say essential than it has some content validity
Criterion- related validity
- How adequate is a test score for making inferences about a persons standing on some criterion measure
- Criterion- outcome of interest- measure that can be used to determine the accuracy of decisions- can be just about anything.
2 types- predictive & concurrent
- Difference lies in time delay
- advantages/ disadvantages
Factors influencing validity
- Restriction of range
- Reliability sets the upper limit on validity
- Limited to linear relationships
Construct validity
How appropriate are the inferences drawn from test scores regarding individuals standings on a variable called a construct
Evidence for construct validity
- Evidence from criterion- related studies
- Evidence from content- related studies
- Convergent & discriminant validity studies
- Homogeneity evidence
- Evidence from different groups- the method of contrasted groups
- Changes with age
- pre/post changes
