3 (1) ASSESSMENTS: Psychometric properties Flashcards
3 psychometric properties we got to assess?
RELIABILITY➖ how stable and consistent the tool is.
〰〰〰〰〰〰〰〰〰〰〰
VALIDITY➖ does the tool measure what it set out to measure?
〰〰〰〰〰〰〰〰〰〰〰
RESPONSIVENESS➖ tool’s sensitivity to change. If a change occurs, will it pick it up?
List the type of RELIABILITY
🔸internal consistency
🔸test-retest
🔸parallel forma and split half
🔸inter-rater
What is internal consistency?
How can all items be consistent?
The scale doesn’t contain any contradictions. It measures whether several items that propose to measure the same general construct produce similar scores.
🔸 all items in scale should measure same concept
🔸 all items of a particular domain only tap into this domain and no other.
How do we measure internal consistency?
CRONBACH's ALPHA ➖ looks to measure the average correlations among the items and the number of items in the scale. Gives coefficient alpha (α). 〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰 Scale of 0 to +/-1 🔸 0 = perfect correlation 🔸 -/+ 1 = no correlation 〰〰〰〰〰〰〰〰〰〰〰〰〰〰〰 Groups - α>0.70 Individual- α>0.90
What’s test-retest reliability?
Same test is given to respondents on two separate occasion. Tests to see whether results are consistent overtime.
Time 1 & time 2 should be separated by 2 to 14 days.
Correlation between T1 + T2 is measured by
🔸Pearson’s r or
🔸Intraclass correlation coefficient (ICC)
ICC
🔹group testing - 0.75+
🔹individuals- 0.85+
Parallel forms and split-half reliability
Parallel – Alternative test is done at T2
Split-half - Half of the test is done at T1 and other half is done at T2. These are then correlated.
What’s interrater reliability?
Refers to the degree of consensus amongst raters; the degree to which judges agree
Uses ICC and Pearson’s r
🔸must be >0.80
What’s external validity?
Refers to the extent to which the results of the study can be generalised to the population.
What is internal validity?
🔸list the types
Refers to the way the study was conducted (such as the design, measures) and the extent to which experimenters take into account the alternative reasons for causal relationships.
🔹face validity
🔹content validity
🔹criterion related validity
🔹construct validity
What is face validity?
Judging the study on the face of it.
🔸Does the study seem like a reasonable way to gain information the researcher has attempted to obtain?
🔸Does it seem well designed?
🔸Does it seem as though it works reliably?
🔸Is the study peer reviewed by experts?
🔸 Doesn’t depend on established theories for support.
What is content validity?
It refers to whether the study/measure takes into account all areas of the concept to be measured.
What is criterion related validity (instrumental validity)?
Its use to demonstrate the accuracy of the measure or procedure by comparing it with another measure or procedure which is been demonstrated to be valid a.k.a. comparing it to the gold standard.
🔸Concurrent validity ➖ The measure you’re looking at correlates with currently available data such as already existing measures.
🔸Predictive validity ➖ the measurement you’re looking at correlates with future outcomes such as predicting academic achievement .
What is construct validity?
Does the instrument measure the concept or not?
What’s responsiveness?
The assessments ability to detect change between T1 and T2.
T-stat➖measures if change is significant.
‼️Effect size, d
🔸SRM - this relates to the change in mean scores over time to the standard deviation of change score.
What is effect size, d?
🔸 how do you calculate it?
🔸 absolute values of d and SRM?
The effects size relate to the change in mean scores over time (T1-T2) to the standard deviation of T1.
(T1-T2)➗SD @ T1
〰〰〰〰〰〰〰
Small 0.20
Moderate 0.50
Large 0.80
