Scale Development Flashcards
Ideally, the variance in a response to a question will be due to the underlying construct we’re measuring. However, it will also be caused by…
Error
What is a reliable scale?
Consistency of the scale in measuring a variable
Split-half reliability
Split into two
Two halves should correlate strongly
Which is the best measure of reliability?
Internal reliability
Split-half reliability
If the scale is reliable, the two halves should…
Correlate strongly
Why is internal reliability a very accurate measure?
Averages over many correlations
The coefficient alpha indicates…
Proportion of variance attributable to TRUE score
DeVellis recommends that a coefficient alpha of <0.6 is
Unacceptable
DeVellis recommends that a coefficient alpha of >.9 indicates
Probably should shorten the scale
3 ways of testing for reliability
Internal reliability
Split-half
Test-retest reliability
How is Cronbach’s alpha limited?
It is not a measure of unidimensionality
What is test-retest reliability?
Scores at one point in time should be correlated with scores at another point in time
A high test-retest reliability means there is a
Strong correlation between two administrations
Low test-retest reliability might reflect ________ rather than unreliability
Temporal instability
Low test-retest reliability does not always indicate unreliability…
May indicate temporal instability
What is validity?
Extent to which scores from scale reflect variable
Scale validity can be increased by (2)
a) Interviews with target sample to generate Qs
b) Ask experts to rate Qs
What is construct validity?
Does the scale correlate to other constructs in line with theoretical expectations?
How to assess construct validity (1)
Administer 2 similar scales
Should correlate
3 kinds of validity
Content
Construct
Predictive
Predictive validity
Can the scale accurately predict a future criterion?
E.g. can an intelligence scale predict University-goers 10 years later?
Do we aim for or avoid kurtosis?
Avoid
How to work out skewness?
Z = S (skewness) / SE
If Z is LARGER than 3.08, this indicates
Highly significant levels of skewness
If Z is larger than _____, this indicates significant levels of skewness/kurtosis
3.08
If __ is larger than 3.08, this indicates significant levels of skewness/kurtosis
Z
Is a highly significant level of skewness/kurtosis a good thing?
No
Indicates too much deviation from normal distribution
If Z = 3.08, P =
< 0.01
If the Z score is greater than 3.08, then the item should be…
Removed from the scale
Reliability can be assessed using C….
Cronbach’s Alpha
Cronbach’s Alpha can assess reliability
Look out for…
Low correlations
Cronbach’s Alpha
SPSS indicates the ________ if the item was to be _________!
Alpha coefficient
Deleted
If data is skewed, the peak will..
Be shifted away from centre of distribution
Skewness is a measure of ________, or more precisely, __________
Symmetry
Lack of symmetry
(Histogram) A distribution/data set is symmetric if it …
Looks the same left to right
Data sets with high kurtosis tend to have heavy…
Tails/outliers
The ________ is an effective graphical technique for showing both the skewness and kurtosis of a dataset
Histogram
The kurtosis for standard normal distribution is
3
This measure is a measure of outliers…
Kurtosis
This measure is a measure of normal/non-normal distribution
Skewness
Skewness tells us the amount/direction of
Skew
Which of the following decreases the probability of a type I error?
a) Increasing the SD
b) Increasing the sample size
c) Decreasing the significance level
Decreasing the significance level
There are three types of scale reliability:
Test-retest
Internal consistency
Inter-rater
Which type of scale reliability refers to over-time reliability?
Test-retest
Which type of scale reliability refers to across items reliability?
Internal consistency
Which type of scale reliability refers to across researchers reliability?
Inter-rater
Assessing test-retest reliability requires using the measure…
- On a group of people
- Readministering the scale at a later date
- The scores should correlate
“The consistency of responses across a multi-item measure”
Internal consistency
(Internal consistency)
If people’s responses to the different items are not correlated with each other, then they may not
Be measuring the same underlying construct
How might internal consistency be assessed?
Split-half consistency
How might internal consistency be assessed statistically?
Cronbach’s Alpha
Cronbach’s Alpha
A value of ___ plus is generally taken to show good internal consistency
.80
Cronbach’s Alpha is a statistical measure of
Internal consistency
Validity is the extent to which scores from a measure
Represent the variable they’re meant to
Content Validity can also be assessed by comparing the scale against the ___________ of the variable
Conceptual definition
Skewness and Kurtosis are both worked out by
S or K divided by S SE or K SE
3 ways of checking for normality of scales
- Skewness
- Kurtosis
- Histogram
The X item was found to be significantly skewed (Z = x, p = x)
And was therefore dropped from subsequent analyses
___________ measures internal consistency
Coefficient Alpha
Coefficient alpha measures
Internal consistency
To what extent does a scale of depression look like its measuring depression?
Content validity
How to work out SE from S/K + Z?
S/K DIVIDED BY Z
