Scale development

Question 1

difference between reliability and validity of scales

Answer 1

reliability is a measure of consistency (in terms of a construct measuring the same thing in a sample), due to a latent variable exerting an effect on the variation of scores on a scale
while validity is concerns whether the latent variable of interest is actually being measured

Question 2

how could you assess the reliability of a scale

Answer 2

split half
internal consistency
test re test

Question 3

how could you assess the validity of a scale

Answer 3

content
construct
predictive

Question 4

guidelines for scale development

Answer 4

De vellis:

1. define the latent variable
2. generate an item pool
3. review the items for content
4. administer the items to a development sample
5. evaluate the items (reverse code, mean and SD, skewness and kurtosis)
6. compute coefficient alpha
7. validate the scale - correlate it with other constructs / does it predict anything

Question 5

kurtosis

Answer 5

spikey/ peak-ness of the distribution

Question 6

skewedness

Answer 6

postively (left) i.e. lots of low scores

negtively (right) i.e. lots of high scores

Question 7

if z > 3.08

Answer 7

equates to p

Question 8

a reliable measure should have

Answer 8

inter-correlations between its items (caused by the same latent variable)
because variance in the items are hopefully due to the underlying construct

Question 9

split half reliability

Answer 9

when scores on the first half of a scale (e.g. sensation seeking) are correlated with scores on the second half
e.g. where scores on the odd items are correlated with scores on the even items

Question 10

internal consistency

Answer 10

gives a much better idea of consistency than split half - why ??

alpha coeffcient indicates the proportion of the variance in the scale scores that is attributable to the true score

• based on correlations between each scale item score and the total score
• provides a more accurate measure of internal reliability than a split half correlation!

Question 11

alpha above .90

Answer 11

can shorten the scale

redundancy

Question 12

caveats of alpha

Answer 12

dependent on

1) magnitude of correlations among items
2) number of items

it’s not a measure of dimensionality or uni dimensionality
(i.e. a high alpha may not indicate a single underlying construct).

Question 13

test re test

Answer 13

scores on at one time point correlate with rest at another time point

however lower test re test reflect poor temporal stablility in the underlying construct (e.g. personality might not change but stress might)

Question 14

content validity

Answer 14

do the content of the items reflect the variable of interest
can judge this through interviews of members of a target population to devise items (pilot study)
or ask experts

Question 15

construct validity

Answer 15

does the scale relate to another construct (based on theoretical expectation) e.g. anxiety and depression

or does it correlate with an established measure of the same construct

Question 16

delphi method

Answer 16

panel of experts to comment soon or rate items to establish whether its measuring what you want it to

The Delphi method is a forecasting process framework based on the results of multiple rounds of questionnaires sent to a panel of experts. Several rounds of questionnaires are sent out to the group of experts, and the anonymous responses are aggregated and shared with the group after each round.

Question 17

predictive validity

Answer 17

‘external criterion related validity’

an association with some other factor
can it predict an outcome variable

e.g. a measure of medication belief would be expected to correlate with an objective measure of medication adherence

Question 18

skewness and kurtosis test

Answer 18

a standardised z score

Question 19

z =

Answer 19

K or S/ SE of K or S