Midterm 2 - Scale Development/Evaluation (Boateng et al.) Flashcards
What are the 4 main test construction strategies?
- correspondence
- projective
- empirical (criterion-group)
- construct (rational/theoretical)
Which test construction strategy is mostly used for questionnaires and was dominant in the 20th century?
correspondence strategy
What is the correspondence test construction strategy? What are the 4 assumptions? What are the problems with it?
- intuitive and atheoretical
- assumes each item corresponds to a characteristic of interest
- assumes items have common meaning for all test-takers/examiner
- assumes test-taker can accurately assess requested info
- assumes test-taker will be honest
- issue: hard to meet last 2 assumptions
What is the empirical strategy for test construction?
- items selected on basis of relations to external criteria (eg contrasted groups)
- meaning of an item is not equal to verbal content of item
- interpretation of scores by “cookbook”: empirically known correlates of high and low scores
The empirical strategy emerged in response to the understanding that ____
people aren’t always honest when they answer questionnaires!
Is face-validity a concern with the empirical test construction strategy?
NO! (verbal content of items not equal to meaning)
What is a popular example of the empirical test construction strategy?
- MMPI
- items can differentiate btw diff psycopathology groups
the Cambridge analytica drama was a result of which type of test construction/interpretation strategy?
empirical strategy
What are the 4 primary concerns of the empirical test construction strategy?
- are indivs in contrasting groups diff as intended (is criterion used to create groups valid–need to make sure the groups don’t share characteristics)
- unintended group differences (within group diffs)
- problem of generalization (items may function differently in diff contexts)
- item overlap (eg items flag either depression or anxiety so know something is up but don’t know specifics)
What is the construct (rational/theoretical) strategy for test construction? What are the 3 assumptions?
- emphasizes theoretical definitions of domains to be measured
- each item is a referent for a construct
- incorporates stats methods for reliability/validity
- think nomologic net
- assumes a person possesses some degree of a construct
- assumes nontest behaviours can be identified which are referents for the construct
- assumes test responses are referents for the construct
The construct (rational/theoretical) strategy for test construction originated in the ____s and was prominent from ____ to ____
1950s; prominent from 60s to now!
What are the 9 typical steps in construct (rational/theoretical) test construction approach? Which of the 3 phases is each a part of?
Item Development (2 steps)
Scale Development (4 steps)
Scale Evaluation (3 steps)
- identification of domain and item generation (item dev)
- content validity (item dev)
- pre-testing of questions (scale dev)
- sampling and survey admin (scale dev)
- item reduction (scale dev)
- extraction of factors (scale dev)
- tests of dimensionality (scale eval)
- tests of reliability (scale eval)
- tests of validity (scale eval)
What was the goal of Boateng’s article on best practices for scale development?
- provide amalgamation of technical literature and lessons learned through experience
- essentially a primer for scientists and practitionners
What are the 3 phases of scale development according to Boateng et al?
- item development
- scale development
- scale evaluation
What is step 1 of test construction according to Boateng et al?
- identification of domain and item generation
- part of item development phase
- can use deductive or inductive method to generate items
- need to specify purpose of domain and confirm there are no existing instruments
- need to specify boundaries of domain and identify appropriate questions
- specify and define dimensions of domain
