Developing typical Performance Tests Flashcards
How does a response style vary?
Between people but not much between tests; it is person specific
Name the four main response styles
Acquiescence is the tendency to agree with the endorsement statement
Dissentience is the tendency to disagree with an endorsement statement
Extreme response style is the tendency to choose extremes of the item response scale
Midpoint response style is the tendency to choose the middle of the response scale
What is meant by the response set?
The differential use of the item response scale by different persons and with different constructs
How does the response set vary?
May differ between persons and between constructs but stays relatively stable across a person/ construct-specific property; person/ construct specific property
What do typical performance tests asses?
Behaviour that is typical of a person
What are the three main types of typical performance tests?
- Personality tests
- interest inventories
- Attitude questionnaires
What are the three classes of strategies for the conceptual framework?
Intuitive class (no/ informal knowledge) Inductive class (weak theory/ knowledge) Deductive class (strong theory/ knowledge)
What methods are involved in the intuitive class?
- Rational method- using all knowledge there is to find about a construct
- Prototypical method- items based on a prototype of a behaviour/ construct
What methods are involved in the inductive class?
- Internal method- Items seem related to the construct are gathered and administered. Highly correlated items represent a construct.
- External method- Different items are selected and administered. Items that correlate highly with a criterion are selected.
What methods are involved in the deductive class?
- Construct method- construction on basis of a strong theory about the construct and its relation to other constructs
- Facet method- Conceptual analysis of the construct and every aspect (facet) of the construct is measured systematically
What item writing guidelines are there for typical performance tests? (11)
- Elicit different answers at different construct positions- Test takers who have completely different construct positions should give different answers to the item
- Focus on one aspect per item
- Avoid making assumptions about the test makers
- Use correct language
- Use clear and comprehensive wording
- Use non-sensitive language and content
- Put the situational or conditional part of a statement at the beginning and behavioural at the end
- Use positive statements
- Use 5-7 categories in ordinal polygamous response scales
- label each of the categories in the response scale and avoid the use of numbers alone
- Format response categories vertically
What is meant by an indicative item?
An item where a high frequency or endorsement indicates a high level of the construct
What three different types of research methods can be used toga information on the construct?
Focus groups (experienced people of the topic, eg patients)
Key informant )
Key informant method (researchers meet with experts)
Observation method- observing eg patients
What different measurement modes are there in typical performance tests?
Self report
Other report
Somatic indicator mode
Physical trace- eg driving record for recklessness
What is meant by reactive and non reactive measurement procedures?
It is reactive when test makers can deliberately distort their score, it is nonreactive when test takers cannot distort their construct value
What are meant by projective tests?
Tests which present the test taker with ambiguous stimuli, and the test taker is asked to react to this stimuli
What is meant by a frequency or endorsement scale
Frequency- how frequently you do something
Endorsement- how much you endorse (agree with something)
What is meant by a contra-indicative item?
A contraindicative item is an item where a high frequency or endorsement indicates a low level of the construct.
What are meant by response tendencies?
Response tendencies are the differential application of the response scales
Name three response sets
Social desirability
Self-deception
Impression management
What is meant by a visual analogue scale
A continuous line response scale
What two types of oral administration are there?
face to face and telephone
What two types of pencil and paper administration are there?
Personal and mail
What guidelines should open ended questions follow?
Same as maximum performance tests
What three types of pilot studies are described?
1) Experts
2) Test takers
3) Raters pilot studies
What kindve experts are used in expert pilots
Experts for concept questions, Experts for technical matters of the test and sensitivity experts for minority groups
What is meant by concurrent and retrospective test takers
Concurrent describes their thinking while doing the test and retrospective are asked after they do the test
What is Cohen’s Kappa used for?
A measure of the degree of perfect agreement or consistency
What does Cohen’s Kappa fail to take into account, what helps this?
A measure of the degree of disagreement or inconsistency. Cohen’s weighted coefficient Kappa takes this into account
How can social desirability be detected?
By putting unrelated questions measuring social desirability
