Developing typical Performance Tests

Question 1

How does a response style vary?

Answer 1

Between people but not much between tests; it is person specific

Question 2

Name the four main response styles

Answer 2

Acquiescence is the tendency to agree with the endorsement statement
Dissentience is the tendency to disagree with an endorsement statement
Extreme response style is the tendency to choose extremes of the item response scale
Midpoint response style is the tendency to choose the middle of the response scale

Question 3

What is meant by the response set?

Answer 3

The differential use of the item response scale by different persons and with different constructs

Question 4

How does the response set vary?

Answer 4

May differ between persons and between constructs but stays relatively stable across a person/ construct-specific property; person/ construct specific property

Question 5

What do typical performance tests asses?

Answer 5

Behaviour that is typical of a person

Question 6

What are the three main types of typical performance tests?

Answer 6

1. Personality tests
2. interest inventories
3. Attitude questionnaires

Question 7

What are the three classes of strategies for the conceptual framework?

Answer 7

Intuitive class (no/ informal knowledge)
Inductive class (weak theory/ knowledge)
Deductive class (strong theory/ knowledge)

Question 8

What methods are involved in the intuitive class?

Answer 8

1. Rational method- using all knowledge there is to find about a construct
2. Prototypical method- items based on a prototype of a behaviour/ construct

Question 9

What methods are involved in the inductive class?

Answer 9

1. Internal method- Items seem related to the construct are gathered and administered. Highly correlated items represent a construct.
2. External method- Different items are selected and administered. Items that correlate highly with a criterion are selected.

Question 10

What methods are involved in the deductive class?

Answer 10

1. Construct method- construction on basis of a strong theory about the construct and its relation to other constructs
2. Facet method- Conceptual analysis of the construct and every aspect (facet) of the construct is measured systematically

Question 11

What item writing guidelines are there for typical performance tests? (11)

Answer 11

1. Elicit different answers at different construct positions- Test takers who have completely different construct positions should give different answers to the item
2. Focus on one aspect per item
3. Avoid making assumptions about the test makers
4. Use correct language
5. Use clear and comprehensive wording
6. Use non-sensitive language and content
7. Put the situational or conditional part of a statement at the beginning and behavioural at the end
8. Use positive statements
9. Use 5-7 categories in ordinal polygamous response scales
10. label each of the categories in the response scale and avoid the use of numbers alone
11. Format response categories vertically

Question 12

What is meant by an indicative item?

Answer 12

An item where a high frequency or endorsement indicates a high level of the construct

Question 13

What three different types of research methods can be used toga information on the construct?

Answer 13

Focus groups (experienced people of the topic, eg patients)
Key informant )
Key informant method (researchers meet with experts)
Observation method- observing eg patients

Question 14

What different measurement modes are there in typical performance tests?

Answer 14

Self report
Other report
Somatic indicator mode
Physical trace- eg driving record for recklessness

Question 15

What is meant by reactive and non reactive measurement procedures?

Answer 15

It is reactive when test makers can deliberately distort their score, it is nonreactive when test takers cannot distort their construct value

Question 16

What are meant by projective tests?

Answer 16

Tests which present the test taker with ambiguous stimuli, and the test taker is asked to react to this stimuli

Question 17

What is meant by a frequency or endorsement scale

Answer 17

Frequency- how frequently you do something

Endorsement- how much you endorse (agree with something)

Question 18

What is meant by a contra-indicative item?

Answer 18

A contraindicative item is an item where a high frequency or endorsement indicates a low level of the construct.

Question 19

What are meant by response tendencies?

Answer 19

Response tendencies are the differential application of the response scales

Question 20

Name three response sets

Answer 20

Social desirability
Self-deception
Impression management

Question 21

What is meant by a visual analogue scale

Answer 21

A continuous line response scale

Question 22

What two types of oral administration are there?

Answer 22

face to face and telephone

Question 23

What two types of pencil and paper administration are there?

Answer 23

Personal and mail

Question 24

What guidelines should open ended questions follow?

Answer 24

Same as maximum performance tests

Question 25

What three types of pilot studies are described?

Answer 25

1) Experts
2) Test takers
3) Raters pilot studies

Question 26

What kindve experts are used in expert pilots

Answer 26

Experts for concept questions, Experts for technical matters of the test and sensitivity experts for minority groups

Question 27

What is meant by concurrent and retrospective test takers

Answer 27

Concurrent describes their thinking while doing the test and retrospective are asked after they do the test

Question 28

What is Cohen’s Kappa used for?

Answer 28

A measure of the degree of perfect agreement or consistency

Question 29

What does Cohen’s Kappa fail to take into account, what helps this?

Answer 29

A measure of the degree of disagreement or inconsistency. Cohen’s weighted coefficient Kappa takes this into account

Question 30

How can social desirability be detected?

Answer 30

By putting unrelated questions measuring social desirability