Introduction

Question 1

How do maximum and typical performance tests differ?

Answer 1

Maximum- you have to solve a problem, answer is (partly) correct or incorrect
Typical- Respond to a task in a way that is typical of you, no right or wrong answers

Question 2

Name the seven steps in your test construction process

Answer 2

1) The construct of interest
2) The measurement mode
3) The objective
4) The population
5) The conceptual framework
6) The response mode
7) The administration mode

Question 3

What are ability tests?

Answer 3

measuring a skill which is not explicitly taught or trained

Question 4

What are achievement tests

Answer 4

measuring a skill which is explicitly taught or trained

Question 5

Give three modes of the measurement modes of maximum performance tests

Answer 5

Self-performance mode- subject completes the test themselves (most common)
Self-evaluation mode- E.g do you know the answer to this problem
Other-evaluation mode- E.g nurse determining cognitive ability of a patient who cannot themselves

Question 6

Give two examples of different objectives in the construction process of maximum performance tests

Answer 6

Scientific (E.g for a given population) or Practical (job application of patient)
Individual level or group level

Question 7

What is meant by the conceptual framework?

Answer 7

A framework in which the items (questions/ knowledge required) are contained

Question 8

How is a conceptual framework obtained in achievement tests?

Answer 8

By consulting the learning objectives (regression analysis/ knowledge)

Question 9

How is a conceptual framework obtained in ability tests?

Answer 9

Decided by the theory used on the ability (different theories for intelligence: fluid and crystallised etc)

Question 10

What two response modes are often utilised in maximum performance tests?

Answer 10

Free response mode- open questions

Choice response mode- multiple choice

Question 11

Give three administration modes in maximum performance tests

Answer 11

Paper and pencil administration
• Computerized administration
— All subjects are administered all items
• Computerized adaptive administration
— The computer sequentially selects the next item for a given subject (difficulty)

Question 12

Name the book by Freud and by Watson

What mistake was made here?

Answer 12

Asking two questions in one

Question 13

What is a more reasonable alternative to using dependent item content in questions?

Answer 13

Testlets- multiple items that have the same context- ie answering from a reading piece

Or just using independent item content

Question 14

There are 4 children among which you have to distribute 10 pieces of gummy bears. How many does each child get?

a. 2
b. 2.5
c. 3
d. 3.5

What is wrong with this question?

Answer 14

Trick question distracts from the arithmetic question and turns it into a different question; don’t use trick questions

Question 15

How many options should usually be utilised in choice response mode?

Answer 15

Use three options, unless it is easy to write plausible distractors

Question 16

Which of the below is not an example of an observational study?
a. A researcher tests for differences between males and females on
working memory
b. A researcher studies the effect of income on happiness
c. A researcher manipulates stress and look at the effect on working
memory

What is wrong with this question

Answer 16

Question is worded negatively rather than positively

Question 17

Name two qualities that should exist in the correct answers?

Answer 17

Should avoid giving clues or making the correct answer longer, Should make one answer unambiguously correct

Question 18

Name three typical performance tests

Answer 18

Personality tests- measuring a personality characteristic
Interest inventories- interest in a topic
Attitude questionnaires- opinion

Question 19

Give four measurement modes of typical performance tests

Answer 19

Self report
Other report (eg- observation)
Somatic mode (eg physiological)
Physical traces (traces left behind ie diaries)

Question 20

What three broad methods are often utilised in conceptual frameworks of typical performance tests?

Answer 20

Intuitive method- No/informal theory/knowledge about construct
Inductive method- Weak theory/knowledge about construct
Deductive method- • Strong theory/knowledge about construct

Question 21

Name and describe two methods utilised within the intuitive methods

Answer 21

Rational method- Use everything you can find about the construct e.g., (expert) opinions, informal sources, etc
– E.g., measuring love
• Wikipedia (informal ‘theory’): “Love is an emotion of strong
affection and personal attachment.”
• Item: Do you feel affection for ___?
• Item: Are you personally attached to ____?
Prototypical method- Test is constructed based on a prototype
• Imagine to have a high position on the construct
• Report all behavior, thoughts, emotions
ie How they feel when they’re in love- butterflies in stomach etc, construct question- ‘when is the last time you felt butterflies in your stomach’ etc

Question 22

What methods are often utilised in the inductive methods, how are these utilised?

Answer 22

Internal method
-gather items that seem related to the construct of interest
-administer and select the most homogenous items (validity is doubtful) eg- insecurity-extraversion, openness to experience
External method
-Select different items
-Administer, and select the items that correlate
highly with a criterion

Question 23

Describe the two methods used in deductive methods

Answer 23

Construct method
-Construction on basis of a strong theory bout the construct and its relation to other constructs (love example)
Facet method
– Item construction by conceptual analysis of the
construct (using existing theory/knowledge)
– Every aspect (facet) of the construct is measured
in a systematic way
eg • Two facets:
• Process (cognitive – emotional – behavior)
• Situation (home – close to him/her – with others)

Question 24

What two types of close ended questions are given?

Answer 24

Frequency and endorsement

Question 25

How can endorsement questions be further broken down

Answer 25

All or none and intensity

Question 26

How may intensity questions differ

Answer 26

discrete or continuous

Question 27

What two names are given to question types which allude to the amount of response options?

Answer 27

ordinal-polytomous response - multiple options

Dichotomous response- two options

Question 28

What name is given to continuous response options?

Answer 28

Bounded continuous

Question 29

What is wrong with this question in a general questionairre

“How long do you spend on your schoolwork per week?”

Answer 29

Do not make assumptions

Question 30

“I giggle a lot in awkward situations”

What is wrong with this item

Answer 30

Put the situation first before the condition

Question 31

What is meant by contraindicative items

Answer 31

items of which a high score is indicative of a low level on the construct
eg At parties, I entertain everyone
At parties I often feel uncomfortable

Question 32

Why or why not use contraindicative items?

Answer 32

Do, to encourage people paying attention and prevent confounding results through people giving false results

Question 33

What are the three roots of modern testing?

Answer 33

Civil service examinations, educational achievement and the study of individual differences of behaviour

Question 34

What is seen as the first modern test?

Answer 34

1905- intelligence test in france to distinguish between normal and retarded children

Question 35

What does it mean that a test measures latent attributes?

Answer 35

test performance may be observable but the latent attributes underlying these operalisations

Question 36

How are tests distinguished from surveys?

Answer 36

It is not assumed that surveys measure a latent attribute however these can be used to form a measurement index

Question 37

What links a subtest and an item?

Answer 37

A subtest is an independent part of a test while an item is the smallest possible subtest of a test

Question 38

What is meant by dimensionality?

Answer 38

The number of latent attributes (variables) which affects test performance

Question 39

How may a maximum performance test be further subdivided in regards to how the test is conducted?

Answer 39

Accuracy (power) or speed tests;

Question 40

How may accuracy tests be presented?

Answer 40

a pure power test would contain questions the person has to solve with an excess of time, This is usually hard to carry out in practice and so in time-limited power test, where the majority will have enough time and the minority will need more is often utilised