Introduction Flashcards
How do maximum and typical performance tests differ?
Maximum- you have to solve a problem, answer is (partly) correct or incorrect
Typical- Respond to a task in a way that is typical of you, no right or wrong answers
Name the seven steps in your test construction process
1) The construct of interest
2) The measurement mode
3) The objective
4) The population
5) The conceptual framework
6) The response mode
7) The administration mode
What are ability tests?
measuring a skill which is not explicitly taught or trained
What are achievement tests
measuring a skill which is explicitly taught or trained
Give three modes of the measurement modes of maximum performance tests
Self-performance mode- subject completes the test themselves (most common)
Self-evaluation mode- E.g do you know the answer to this problem
Other-evaluation mode- E.g nurse determining cognitive ability of a patient who cannot themselves
Give two examples of different objectives in the construction process of maximum performance tests
Scientific (E.g for a given population) or Practical (job application of patient)
Individual level or group level
What is meant by the conceptual framework?
A framework in which the items (questions/ knowledge required) are contained
How is a conceptual framework obtained in achievement tests?
By consulting the learning objectives (regression analysis/ knowledge)
How is a conceptual framework obtained in ability tests?
Decided by the theory used on the ability (different theories for intelligence: fluid and crystallised etc)
What two response modes are often utilised in maximum performance tests?
Free response mode- open questions
Choice response mode- multiple choice
Give three administration modes in maximum performance tests
Paper and pencil administration
• Computerized administration
— All subjects are administered all items
• Computerized adaptive administration
— The computer sequentially selects the next item for a given subject (difficulty)
Name the book by Freud and by Watson
What mistake was made here?
Asking two questions in one
What is a more reasonable alternative to using dependent item content in questions?
Testlets- multiple items that have the same context- ie answering from a reading piece
Or just using independent item content
There are 4 children among which you have to distribute 10 pieces of gummy bears. How many does each child get?
a. 2
b. 2.5
c. 3
d. 3.5
What is wrong with this question?
Trick question distracts from the arithmetic question and turns it into a different question; don’t use trick questions
How many options should usually be utilised in choice response mode?
Use three options, unless it is easy to write plausible distractors
Which of the below is not an example of an observational study?
a. A researcher tests for differences between males and females on
working memory
b. A researcher studies the effect of income on happiness
c. A researcher manipulates stress and look at the effect on working
memory
What is wrong with this question
Question is worded negatively rather than positively
Name two qualities that should exist in the correct answers?
Should avoid giving clues or making the correct answer longer, Should make one answer unambiguously correct
Name three typical performance tests
Personality tests- measuring a personality characteristic
Interest inventories- interest in a topic
Attitude questionnaires- opinion
Give four measurement modes of typical performance tests
Self report Other report (eg- observation) Somatic mode (eg physiological) Physical traces (traces left behind ie diaries)
What three broad methods are often utilised in conceptual frameworks of typical performance tests?
Intuitive method- No/informal theory/knowledge about construct
Inductive method- Weak theory/knowledge about construct
Deductive method- • Strong theory/knowledge about construct
Name and describe two methods utilised within the intuitive methods
Rational method- Use everything you can find about the construct e.g., (expert) opinions, informal sources, etc
– E.g., measuring love
• Wikipedia (informal ‘theory’): “Love is an emotion of strong
affection and personal attachment.”
• Item: Do you feel affection for ___?
• Item: Are you personally attached to ____?
Prototypical method- Test is constructed based on a prototype
• Imagine to have a high position on the construct
• Report all behavior, thoughts, emotions
ie How they feel when they’re in love- butterflies in stomach etc, construct question- ‘when is the last time you felt butterflies in your stomach’ etc
What methods are often utilised in the inductive methods, how are these utilised?
Internal method
-gather items that seem related to the construct of interest
-administer and select the most homogenous items (validity is doubtful) eg- insecurity-extraversion, openness to experience
External method
-Select different items
-Administer, and select the items that correlate
highly with a criterion
Describe the two methods used in deductive methods
Construct method
-Construction on basis of a strong theory bout the construct and its relation to other constructs (love example)
Facet method
– Item construction by conceptual analysis of the
construct (using existing theory/knowledge)
– Every aspect (facet) of the construct is measured
in a systematic way
eg • Two facets:
• Process (cognitive – emotional – behavior)
• Situation (home – close to him/her – with others)
What two types of close ended questions are given?
Frequency and endorsement
How can endorsement questions be further broken down
All or none and intensity
How may intensity questions differ
discrete or continuous
What two names are given to question types which allude to the amount of response options?
ordinal-polytomous response - multiple options
Dichotomous response- two options
What name is given to continuous response options?
Bounded continuous
What is wrong with this question in a general questionairre
“How long do you spend on your schoolwork per week?”
Do not make assumptions
“I giggle a lot in awkward situations”
What is wrong with this item
Put the situation first before the condition
What is meant by contraindicative items
items of which a high score is indicative of a low level on the construct
eg At parties, I entertain everyone
At parties I often feel uncomfortable
Why or why not use contraindicative items?
Do, to encourage people paying attention and prevent confounding results through people giving false results
What are the three roots of modern testing?
Civil service examinations, educational achievement and the study of individual differences of behaviour
What is seen as the first modern test?
1905- intelligence test in france to distinguish between normal and retarded children
What does it mean that a test measures latent attributes?
test performance may be observable but the latent attributes underlying these operalisations
How are tests distinguished from surveys?
It is not assumed that surveys measure a latent attribute however these can be used to form a measurement index
What links a subtest and an item?
A subtest is an independent part of a test while an item is the smallest possible subtest of a test
What is meant by dimensionality?
The number of latent attributes (variables) which affects test performance
How may a maximum performance test be further subdivided in regards to how the test is conducted?
Accuracy (power) or speed tests;
How may accuracy tests be presented?
a pure power test would contain questions the person has to solve with an excess of time, This is usually hard to carry out in practice and so in time-limited power test, where the majority will have enough time and the minority will need more is often utilised
