Lecture 3 - Test Development 1: part2 - Construction (DN) Flashcards
What is the difference between scaling & measurement?
- Measurement: the assigning of numbers to some attribute
- Scaling: the method (mechanism) by which we assign numbers (rule setting)
What types of rules guide measurement?
-
Nominal rule
- classifies only
-
Ordinal rule
- slightly more sophisticated
- it is ordered
- train example - order says nothing about distances between stations
- has both nominal & ordinal properties
-
Interval rule
- has nominal & ordinal properties
- equal distance between
- no absolute zero
- zero is just arbitrary
- e.g., celsius (zero = freezing, but not the absence of tempretaure)
-
Ratio rule
- true zero point
- not a lot in psych testing
57:00
Which rule would apply if the following exist
- Categories
- Ordinal
- Equal Intervals
- Absolute Zero Point
Ratio Rule
Which rule would apply when the following exists
- Categories
Nominal rule
Which rule would apply if the following exist
- Categories
- Ordered (ordinal)
- Equal Intervals
Interval rule
Which rule would apply when the following are present
- Categories
- Ordering
Ordinal rule
What level of scale would you require if you are wanting to get an aggregate (summative) score?
i.e., Summative Scale
- At least Interval
- 1:09*
What level of data is produced when using a likert scale?
1:10:30
- ordinal-level data
- because they do not actually have equal levels between responses
- however they are treated as interval (summative) scale
- because they behave interval like (although theoretically there are not equal distances between strongly disagree & disagree)
What level of data is derived from binary choice scale?
- nominal
- not as much info as a likert, but its simpler
1:13:25
Why would you use a faces scale?
- good for children
- they dont understand things in the same way adults do
1:14:20
What are some other scaling methods that yield ordinal data?
1:17
- method of paired comparisons (1:17:19)
- must choose one on basis of some rule
-
comparative scaling (1:19:19)
- sorting/ranking along a dimension
- what is worse cheating on taxes or cheating on test
-
categorical scaling (can be nominal data)
- sorting stimuli according to rule
- good or bad / best or worst
- essential - not essential (our questionnaire)
- depends on how question is framed as to whether you get ordinal or nominal data (with categorical)
What are the main item formats used when writing test items?
1:27:10
-
Constructed-response format
- Completion item
- Short-answer
- Essay
-
Selected-response format
- Matching items
- True-False
- Multiple choice
What are the different types of constructed-response format?
- Completion item (1:25)
- provides word to complete sentence
- must be a specific correct answer
- poor design can lead to scoring problems
-
Short-answer (1:31:15)
* word, sentence, paragraph in answer to a highly specific question - Essay (132:10)
What are the different types of selected-response formats?
-
Matching
2. True-False (1:34:20)
- must contain a single, unambigous idea & be short
- use kuder-richardson for binary
- Multiple choice (1:35:35)
- a stem
- a correct item
- multiple incorrect alternative options (distractors, foils)
- stem - expectancy tables are used to evaluate this
- could use kuser-richardson & look at it internally
- or look at all 4 and use chronbach alpha
What are the four complex multiple choice formats?
1:38:45
- Classification
- If-then conditions
- Multiple conditions
- Multiple true-false
- Oddity
Which scaling methods did we use & why?
1:23:20
Previous research (Talbot et. al.)
- used Likert (ordinal)
- made it summative (assumed interval) to get a composite score
We will use
- Likert (assume interval)
- how felt if not enough sleep
- Categorical
- emotion is essential, useful, unnecessary
- to behaviour of whether you get enough sleep
What is an If-then condition?
1:39:35
- one of four complex multiple-choice format
- The test-taker must decide the consequence of one or more conditions being present
“If the true variance of a test increases but the error variance remains constant, which of the following will occur?”
a. Reliability will increase
b. Reliability will decrease
c. Observed variance will decrease
d. Neither reliability nor observed variance will change
if variance has an increased proportion of true variance - this will always increase reliability
What is meant by the term multiple conditions in terms of item format?
1:41:05
- it is one of the four complex multiple-choice formats
- The test-taker uses two or more conditions in the statements listed to draw a conclusion
“Given that Mary’s raw score on a test is 60, the test mean is 59, and the standard deviation is 2, what is Mary’s z score?”
a. -2.00
b. - .50
c. .50
d. 2.00
z score = difference between raw score & mean / standard deviation
1 / 2 = .50
What are multiple true-false items?
1:42:10
- one of four complex multiple-choice formats
The test-taker decides whether one, all, or none of the two or more conditions or statements listed in the stem are correct
“Is it true that (1) Alfred Binet was the father of intelligence testing and the (2) his first intelligence test was published in 1916?”
a. Both 1 and 2
b. 1 but not 2
c. 2 but not 1
d. Neither 1 nor 2
What is an oddity item?
1:43:16
- one of four complex multiple-choice formats
- The test-taker indicates which option does not belong with the others
- which is the odd one out?
“Which of the following names does not belong with the others?”
a. Alfred Adler
b. Sigmund Freud
c. Carl Jung
d. Carl Rogers
Freud, Jung & Rogers = listeners
Adler = confronter
What is the first step in writing test items?
- create an item bank or pool
- start off with at least twice as many as you want to end up with
What is a classification item format?
- one of four complex multiple-choice formats
- The test-taker classifies a person, object or condition into one of several categories designated in the stem
Jean Piaget is best described as a _____________ psychologist
a. Clinical
b. Developmental
c. Psychometric
d. Social
