Week 4

Question 1

describe test conceptualisation

Answer 1

Review of exisiting tests –> decide if need to develop own. target contruct written as a definition of what the test measures and applicable population.

Question 2

Describe test contruction

Answer 2

aim to generate an item pool with good content validity. over inclusion of items.

Question 3

what are some giudlines for test contruction

Answer 3

use staightfoward language
avoid double barrelled items, slang and colloquial terms (they become obsolete)
consider the impact of positive and negative items
items can be reponded to by majority
straightfoward and non judgment language

Question 4

what is nominal data

Answer 4

simplest form of measurement in which infomation is placed into catergories (football team, yes/no.

Question 5

what is ordinal data

Answer 5

catergorisation incorporates rank but no unit of measurment (itunes top 20 countdown, runners in a race)

Question 6

what is interval data

Answer 6

categories are of measument and there are equal interval before each number. E.g. height average of adults, results of an IQ test

Question 7

what is ratio data

Answer 7

measurement with equal intervals which contain an ultimate zero. E.g. ruler, or number of times someone has experienced something, measuring grip strength.

Question 8

what are some pros and cons of the likert scale

Answer 8

pros: measures degree ot trait. easy to use. informative.

Cons: number of reposnses need to be considered
Odd vs even number responses.

Question 9

what are some pros and cons of the Binary choise scale (T/F)

Answer 9

pros = easy to construct, easy to score, quick, can have more items bc quick.

Cons = allows guessing
interpretation of question may be vague. only suits questions with dichotomour repsonses. content not a rich.

Question 10

what is a paired comparison?

Answer 10

Test-taker has to choose one of two options (e.g., a statement, object, picture) on the basis of some rule. The value (e.g., 1 or 0) of each option in each paired comparison is determined by judges prior to test administration.

Question 11

what is comparitive scaling?

Answer 11

Sorting or ranking stimuli (e.g., statements, objects, photographs, etc.) according to some rule (e.g., best to worst, most justifiable to least justifiable, etc.)

Question 12

what is catergorical scaling?

Answer 12

Categorising stimuli (e.g., statements, objects) according to some rule (e.g., “best” and “worst”, “always justifiable” and “never justifiable”)

Question 13

What are some pros and cons of MCQ

Answer 13

pros = lots of content, quick administer and score, prevents bluffing.

Cons = suppresses creativity. doesnt suit all subject matter.

Question 14

what are some pros and cons of Essays?

Answer 14

pros = complex, imaginitve original knowledge used. written comunication. infomation generated no recognised (memory reliance)

Cons = narrow content, bluffing, hiding behing good writing, time consuming to score, inter-rater reliability issues.

Question 15

describe the test tryout phase

Answer 15

test is administered to a representative sample with standadised instructions.

Question 16

Describe item analysis

Answer 16

data from tryout used to improve psyhometrics through assessment of difficulty, dimentionality and distribution.

Question 17

what item difficulty and how is it assessed?

Answer 17

performance of each individual item should differe on the basis of the test takers knowledge.

if one item is answere 100% correct or incorrect it is bad.

IDI = examiness who answered correctly / total number of examiness (m = .5 range = .3 - .8).

Question 18

what is item distribution?

Answer 18

remove items which a skewed.
i.e. items everyone answeres the same

keeps items with high varience or with a mean close to centre or possible range.

Question 19

describe how dimentionality is assessed?

Answer 19

Usually starts with exploratory factor analysis (EFA) to identify a manageable number of factors to extract.

Confirmatory factor analysis (CFA) used when number of factors is known

Question 20

what does Factor analysis assist with?

Answer 20

determine the number or underlying latent variable or contructs (do all items measure one thing?)

Question 21

destinguish between oblique and othagonal rotations

Answer 21

Oblique = assumed factors correlated 
Othagonal = factors uncorrelated.

Question 22

what is an item-scale correlation when is it used?

Answer 22

the correlation between the score for the test item and the scale score

Question 23

what is test revision?

Answer 23

used when modifying an old or creating a new test. after analsysis performed and items removed or edited goes back to test tryout phase.

Question 24

why do tests ‘age’?

Answer 24

domains and  interpretations change
stimuli age
Words chnage meaning
Test norms become outdates
Theories behind tests change

Question 25

what is cross validation?

Answer 25

administer same test to 2 different sample? are results correlated?

Question 26

what is validity shrinkage

Answer 26

lower validity second time round - but near enough it good enough.

Question 27

what is co-validation ?

Answer 27

two different tests measuring the same construct administered to the same sample should have similair scores. (useful for test batteries)