Evaluating test items

Question 1

Invalid test

Answer 1

-Measurements , predictions and results may be inaccurate and invalid

Question 2

Steps of test construction

Answer 2

• Research question: operationalise construct
• develop test: research and develop items
• pilot test: 10 ppts per item
• evaluation: internal consistency, factor analysis
• distribution of responses: normal distribution
• Reliability
• Validity
• Administer new test

Question 3

Research question

Answer 3

-Operationalising constructs allows us to define behaviour like, personality and intelligence. Some concepts are hard to define as they can be interpreted in different ways. If the construct isn’t directly observable there must be many test items on that construct to try and quantify it.

Question 4

Develop test

Answer 4

• DO research so you know the concept well, this helps develop test items with face and content validity. Develop test stimuli e.g. Likert scale or MCQs.
• Language must be appropriate for the audience (use Flasch reading ease scale)

Question 5

Evaluation

Answer 5

-Internal consistency

Question 6

Factor analysis

Answer 6

• Identifies patterns of correlations (correlation matrix) and a data reduction technique (reduce variables in large data sets)
• Extension of multiple regression and ANOVA
• Confirmatory Factor Analysis: assumes the structure of a factor using the researches ideas of how the construct functions and if the data will fit
• Exploratory Factor Analysis: The structure of the construct is determined by the data

Question 7

Distribution of responses

Answer 7

• check for normal distributions

- Remove items that reduce internal consistency, are non-normally distributed, have poor face validity

Question 8

Reassess the reliability and validity

Answer 8

• Internal consistency

- check validity of the test face and convergent validity

Question 9

Administer test to new sample

Answer 9

• Check reliability and validity

- factor analysis