Module 2: What Makes a Good Test?

Question 1

What is the Rational-Empirical Approach to Test Development?

Answer 1

Rational: knowledge of the construct and psychological theory drives the process.
Empirical: collecting data to evaluate individual items and overall test.

Question 2

How many Assumptions to Test Development?

Answer 2

7

Question 3

What are the Assumptions of Psychological Testing?

Answer 3

1. Psychological traits/states actually exist.
2. Psychological traits can be measured.
3. Psychological traits predict future behaviour.
4. Tests have strengths and Weaknesses.
5. Various sources of error are part of assessment.
6. Testing/assessment can be conducted in a fair and unbiased manner.
7. Testing and assessment benefits society.

Question 4

1. Psychological Traits/States Actually Exist:

Answer 4

Traits: any distinguishable, relatively enduring way in which one individual varies from another.
States: more temporary.
Assumes that people have recognisable traits and that they differ on these - stable over time.

Question 5

2. Psychological Traits can be Measured:

Answer 5

Can infer their existence from behaviour.
Definition > content > consistent scoring system.

Question 6

3. Psychological Traits Predict Future Behaviour:

Answer 6

Traits should predict future behaviour - provides insight on the person.

Question 7

4. Tests have Strengths and Weaknesses:

Answer 7

No matter how well constructed - need to appreciate the limitations and used the tools as well to compensate.

Question 8

5. Various Sources of Error are Part of Assessment:

Answer 8

Error: factors other than what test attempts to measure will always influence the test.
Error variance: component of a test score attributable to sources other than the trait or ability measured.

Question 9

6. Testing/Assessment can be Conducted in a Fair and Unbiased Manner:

Answer 9

Tests give a standardised set of instructions, intended populations.

Question 10

7. Testing and Assessment Benefits Society:

Answer 10

Good tests can take subjectivity out of evaluations.

Question 11

What Makes a Good Test?

Answer 11

Reliability: consistency.
Validity: measures what it’s supposed to.
Other considerations: clarity, beneficial.

Question 12

What are Criterion-Referenced Tests?

Answer 12

Assess whether a criteria is met; scoring high or coming first not important - pass/fail.

Question 13

What are Norm-Referenced Testing and Assessment?

Answer 13

Evaluating performance by comparing an individual’s test score to scores of a group.

Question 14

What Makes a Good Score?

Answer 14

Have different scoring systems.
Find a typical score (mean) and how far scores typically deviate (SD).

Question 15

What are Norms (Tests)?

Answer 15

Test performance data of a particular group of test-takers that are designed to use as a reference when evaluating or represent individual test scores.

Question 16

What is Standardisation?

Answer 16

Administering a test to a representative sample of test-takers for the purpose of establishing norms (if it’s standardised, it has clearly specified procedures and normative data).

Question 17

What is Stratified Sampling?

Answer 17

Recruiting different subgroups to recruit a representative sample (minimises selection bias).

Question 18

What is Stratified-Random Sampling?

Answer 18

Every member of the population has an equal opportunity of being included (gold standard but costly).

Question 19

What is Purposive Sampling?

Answer 19

Specifically selecting a sample that is believed to be representative of a population (hard and expensive to access).

Question 20

What is Incidental/Convenience Sampling?

Answer 20

Convenient to access, and available to use.

Question 21

What is the Problem with Extremely Small Samples?

Answer 21

Few individuals might skew the interpretation of data.

Question 22

What is the Problem with Extremely Large Samples?

Answer 22

Measurement error decreases as sample size increases, further increases have minimal effect.

Question 23

What is the Ideal Sample Size?

Answer 23

At least 30+.

Question 24

What are Percentile Type Norms?

Answer 24

The percentage of people who achieve that raw score on the test.

Question 25

What are the Age/Grade Type Norms?

Answer 25

Average performance at different ages or year levels.

Question 26

What are National Type Norms?

Answer 26

Derived from a sample that was nationally representative.

Question 27

What are National Anchor Type Norms?

Answer 27

Equivalency table to allow for conversion between two tests.

Question 28

What are Subgroup Type Norms?

Answer 28

Seperate norms may be developed for specific demographic subgroups.

Question 29

What are Local Type Norms?

Answer 29

Provides normative information for a local population if locals are believed to differ from the national norms.

Question 30

What are the Limitations of Norms?

Answer 30

Assumption of norms is that they remain valid over time.

Question 31

What are the Fixed-Reference Group Scoring Systems?

Answer 31

The distribution of scores obtained from the initial standardisation is used as the basic for calculating scores in the future.

Question 32

What to Consider with Culture in Test Development/Choosing a Test?

Answer 32

Check how appropriate the tests norms are within the targeted population / when interpreting consider culture.