Module 2: What Makes a Good Test? Flashcards
What is the Rational-Empirical Approach to Test Development?
Rational: knowledge of the construct and psychological theory drives the process.
Empirical: collecting data to evaluate individual items and overall test.
How many Assumptions to Test Development?
7
What are the Assumptions of Psychological Testing?
- Psychological traits/states actually exist.
- Psychological traits can be measured.
- Psychological traits predict future behaviour.
- Tests have strengths and Weaknesses.
- Various sources of error are part of assessment.
- Testing/assessment can be conducted in a fair and unbiased manner.
- Testing and assessment benefits society.
- Psychological Traits/States Actually Exist:
Traits: any distinguishable, relatively enduring way in which one individual varies from another.
States: more temporary.
Assumes that people have recognisable traits and that they differ on these - stable over time.
- Psychological Traits can be Measured:
Can infer their existence from behaviour.
Definition > content > consistent scoring system.
- Psychological Traits Predict Future Behaviour:
Traits should predict future behaviour - provides insight on the person.
- Tests have Strengths and Weaknesses:
No matter how well constructed - need to appreciate the limitations and used the tools as well to compensate.
- Various Sources of Error are Part of Assessment:
Error: factors other than what test attempts to measure will always influence the test.
Error variance: component of a test score attributable to sources other than the trait or ability measured.
- Testing/Assessment can be Conducted in a Fair and Unbiased Manner:
Tests give a standardised set of instructions, intended populations.
- Testing and Assessment Benefits Society:
Good tests can take subjectivity out of evaluations.
What Makes a Good Test?
Reliability: consistency.
Validity: measures what it’s supposed to.
Other considerations: clarity, beneficial.
What are Criterion-Referenced Tests?
Assess whether a criteria is met; scoring high or coming first not important - pass/fail.
What are Norm-Referenced Testing and Assessment?
Evaluating performance by comparing an individual’s test score to scores of a group.
What Makes a Good Score?
Have different scoring systems.
Find a typical score (mean) and how far scores typically deviate (SD).
What are Norms (Tests)?
Test performance data of a particular group of test-takers that are designed to use as a reference when evaluating or represent individual test scores.
What is Standardisation?
Administering a test to a representative sample of test-takers for the purpose of establishing norms (if it’s standardised, it has clearly specified procedures and normative data).
What is Stratified Sampling?
Recruiting different subgroups to recruit a representative sample (minimises selection bias).
What is Stratified-Random Sampling?
Every member of the population has an equal opportunity of being included (gold standard but costly).
What is Purposive Sampling?
Specifically selecting a sample that is believed to be representative of a population (hard and expensive to access).
What is Incidental/Convenience Sampling?
Convenient to access, and available to use.
What is the Problem with Extremely Small Samples?
Few individuals might skew the interpretation of data.
What is the Problem with Extremely Large Samples?
Measurement error decreases as sample size increases, further increases have minimal effect.
What is the Ideal Sample Size?
At least 30+.
What are Percentile Type Norms?
The percentage of people who achieve that raw score on the test.
What are the Age/Grade Type Norms?
Average performance at different ages or year levels.
What are National Type Norms?
Derived from a sample that was nationally representative.
What are National Anchor Type Norms?
Equivalency table to allow for conversion between two tests.
What are Subgroup Type Norms?
Seperate norms may be developed for specific demographic subgroups.
What are Local Type Norms?
Provides normative information for a local population if locals are believed to differ from the national norms.
What are the Limitations of Norms?
Assumption of norms is that they remain valid over time.
What are the Fixed-Reference Group Scoring Systems?
The distribution of scores obtained from the initial standardisation is used as the basic for calculating scores in the future.
What to Consider with Culture in Test Development/Choosing a Test?
Check how appropriate the tests norms are within the targeted population / when interpreting consider culture.
