Week 8- The Assessment Process Flashcards
What is a basic explanation of Norm-Reference Assessments?
What a persons score is compared to the scores o other people.
How are Norms obtained?
By testing a clearly defined, standard, group and looking at the scores they obtain.
How do you work out where a person ranks among the Norm group?
By converting the raw score into a derived score.
What type of Norm Assessment is useful in achievement testing?
Age Norms.
What type of Norm Assessment is useful in testing the achievement levels of school children?
Grade Norms
What are Percentile Ranks (in Norm Assessment)?
The percentage of the norm group that earn the same or less than the person.
What are the 3 Benefits of having Norms?
- Provides information about the person’s level of functioning.
- Takes little time to administer for the amount of information obtained.
- Provides more information than could be obtained through just observation.
What are the 4 Cons of Norms?
- Norms are based on samples that DO NOT adequately represent the type of population to which the scores are compared to.
- Grade, and age norms, can be seen as saying that the person should perform at that level for the current test and also for other tests in other areas of function.
- Quickly out of date.
- Sample size needs to be adequate.
What are the three steps in developing Norms?
- DEFINE the target population
- SELECT the (representative) sample
- STANDARDISATION. This then generates the Norms.
What are the 4 ways in which we can evaluate the Norm Groups?
- REPRESENTATIVENESS of the group (i.e. Cultural Norms)
- DATE the norms were established
- SIZE of the norm group
- RELEVANCE of the Norms to the test taker
How large should the sample size of the Norm group be?
Over 100 but usually over 1000
Size of the Norm Group should be large enough to:
- Ensure stability of the test
2. Ensure inclusiveness
What are Criterion-Referenced Tests concerned with?
With what a person does, not how they compare with others.
What type of test assess’s the performance on a test in comparison to an established standard?
Criterion-Referenced Tests
What kind of test would you use to establish if a person can to can’t use Public Transport?
Criterion-Referenced Test
What test focuses on specific content domains?
Domain Referenced Tests
What are the benefits of Criterion-Referenced Tests?
It provides information that is relevant when making a decision like: i) Is the person is ready to go onto the next level. ii) Are there any sub-skills what need more attention?
What are the Cons of Criterion-Referenced Tests?
- Focuses on specific facts, not understanding
- Need to ask if the items fully represent the knowledge you want to assess?
- Puts performance in yes/no, black/white categories?
- Reporting results can be quite lengthy
What are the 4 steps when writing a Criterion-Referenced Tests?
- State the general objective.
- Task analyse the skill
- For each task write a test item.
- Decide on the level of success required for mastery of each component.
What is Terminal Objective?
When writing a Criterion-Referenced Test, it is when test authors decide on the level of success required for mastery.
What is involved in the task analysis component of writing a Criterion-Referenced Test?
List the component tasks needed that make up the skill.
What are three ways in which to reexamine a test to gauge reliability?
- Same test at different times (re-test reliability)
- Use different items that measure the same construct
- Take the test under different conditions
What are 4 Assumptions to the theory of reliability?
- People’s traits are stable
- Errors are random and occur because of natural variance.
- The Observed score is the person’s true/actual score
What is an OK reliability coefficient?
anything above .80
What are the types of reliability coefficient?
- Test-Retest
- Parallel Form
- Internal Consistency
What does Test-Retest reliability measure?
It measures stability over time.
How short should the interval be in Test-Retest reliability?
2-4 Weeks
What are some of the issues of Test-Retest reliability?
Results are influenced by:
- How the test is given
- How much the person remembers or has learnt.
- Practice effects
- Memory
Describe Parallel-Form Reliability?
Alternate forms of the same test is given to the same participant. There should be equivalent means and variances and a high reliability coefficient.
What are the Pro’s of Parallel-Form Reliability?
- Memory and Practice effects are reduced
2. Good for follow up studies
What are the Con’s of Parallel-Form Reliability?
- Some tests (like the WISC) don’t have alternate forms
2. Carryover effects. The strategies used on one test can be used on the other
What are the 2 ways in which Internal Consistency is gained?
- Split-Half Reliability
2. Kuder-Richardson Reliability and Coefficient Alpha
Define Split-Half Reliability?
It is a way of obtaining Internal Consistency. The test is divided into 2 half are the items are correlated to each other.
When should you use the Kuder-Richardson?
If the test is not equivalent (i.e if the first half of the test is harder than the second half).
What is the KR-20 good for measuring?
Dichotomous items
What is the Cronbachs A good for measuring?
Things on a continuous variables (i.e Height, Weight)
What are the 5 factors that affect reliability?
- Test length- Longer test = higher reliability
- Test-Retest interval- Shorter= Less reliability
- Variability of Scores
- Guessing
- Variation within test situation
What is the formula for SEM?
SEM=SDx√(1-r)
What does the SEM actually measure?
It measures the average amount the person’s observed score diverts from their true score.
What is the formula for CI?
CI=Obtained Score +/- Z (SEM)
What are the four things Validity must have?
- Must be established in reference to the purpose of the test
- Is a matter of degree, not just a yes/no answer
- Must not use just one type of validity, no one type covers everything
- Tests are not valid if used in isolation from the social system they are used in.
What are the three types of Validity?
- Face Validity
- Content Validity
- Criterion Related Validity
Describe Content Validity?
Refers to if the items on the test represent the domain being tested.
What are the core issues of Content Validity?
- APPROPRIATENESS of the items
- COMPREHENSIVNESS. Do they cover the whole domino?
- MASTERY. Does it cover all levels of mastery?
What are the three Procedures for getting Content Validity?
- Specialists map the domain
- Make the test accurate
- Put the validation process in the test manual
What is/ isn’t Content Validity good for?
Good for achievement and occupational tests.
Not good for personality or aptitude tests.
Define Criterion-Related Validity?
When you compare the test scores to another test.
What are the two types of Criterion-Related Validity?
- Concurrent- Uses a current criterion to use as a comparison.
- Predictive- Uses the test score at one time and compared it to another criterion at a future time.
