Appraisal Flashcards
Appraisal
- the process of assessing or estimating attributes
ACA ethics consider it unethical to administer a test to a client from a given population unless that particular test or inventory has been normed on that specific population.
- Always inform clients about the limitations of any test
Subject Paradigm
- relies mainly on the scorer’s opinion
Objective test
- rater’s opinion plays little or no part in the scoring process
Halo Effect
- If the rater knows the test taker’s attributes, the rater’s personal bias significantly impact the rating
- colleagues being familiar with the the test taker so it rates him/her higher
Short Answer
- It’s a free choice or free response test (short open-ended)
Free choice (free response)
- test taker can respond in any manner
- short answer questions
Recognition Item
- give the examinee 2 or more alternatives to choose from
Dichotomous recognition items
- present two opposing choices
Multipoint item
- When a test gives the examinee 3 or more forced choices in an item
Social desirability phenomenon
- asserts that individuals will choose answers that they feel are socially acceptable
Difficulty index
- percentage of individuals who answered each items; individual’s score is evaluated by comparing it to others who took it
Normative
- each item is independent of all other items; individual’s score is evaluated by comparing it to other who took it
Ipsative
- compare traits within the same individual, do not compare a person to other persons, does not reveal absolute strengths, measured in response to his or her own standard of behavior, points out highs and lows
Speed test
- limited testing time to prevent perfect scores
Power test
- designated to evaluate the level of mastery without a time limit- include difficult test items
Projective test
- are free responses
- subjective vs. objective
Achievement test
- measures maximum performance
Personality/interest inventory
- measure typical performance
Spiral test
- items get progressively more difficult
Cyclical test
- several sections, which are spiral in nature
Vertical test
- versions for various age brackets, levels of education
Horizontal test
- measures various factors during the same testing procedure
- e.g battery test
Validity
- whether the test measures what is supposed to measure
- Content Validity
- whether or not the test examines or samples the behavior under scrutiny
- is the instruments’ content appropriate for its purpose? it must contain items measuring physical, cognitive, and psychological factors
- Construct Validity
- test ability to measure a theoretical construct like intelligence
- experiment factor analysis
- any trait that you cannot directly measure or observe
- Concurrent Validity
- how well the test compares to other instruments that are intended for the same purpose
- Criterion Validity
- the effectiveness of an instrument predicting an individual’s performance on a specific criterion
- Consequential validity
- tries to ascertain the social implications of using tests
- Convergent Validity
- the relationship/correlation of a test to an independent measure/trait
- Predictive Validity
- test’s ability to predict performance now and in the future
- high criterion
e. g: GRE, LSAT, SAT
- Face Validity
- looks or appears to measure the intended attribute
- Incremental Validity
- process by which a test is refined and becomes more valid as contradictory items are dropped
- Synthetic Validity
- helper looks for test that have been shown to predict each job element
- Discriminant Validity
- test will not reflect unrelated variables
- depression measure are not related to achievement instrument
Physical vs. Psychological Measurements
- Physical measurements are more reliable than psychological ones
Factor Analysis
- analysis the interrelationships of an instruments items revealing hidden traits of dimensions (data reductions)
Validity Coefficient
- A correlation between a test score and the criterion measure
A RELIABLE TEST is NOT Always VALID! bUT…
A VALID TEST is Always Reliable!
Standard Error of Measurement
- Indicates expected margin of error in a predicted criterion score due to imperfect validity
Construct
- any trait that cannot be directly measured or observed
- abstract: intelligence
Observed Score
- his/her true score plus the amount of error during the administration
Reliability
- How consistent a test measures an attribute
Equivalent alternate forms reliability
- Occurs when half of a group takes one form of a test and the other half takes another parallel form, after which a reliability coefficient is determined for all scores. This method control for fatigue, practice, and motivation
Tes-retest reliability
- give the same test to the same group of people two times and then correlate the scores
Split-half method
- the individual takes the entire test as a whole then the test is divided into halves, the correlation between the half scores yields a reliable coefficient
1. Kudar Richardson Formula 20: when items are dichotomous
2. Cronbach coefficient alpha used when there are multipoint responses
Interrater/interobserver reliability
- utilized with subjective tests, several raters assess the same performance
Increasing a test length raises reliability
- Increasing a test length raises reliability
Reliability of 1.00
- indicates a perfect score that has no error. This generally occurs only in physical measurements.
Reliability of 0.90
- Indicates that 90% of the score measured the attribute in question while 10% of the score is indicative of error. 90% id true variance while 10% is error variance. Meaning that 90% of the score is accurate whole 10% is inaccurate.
To calculate a coefficient of determination or true variance of a reliability coefficient, square the correlation
- Example: 0.70 x 0.70 = .49 x 100 = 49%
IQ =
- Mental Age (MA)/Chronological Age (CA) X 100
- IQ formula has been replaced by the standard age score (SAS)
Francis Galton
- Intelligence primarily genetic, unitary factor
- unitary faculty
Spearman
- Intelligence was comprised of 2 factors: a general ability (G) and a Specific (S) which were thought to be applicable to any mental task
J.P. Guilford
- Isolated 120 factors which added up to intelligence
- Convergent thinking: occurs when divergent thoughts/ideas are combined into a singular concept
- Divergent thinking: ability to generate a novel idea
Convergent thinking:
- occurs when divergent thoughts/ideas are combined into a singular concept
Divergent thinking:
- ability to generate a novel idea
Alfred Binet and Theodore Simon
- created 1st intelligence test
- test was created to discriminate normal from retarded Parisian children
Lewis Terman
- Americanized the Binet, associated with Stanford University
Classical test theory
- Most influential sum of true + amount of error presents item response - detects bias.
Survey Achievement Batteries
- SAT 10
- Iowa test of basic skills
- MAT 8
- Weschler Scales - most widely used
The Weschler Preschool and Primary Scale of Intelligence
WPPSI-III
- It’s an IQ test designed for children between the ages of 2 years 6 months and 7 years and 3 months
The Weschler Adult Intelligence Scale (WAIS-III)
- It’s an IQ test intended for ages 16 and up
The Weschler Intelligence Scale for Children (WISC-IV)
- It’s appropriate for kids ages 6 to 16 years 11 months
The Merrill-Palmer Scale of Mental Tests
- It’s an intelligence test for infants
WWI
- provided the impetus for the group testing movement
Culture fair test
- items are known to be subject regardless of his/her culture
Arthur Jensen
- Black vs. White IQ controversy, Blacks were bred for strength not intelligence
John Ertl
- claimed he invented an electronic machine to analyze neural efficiency and take the place of a paper/pencil IQ test
MMPI-2
- Standardized personality test, self-report personality inventory intended to help clinicians diagnose and treat patients over the age of 18
Psychometric
- form of mental testing
Francis Galton
- 1st intelligence theory
Projective testing
- Association: what comes to mind when you look at this?
- Completion: complete these sentences with real feelings
- Construction: drawing a person
- neutral stimuli
- Examiner bias is common when using projective testing
Howard Garner
- 8 primary intelligence
Raymond Cattell
- 16 PF (Personality Factor) called factor analytic test, measures personality factors such as assertiveness, emotional maturity
- Fluid and Crystallized Intelligence Theory
- Fluid: measured by content-free reasoning such as block design or pectoral analogy problem, think logical.
- Crystallized: measured by test that focus on content, ability to use skills, knowledge gained through learning
Buros
- Mental Measurement Yearbook
Thematic Apperception Test (TAT)
- by Murray
- uses pictures, intentionally ambiguous, client is asked to make up a story for each of them
Test Bias
- results from a test being normed solely on white middle clients
Bender Visual Motor Gestalt Test
- known for its ambiguity to discern whether brain damage is evident
Interest Inventories
- work best with clients in high school or above, criticism is it emphasizes professional positions and minimizes blue collar jobs, not highly correlated, reliable, not threatening
AMECD
- Association for measurement and evaluation counseling development
Standard Error of Measurement
- tells you how accurate/inaccurate a test score is, low standard of error means high reliability
Informal assessment techniques
- self-report, case notes, checklist, interviews
Clients should know a test is merely a single source of data and not infallible
Clients should know a test is merely a single source of data and not infallible
More public education is needed in the area of testing
More public education is needed in the area of testing
Never generalize on the basis of a single test score
Never generalize on the basis of a single test score
Buckley Amendment
- individuals over 18 can inspect their own records and those of their children
Item Difficulty
- number of people who answered the item correctly Divided by Total of people tested
Parallel Form of test
- An equivalent form of the same test
The Kuder-Richardson coefficients of equivalence
-
Cross validation
- it’s indeed smaller than the initial validity coefficient
- called shrinkage
SAS
- standard age score replace Binet;s IQ formula
Wechsler IQ replaced Binet
- because Binet didn’t seem to be the best test for adults
Mean 100 and SD 15
- IQ test Wechsler and Binet
The IQ movement began with
- the work of Binet
MMPI-2
- standardize personality test
Psychometric
- any form of mental test
Psychometric
- any form of mental test
- branch of counseling or psychology that dedicates to testing
projective test
- neutral stimuli
16 PF (personality factors)
- Raymond B. Cattell
aptitude test
- potential and predicts
Achievement test
- examines what you know
TAT (thematic apperception test)
- pictures
- 30 cards and one blank. the cards are ambiguous so the client can make up a story
Bender Gestalt - the Bender visual motor gestalt test
- utilized when counselor suspects the clients has a motoric disability
AMECD
- Association for Measurement and Evaluation in Counseling Development
social desirability
- person often answers questions in a socially acceptable manner
Standard error of measurement
- it tell you how accurate or inaccurate a test score is
- Low SEOM = high reliability
- x = t + or - e
sefl-reports
- counselor shy away from them since clients can provide inaccurate answers
clinical psychologist
- have the most training in testing
IQ TEST
- SD = 15
- Mean = 100
Public Law 93-380 Buckley Amendment
- a person over 18 years old can inspect their own records and their children
Lewis Terman
- Americanized the Binet
- he was associated with the Stanford University
- the test became Standford-Binet
