Psy Testing Flashcards
Importance of Psychological Tests (5)
Decisions for:
- Early School Placement
- College Entrance Decisions
- Military Job Selections
- Career Choices
- Psychological Adjustments
Characteristics of Psychological Tests (3)
- Sample of behavior
- Obtained under standardized conditions
- Established scoring rules for obtaining quantitative information from behavior sample
Why is standardization vital?
- Referential in nature - performance is measured relatively to everybody else’s performance
- Reduces between subject variability due to extraneous variables.
- Administered in mass
Difference between subjective and objective scoring rules
Objective Scoring Rules: Most mass produced tests fall into this category. Different qualified examiners will all come to the same score for an identical set of responses.
Subjective Scoring Rules: When the judgment of the examiner is an important part of the test, different examiners can legitimately come to different conclusions concerning the same sample of behavior. There conclusions should be similar, however.
Categories of Psychological Tests (3)
- Specific Task Performance Tests
- Observations of the Subject’s behavior within a particular context
- Self-report measures
What are specific tasks performance tests
Referred to as “Tests of maximal performance”; designed to uncover what an individual can do, given the specific test conditions.
- Two underlying assumptions:
- The subject understands what is required of the test.
- The subject exerts maximal effort to succeed.
What is an observation of the subject’s behavior within a particular context?
Examiner might observe subject having a conversation or some other social interaction.
What are self-report measures?
Subject describes their feelings, attitudes, beliefs, or interests.
Frequently subject to self-censorship.
Items are frequently included to measure the extent to which people provide socially desirable responses s/t self-serving bias
History: Circa 1000 BC (Chinese)
Chinese introduced written tests for civil service positions
History: 1850 (US)
US begins civil service examinations
History: 1890 (Cattell)
Mental test for college students - strength, resistance to pain and reaction time
History: 1905(Binet-Simon)
Scale of mental development used to classify mentally retarded children in France
History: 1914 (US)
WWI army recruitment - Alpha and Beta test
History: 1916 (Terman)
Develops Stanford-Binet Test and coined IQ
History: 1920-1940
Factor Analysis, Projective tests and Personality Inventories
History: 1941-1960
Vocational interest measured
History: 1961-1980
Item response theory and neuropsychological testing developed
History: 1980 - present
Computerized testing
Examples of Fluid Attributes
Mood
Attitude
Opinions
Personal values
Example of Stable Attributes
Intelligence
Interest
Example of Relative Attributes
Ability
Interest
Personality
Reasons why intelligence is a valid and useful construct
- wide variety of mental processing tasks show systematic individual variation.
- related to success in a wide variety of life tasks: school performance, training programs, and work behaviors.
What is General Mental Ability/Intelligence?
performance of tasks involving the manipulation, retrieval, evaluation , and/or processing of information which shows individual differences.
What are the 7 primary mental abilities according to Thurstone (1938)?
- Verbal Comprehension - vocabulary, reading, verbal analogies
- Word Fluency — anagrams, rhyming tests
- Number – mathematical operations
- Space - spatial visualizations and mental transformation.
- Associative Memory – rote memory
- Perceptual Speed – quickness in noticing similarities and differences
- Reasoning - skill in inductive, deductive, and math problems
What was spearman’s theory of intelligence (1904)?
Two factor Theory - He believed that two cognitively demanding tasks are positively correlated.
Test = g + S +e
g = general intellectual factor* S = measurement error
What are Catell’s (1963) 2 types of general intelligence
Fluid Intelligence: the ability to see relationships, i.e. analogies and number and digit series completion.
Crystallized Intelligence: an individual’s acquired set of knowledge and skills.
In cognitive Psychology, crystallized intelligence is furthered classified into?
Declarative Knowledge: Fact based information
Procedural Knowledge: How to do things.
Two major group factors of general intelligence according to Vernon (1960)?
Verbal-Educational, and Spatial - Motor
Carrol (1993) has created 7 classes of broad abilities from the general factor g developed with the aid of factor analysis
- Fluid Intelligence
- Crystallized Intelligence
- General Memory
- Visual Perception
- Auditory Perception
- Retrieval Ability
- Cognitive Speediness
What was Guilford’s structure of intellect model?
Rejected the idea of g, but instead best seen in function of content, operations and product creating 180 different types of specific intellgience
What was Robert Sternberg’s conceptualization of intelligence?
Deals with how intelligent behavior is generated, what behaviors are intelligent in specific environments, and when a specific behavior is intelligent.
How was the IQ calculated according to the Stanford-Binet test?
IQ = (MA/CA) X 100
MA = mental age CA = Chronological (adult) age
In modern tests, how was the deviation of IQ obtained?
Average and scale
Characteristics of a good test of general mental ability?
- Broad sampling of tasks
- Sufficient number of items within task type
- Not test specific content
- Indifference of the indicator
Major classes of worker productivity measured by?
- Production counts
- Personnel Data
- Judgemental methods
What is the Hawthorne effect?
When people know their behavior is being monitored, they will change their behavior to create a favorable impression
2 Major Factors on the Job Performance Construct
- Performance on specific individual tasks in the JD
2. Behaviors necessarily for the organization to function smoothly
2 Judgement types
Rankings - comparison among workers
Ratings - performance based on standards; most common
Ranking Techniques
1 Forced Distribution - Division into categories; recommended for layoffs
- Full Ranking - Rank order
- Pair-comparison - supervisor compares each worker to every other worker; suitable for small businesses
What is a graphic rating scale?
The supervisor makes a direct judgment about the quality of each workers’ performance on a specific response scale
Different types of response scales (4)
- Continuous Scales: A score is computed by measuring the distance from one end of the scale.
- Verbally Anchored Scale: A small number of discrete categories that is “anchored” on either end with the range of abilities measured. These scales can vary as to the specificity of the verbal anchors.
- Numeric Scales: Verbal Anchors contain a numerical range within each category.
- Graphic Scales are simple to use and allow for the computation of scores to compare workers on overall job performance.
Behavior-based scales
- Mixed Standard Scale (MSS) -Good, average, and poor performance are assessed with respect to specific job-related behaviors. advantages of the MSS are they refer to concrete, observable behavior, and they require relatively simple judgments on the part of the supervisor
- Behavior anchored rating scales - Similar to graphic rating scales, but uses specific behaviors to anchor the scale.
- Behavioral Observation Scales (BOS): list of “critical” behaviors which the supervisor has to rate in terms of frequency.
Items indicate either desired or undesirable aspects of work performance:
Common rater errors (4)
- Halo Errors: Because of a general impression of a worker, there is little discrimination when rating this worker on different work related behaviors.
- Leniency Errors: A supervisor has a general tendency to rate all workers higher, or all workers lower.
- Range Restriction Errors: A supervisor fails to use the entire response range available, therefore making it difficult to make fine distinctions between the work performances of similar workers.
- Memory distortions may make it difficult for a supervisor to remember all the work related behavior of a particular worker that she has observed since the previous rating period. Cons for large workforce
Standard deviation of performance
method assess the difference, in $$$ terms, between the value of an average worker (a worker at the 50th percentile) and the value of an exceptional worker ( a worker at the 85th percentile).
Classification of Psychological Tests based on the number of continuum (6)
- Individual or Group Test
- Speed or Power Test
- Cognitive or Affective Test
- Aptitude Tests
- Achievement Tests
- Affective Tests
What is an individual or Group test?
Indicates how the test is administered. Many versions of I.Q. tests are given in a one to one situation.
What is a Speed or Power Test?
Refers to whether any time constraints are built into the test.
Difference between a speed and power tests?
Speed test - simple with strict deadline
Power test - difficult without deadline
What is a cognitive or affective test?
Measures activity and cognitive
What is an achievement test?
assess knowledge of information already learned.
What is an aptitude test?
Tests attempt to gauge whether a person is capable of learning a specific knowledge base.
What is an affective test?
designed to assess interests, attitudes, and personal values of an individual.
What is an objective scoring?
Objective scoring procedures are fully specified before grading begins so that anyone grading the test would calculate the same score for a particular set of answers.
T or F: Standardized tests have established norms to which you can compare an individual’s performance.
T
T or F: 98 % of all scholastic exams you have taken have been nonstandardized.
T
How would you determine norms?
Determined by the test standardization group where there is a normal distribution of scores on a standardized tests
A guide to all currently available psychological tests
The Mental Measurement Yearbook (MMY)
What are the content classifications in MMY?
- Achievement
- Behavior Assessment
- Developmental
- Education
- English & Language
- Fine Arts
- Foreign Languages
- Intelligence and General Aptitude
- Mathematics
- Neuropsychological
- Personality
- Reading
- Science
- Sensory-Motor
- Social Studies
- Speech and Hearing
- Vocations
T or F: Psychological Test data care considered “privileged communication” and may not be widely distributed without the consent of the examined.
T
Guidelines for test design and construction
1: Defining the constructs you want to measure and outline the proposed content of the Test
What is a job analysis?
Lists of important components for the job, list of work related behaviors and measures entire cross-section of critical incidents
What issues should test planners consider in creating a representative sample?
- What are the topics and materials to be tested?
- What kind of questions should be constructed?
- What item and test formats should be used?
- When, where, and how is the test to be given?
- How should the tests be scored?
What is content analysis?
Used in achievement tests which the key subject areas are listed and the percentage of the test to be devoted to each individual subject area is decided.
What is the taxonomy of cognitive domain established by Bloom and Krathwohl
I. Knowledge: recall of specific facts - define, identify, list and name
II. Comprehension: understanding the purpose or meaning - convert, explain, and summarize.
III. Application: using information and ideas in novel situations - compute, determine, and solve.
IV. Analysis: Breaking down large pieces of information in order to examine the structure and interrelationships among its component parts - analyze, differentiate, and relate.
V. Synthesis: Combining various elements or parts into a structural whole - design, devise, formulate, and plan.
VI. Evaluation: making a judgment based upon reasoning - compare, critique, evaluate, and judge.
What is the taxonomy of cognitive domain established by Geriach and Sullivan
- Identifying: consists of indicating which member of a set belongs in a particular category
- Naming: Supplying the proper verbal label for a referent, or a set of referents.
- Describing: consists of reporting relevant categories of objects, events, properties, or relationships.
- Constructing: creating a product according to certain specifications.
- Ordering: consists of arranging two or more referents in a specific ranking.
- Demonstrating: Performing a certain behavior to accomplish a test relevant task.
What is a table of specification?
allows for a thorough analysis of content and difficulty, and provides a framework for specific test item construction.
Rational vs Empirical approach in creating distractor items
The Rational Approach: The test developers understanding of the subject material and their ability to organize that material lead them to adopt specific distracters for specific test items
The Empirical Approach: You select distractors based on pre-test data.
Factors that are not under the control of test administrators
1 Fatigues experienced by test taker
- Motivation level
- Physical Discomfort
- Test Anxiety
Physical factors that can be controlled by test administrator
1 Light levels 2 Temperature 3 Ambient Noise Level 4 Ventilation 5 Minimal distractions
Responsibilities of a test administrators
- Scheduling the Exam: Of particular concern when testing children
- Inform students well before the test (reduces anxiety):
- When and where test is given?
- What subject material will be given?
- What type of test questions?
- How much time will be allowed?
- Familiar with the test
- Familiar with security procedures\
- Sufficient seating
What is item analysis?
determine the effectiveness of each individual test item.
- How difficult an individual item is?
- How good a job a particular item does in discriminating between high and low performance on the test?
- How do we determine what constitutes high or low performance on a psychological test?
How do you analyze items?
1 Criterion -referenced/Domain referenced testing: comparison to a set list of objectives or standards
- Norming Distributions comparing individual score with score distribution
What is test validity?
does the test actually measure what it intends to measure.
What is item validity?
does the specific test item correlate with what you are trying to measure
What is external criterion
Data from outside the test which we expect to correlate in some meaningful way with our test items.
What is internal consistency?
the relationship between performance on an individual item and performance on the entire test; item difficulty and discriminability
What is item difficulty?
measure of overall difficulty of the test item. The lower the p, the more difficult a particular item is.
What is item discrimination?
tells us how good a job a question does is separating high and low performers.
Item characteristics curve?
graph shows the percent correct on a particular test question as a function of the total test scores.
What is the item response theory?
proportion of correct responses to a particular test item is plotted as a function of the (estimated) true ability of individuals.
What is standardization sample?
a large sample of test takers who represent the population for which the test is intended.
What is simple random sampling?
every person in the target population has an equal chance of being in the standardization sample.
What is stratified sampling?
Most accurate way of developing norm group; Test developer takes into account all demographic variables (age, gender, socioeconomic status, geographic region) which can accurately describe the population of interest and then selects individual at random, but proportional to the demographic portrait of the test population.
What is cluster sampling?
sampling begins by dividing a geographic region into blocks and then randomly sampling within those blocks.
Parallel vs Equated forms
Parallel Forms: If the two tests have the same types and numbers of items of equal difficulty, the alternate versions are said to have parallel form.
Equated Forms: When we can’t develop two alternate forms with the exact same mean and standard deviation, we can still compare tests of equivalent difficulty through the use of a common metric, for example the Z score distribution
Purposes of Achievement Test
. Assess level of competence
- Diagnose strength and weaknesses
- Assign Grades
- Achieve Certification or Promotion
- Advanced Placement/College Credit Exams
- Curriculum Evaluation
- Accountability
- Informational Purposes
Summative vs Formative Evaluation
Summative Evaluation: Testing is done at the end of the instructional unit. The test score is seen as the summation of all knowledge learned during a particular subject unit.
Formative Evaluation: Testing occurs constantly with learning so that teachers can evaluate the effectiveness of teaching methods along with the assessment of students’ abilities.
National Assessment of Educational Progress has developed criterion reference for 10 subjects
- Art
- Occupational Development
- Citizenship
- Literature
- Math
- Music
- Reading
- Science
- History
- Writing
Major categories of standard achievement tests
Survey Test Batteries: Commonly used to determine general standing with respect to group performance.
Single Subject Survey Tests: Longer and more detailed than batteries, but only one subject are is covered by the test.
Diagnostic Tests: Allows for the identification of specific strengths and weaknesses within a subject area by subdividing the subject area into the underlying components.
Prognostic Tests: Aptitude tests which are designed to predict achievement in specific school subjects.
According to Gardner, what is intelligence?
ability to solve problems or to create products which are valued in one or more cultural settings.
What is the PASS model of cognitive processing?
The First Functional Unit (Upper Brain stem and Limbic System): responsible for allocating attentional resources and maintaining a constant sense of awareness. Controls Awareness
The Second Functional Unit (occipital, auditory, and parietal cortex): responsible for visual, auditory, tactile and olfactory perception and the storage of that information. Controls Analysis and Storage
The Third Functional Unit (: Prefrontal and frontal lobe): Responsible for complex cognitive activity including planning, regulation, and
Verification. Controls Complex Thought
What is schema?
mental template we use to organize the world.
What is assimilation?
Fitting in new pieces of information into existing knowledge structures. (Schemata)
What is accommodation?
Changing existing hypothesis (or Schemata) to fit new information
