Module 1: Psychometric Properties and Principles Flashcards
Psychological Testing
+ process of measuring psychology-related variables by means of devices or procedures designed to obtain a sample of behavior
+ numerical in nature
+ individual or by group
+ administrators can be interchangeable without affecting the evaluation
+ requires technician-like skills in terms of administration and scoring
+ yield a test score or series of test score
+ minutes to few hours
Psychological Assessment
+ gathering and integration of psychology-related data for the purpose of making a psychological evaluation
+ answers referral questions through the use of different tools of evaluation
+ individual
+ assessor is the key to the process of selecting tests and/or other tools of the evaluation
+ requires an educated selection of tools of evaluation, skills in evaluation, and thoughtful organization, and integration of data
+ entails logical problem-solving that brings to bear many sources of data assigned to answer the referral question
What are the types of psychological assessment?
- Educational
- Retrospective
- Remote
- Ecological Momentary
- Collaborative
- Therapeutic
- Dynamic
Educational Assessment
evaluate abilities and skills relevant in school context
Retrospective
draw conclusions about psychological aspects of a person as they existed at some point in time prior to the assessment
Remote
subject is not in physical proximity to the person conducting the evaluation
Ecological Momentary
“in the moment” evaluation of specific problems and related cognitive and behavioral at the very time and place that they occur
Collaborative
the assessor and assesee may work as “partners” from initial contact through final feedback
Therapeutic
therapeutic self-discovery and new understanding are encouraged
Dynamic
describe interactive approach to psychological assessment that usually follows the model: evaluation > intervention of some sort > evaluation
Psychological Test
device or procedure designed to measure variables related to psychology
What are the parts/components of the psychological test and other related terms?
- Content
- Format
- Item
- Administration Procedures
- Score
- Scoring
- Cut-Score
- Psychometric Soundness
- Psychometrics
- Psychometrist or Psychometrician
Content
subject matter
Format
form, plan, structure, arrangement, layout
Item
a specific stimulus to which a person responds overtly and this response is being scored or evaluated
Administration Procedures
one-to-one basis for group administration
Score
code or summary of statement, usually but not necessarily numerical in nature, but reflects an evaluation of performance on a test
Scoring
the process of assigning scores to performances
Cut-Score
reference point derived by judgment and used to divide a set of data into two or more classification
Psychometric Soundness
technical quality
Psychometrics
science of psychological measurement
Psychometrist or Psychometrician
refer to professional who uses, analyzes, and interprets psychological data
Ability or Maximal Performance Test
assess what a person can do
What are the types of ability tests?
- Achievement
- Aptitude
- Intelligence
Achievement Test
+ measurement of the previous learning
+ fact-based or conceptual
What are achievement tests used for?
+ used to measure general knowledge in a specific period of time
+ used to assess mastery
What do achievement tests rely on?
relies mostly on content validity
Aptitude Test
refers to the potential for learning or acquiring a specific skill
What do aptitude tests tend to focus on?
tends to focus on informal learning
What do aptitude tests rely on?
rely mostly on predictive validity
Intelligence Test
refers to a person’s general potential to solve problems, adapt to changing environments, abstract thinking, and profit from experience
Human Ability
considerable overlap of achievement, aptitude, and intelligence test
Typical Performance Test
+ measure usual or habitual thoughts, feelings, and behavior
+ indicate how test takers think and act on a daily basis
+ no right and wrong answers
What kind of scale do typical performance tests use?
uses interval scales
Personality Test
+ measures individual dispositions and preferences
+ designed to identify characteristic
+ measured ideographically or nomothetically
What are the different types of personality tests?
- Structured Personality Tests
- Projective Personality Tests
- Attitude Tests
- Interest Inventories
Structured Personality Tests
provide statement, usually self-report, and require the subject to choose between two or more alternative responses
Projective Personality Tests
unstructured, and the stimulus or response are ambiguous
Attitude Tests
elicit personal beliefs and opinions
Interest Inventories
measures likes and dislikes as well as one’s personality orientation towards the world of work
Aside from the main four kinds of psychological tests, what are other kinds of psychological tests?
- Speed Tests
- Power Tests
- Values Inventory
- Trade Test
- Neuropsychological Test
- Norm-Referenced Test
- Criterion-Referenced Tests
Speed Tests
the interest is the number of times a test taker can answer correctly in a specific period
Power Tests
reflects the level of difficulty of items the test takers answer correctly
Interview
method of gathering information through direct communication involving reciprocal exchange
What are the types of interviews?
- Standardized/Structured
- Non-standardized/Unstructured
- Semi-Standardized/Focused
- Non-Directive
Standardized/Structured
questions are prepared
Non-standardized/Unstructured Interview
pursue relevant ideas in depth
Semi-Standardized/Focused
may probe further on specific number of questions
Non-Directive
subject is allowed to express his feelings without fear of disapproval
What are examples of interviews?
- Mental Status Examination
- Intake Interview
- Social Case
- Employment Interview
- Panel Interview (Board Interview)
- Motivational Interview
Mental Status Examination
determines the mental status of the patient
Intake Interview
determine why the client came for assessment; chance to inform the client about the policies, fees, and process involved
Social Case
biographical sketch of the client
Employment Interview
determine whether the candidate is suitable for hiring
Panel Interview (Board Interview)
more than one interviewer participates in the assessment
Motivational Interview
used by counselors and clinicians to gather information about some problematic behavior, while simultaneously attempting to address it therapeutically
Portfolio
samples of one’s ability and accomplishment
Case History Data
refers to records, transcripts, and other accounts in written, pictorial, or other form that preserve archival information, official and informal accounts, and other data and items relevant to an assessee
Case Study
a report or illustrative account concerning a person or an event that was compiled on the basis of case history data
Groupthink
result of the varied forces that drive decision-makers to reach a consensus without critical reasoning or evaluation of the consequences or alternatives
Behavioral Observation
monitoring of actions of others or oneself by visual or electronic means while recording quantitative and/or qualitative information regarding those actions
Natural Observation
observe humans in natural setting
SORC Model
Stimulus, Organismic Valuables, Actual Response, Consequence
Role Play
defined as acting an improvised or partially improvised part in a stimulated situation
Role Play Test
assesses are directed to act as if they are in a particular situation
What are other tools used for psychological assessment?
Biofeedback devices (e.g. computers, physiological devices)
Steps in the Psychological Assessment Process
- Determining the referral question
- Acquiring knowledge relating to the content of the problem
- Data collection
- Data interpretation
Hit Rate
accurately predicts success or failure
Profile
narrative description, graph, table. Or other representations of the extent to which a person has demonstrated certain targeted characteristics as a result of the administration or application of tools of assessment
Actuarial Assessment
an approach to evaluation characterized by the application of empirically demonstrated statistical rules as determining factor in assessors’ judgement and actions
Mechanical Prediction
application of computer algorithms together with statistical rules and probabilities to generate findings and recommendations
How many levels of interpretation are there in psychological assessment?
There are 3 levels.
Level 1 (levels of interpretation)
+ there is a minimal amount of any sort of
interpretation
+ minimal concern with intervening processes
+ data are primarily treated in a sampling or correlate way
+ no concern with underlying constructs
+ found in large-scale selection testing
+ for psychometric approaches
Level 2 (level of interpretation)
a. Descriptive Generalizations
b. Hypothetical Construct
Hypothetical Construct
the assumption of an inner state which goes logically beyond the description of visible behavior
Level 3 (level of interpretation)
+ the effort to develop a coherent and inclusive theory of the individual life or a “working image” of the patient
+ the clinician attempts full scale exploration of the individual’s personality, psychosocial situation, and developmental history
Extra-Test Behavior
observations made by an examiner regarding what the examinee does and how the examinee reacts during the course of testing that are indirectly related to the test’s specific content but of possible significance to interpretation
What are the parties in psychological assessment?
- Test Author/Developer
- Test Publishers
- Test Reviewers
- Test Users
- Test Takers
- Test Sponsors
- Society
Test Author/Developer
creates the tests or other methods of assessment
Test Publishers
they publish, market, sell, and control the distribution of tests
Test Reviewers
prepare evaluative critiques based on the technical and practical aspects of the tests
Test Users
uses the test of assessment
Test Takers
those who take the tests
Test Sponsors
institutions or government who contract test developers for various testing services
Test Battery
selection of tests and assessment procedures typically composed of tests designed to measure different variables but having a common objective
Assumptions about Psychological Testing and Assessment
Assumption 1: Psychological Traits and States Exist
Assumption 2: Psychological Traits and States can be Quantified and Measured
Assumption 3: Test-Related Behavior Predicts Non-Test-Related Behavior
Assumption 4: Test and Other Measurement Techniques have strengths and weaknesses
Assumption 5: Various Sources of Error are part of the Assessment Process
Assumption 6: Testing and Assessment can be conducted in a Fair and Unbiased Manner
Assumption 7: Testing and Assessment Benefit Society
Trait
+ any distinguishable, relatively enduring
way in which one individual varies from another
+ permits people to predict the present from the past
+ characteristic patterns of thinking, feeling, and behaving that generalize across similar situations, differ systematically between individuals, and remain rather stable across time
Psychological Trait
+ intelligence, specific intellectual abilities, cognitive style, adjustment, interests, attitudes, sexual orientation and preferences, psychopathology, etc.
+ exists as a construct
States
+ distinguish one person from another but are relatively less enduring
+ characteristic pattern of thinking, feeling, and behaving in a concrete situation at a specific moment in time
Construct
an informed, scientific concept developed or constructed to explain a behavior, inferred from overt behavior
Overt Behavior
an observable action or the product of an observable action
Assumption 1: Psychological Traits and States Exist
+ Trait is not expected to be manifested in behavior 100% of the time
+ Whether a trait manifests itself in observable behavior, and to what degree it manifests, is presumed to depend not only on the strength of the trait in the individual but also on the nature of the action (situation-dependent)
+ Context within which behavior occurs also plays a role in helping us select appropriate trait terms for observed behaviors
+ Definition of trait and state also refer to a way in which one individual varies from another
+ Assessors may make comparisons among people who, because of their membership in some group or for any number of other reasons, are decidedly not average
Do traits always manifest in behavior?
No. Trait is not expected to be manifested in behavior 100% of the time
A trait being observable is dependent on what factors?
Whether a trait manifests itself in observable behavior, and to what degree it manifests, is presumed to depend not only on the strength of the trait in the individual but also on the nature of the action (situation-dependent)
What is another factor that plays a role in helping professionals select appropriate trait terms for observed behaviors?
Context within which behavior occurs also plays a role in helping us select appropriate trait terms for observed behaviors
What does the definition of trait and state refer to?
Definition of trait and state also refer to a way in which one individual varies from another
What may assessors do in terms of traits and states?
Assessors may make comparisons among people who, because of their membership in some group or for any number of other reasons, are decidedly not average
Assumption 2: Psychological Traits and States can be Quantified and Measured
+ Once the trait, state or other construct has been defined to be measured, a test developer consider the types of item content that would provide insight to it, to gauge the strength of that trait
+ Measuring traits and states means of a test entails developing not only appropriate tests items but also appropriate ways to score the test and interpret the results
Cumulative Scoring
assumption that the more the testtaker responds in a particular direction keyed by the test manual as correct or consistent with a particular trait, the higher that testtaker is presumed to be on the targeted ability or trait
Assumption 3: Test-Related Behavior Predicts Non-Test-Related Behavior
+ The tasks in some tests mimics the actual behaviors that the test user is attempting to understand
+ Such tests only yield a sample of the behavior that can be expected to be emitted under nontest conditions
Assumption 4: Test and Other Measurement Techniques have strengths and weaknesses
Competent test users understand and appreciate the limitations of the test they use as well as how those limitations might be compensated for by data from other sources
Error
+ refers to something that is more than expected; it is component of the measurement process
+ refers to a long-standing assumption that factors other than what a test attempts to measure will influence performance on the test
Error Variance
the component of a test score attributable to sources other than the trait or ability measured
What are the potential sources of error variance?
- Assessors
- Measuring Instruments
- Random errors such as luck
Classical Test Theory
each testtaker has true score on a test that would be obtained but for the action of measurement error
Assumption 6: Testing and Assessment can be conducted in a Fair and Unbiased Manner
+ Despite best efforts of many professionals, fairness-related questions and problems do occasionally rise
+ In all questions about tests with regards to fairness, it is important to keep in mind that tests are tools ꟷthey can be used properly or improperly
Assumption 7: Testing and Assessment Benefit Society
Considering the many critical decisions that are based on testing and assessment procedures, we can readily appreciate the need for tests
