Module 1: Psychometric Properties and Principles Flashcards
1
Q
Psychological Testing
A
+ process of measuring psychology-related variables by means of devices or procedures designed to obtain a sample of behavior
+ numerical in nature
+ individual or by group
+ administrators can be interchangeable without affecting the evaluation
+ requires technician-like skills in terms of administration and scoring
+ yield a test score or series of test score
+ minutes to few hours
2
Q
Psychological Assessment
A
+ gathering and integration of psychology-related data for the purpose of making a psychological evaluation
+ answers referral questions through the use of different tools of evaluation
+ individual
+ assessor is the key to the process of selecting tests and/or other tools of the evaluation
+ requires an educated selection of tools of evaluation, skills in evaluation, and thoughtful organization, and integration of data
+ entails logical problem-solving that brings to bear many sources of data assigned to answer the referral question
3
Q
What are the types of psychological assessment?
A
- Educational
- Retrospective
- Remote
- Ecological Momentary
- Collaborative
- Therapeutic
- Dynamic
4
Q
Educational Assessment
A
evaluate abilities and skills relevant in school context
5
Q
Retrospective
A
draw conclusions about psychological aspects of a person as they existed at some point in time prior to the assessment
6
Q
Remote
A
subject is not in physical proximity to the person conducting the evaluation
7
Q
Ecological Momentary
A
“in the moment” evaluation of specific problems and related cognitive and behavioral at the very time and place that they occur
8
Q
Collaborative
A
the assessor and assesee may work as “partners” from initial contact through final feedback
9
Q
Therapeutic
A
therapeutic self-discovery and new understanding are encouraged
10
Q
Dynamic
A
describe interactive approach to psychological assessment that usually follows the model: evaluation > intervention of some sort > evaluation
11
Q
Psychological Test
A
device or procedure designed to measure variables related to psychology
12
Q
What are the parts/components of the psychological test and other related terms?
A
- Content
- Format
- Item
- Administration Procedures
- Score
- Scoring
- Cut-Score
- Psychometric Soundness
- Psychometrics
- Psychometrist or Psychometrician
13
Q
Content
A
subject matter
14
Q
Format
A
form, plan, structure, arrangement, layout
15
Q
Item
A
a specific stimulus to which a person responds overtly and this response is being scored or evaluated
16
Q
Administration Procedures
A
one-to-one basis for group administration
17
Q
Score
A
code or summary of statement, usually but not necessarily numerical in nature, but reflects an evaluation of performance on a test
18
Q
Scoring
A
the process of assigning scores to performances
19
Q
Cut-Score
A
reference point derived by judgment and used to divide a set of data into two or more classification
20
Q
Psychometric Soundness
A
technical quality
21
Q
Psychometrics
A
science of psychological measurement
22
Q
Psychometrist or Psychometrician
A
refer to professional who uses, analyzes, and interprets psychological data
23
Q
Ability or Maximal Performance Test
A
assess what a person can do
24
Q
What are the types of ability tests?
A
- Achievement
- Aptitude
- Intelligence
25
Achievement Test
+ measurement of the previous learning
+ fact-based or conceptual
26
What are achievement tests used for?
+ used to measure general knowledge in a specific period of time
+ used to assess mastery
27
What do achievement tests rely on?
relies mostly on content validity
28
Aptitude Test
refers to the potential for learning or acquiring a specific skill
29
What do aptitude tests tend to focus on?
tends to focus on informal learning
30
What do aptitude tests rely on?
rely mostly on predictive validity
31
Intelligence Test
refers to a person’s general potential to solve problems, adapt to changing environments, abstract thinking, and profit from experience
32
Human Ability
considerable overlap of achievement, aptitude, and intelligence test
33
Typical Performance Test
+ measure usual or habitual thoughts, feelings, and behavior
+ indicate how test takers think and act on a daily basis
+ no right and wrong answers
34
What kind of scale do typical performance tests use?
uses interval scales
35
Personality Test
+ measures individual dispositions and preferences
+ designed to identify characteristic
+ measured ideographically or nomothetically
36
What are the different types of personality tests?
1. Structured Personality Tests
2. Projective Personality Tests
3. Attitude Tests
4. Interest Inventories
37
Structured Personality Tests
provide statement, usually self-report, and require the subject to choose between two or more alternative responses
38
Projective Personality Tests
unstructured, and the stimulus or response are ambiguous
39
Attitude Tests
elicit personal beliefs and opinions
40
Interest Inventories
measures likes and dislikes as well as one’s personality orientation towards the world of work
41
Aside from the main four kinds of psychological tests, what are other kinds of psychological tests?
1. Speed Tests
2. Power Tests
3. Values Inventory
4. Trade Test
5. Neuropsychological Test
6. Norm-Referenced Test
7. Criterion-Referenced Tests
42
Speed Tests
the interest is the number of times a test taker can answer correctly in a specific period
43
Power Tests
reflects the level of difficulty of items the test takers answer correctly
44
Interview
method of gathering information through direct communication involving reciprocal exchange
45
What are the types of interviews?
1. Standardized/Structured
2. Non-standardized/Unstructured
3. Semi-Standardized/Focused
4. Non-Directive
46
Standardized/Structured
questions are prepared
47
Non-standardized/Unstructured Interview
pursue relevant ideas in depth
48
Semi-Standardized/Focused
may probe further on specific number of questions
49
Non-Directive
subject is allowed to express his feelings without fear of disapproval
50
What are examples of interviews?
1. Mental Status Examination
2. Intake Interview
3. Social Case
4. Employment Interview
5. Panel Interview (Board Interview)
6. Motivational Interview
51
Mental Status Examination
determines the mental status of the patient
52
Intake Interview
determine why the client came for assessment; chance to inform the client about the policies, fees, and process involved
53
Social Case
biographical sketch of the client
54
Employment Interview
determine whether the candidate is suitable for hiring
55
Panel Interview (Board Interview)
more than one interviewer participates in the assessment
56
Motivational Interview
used by counselors and clinicians to gather information about some problematic behavior, while simultaneously attempting to address it therapeutically
57
Portfolio
samples of one’s ability and accomplishment
58
Case History Data
refers to records, transcripts, and other accounts in written, pictorial, or other form that preserve archival information, official and informal accounts, and other data and items relevant to an assessee
59
Case Study
a report or illustrative account concerning a person or an event that was compiled on the basis of case history data
60
Groupthink
result of the varied forces that drive decision-makers to reach a consensus without critical reasoning or evaluation of the consequences or alternatives
61
Behavioral Observation
monitoring of actions of others or oneself by visual or electronic means while recording quantitative and/or qualitative information regarding those actions
62
Natural Observation
observe humans in natural setting
63
SORC Model
Stimulus, Organismic Valuables, Actual Response, Consequence
64
Role Play
defined as acting an improvised or partially improvised part in a stimulated situation
65
Role Play Test
assesses are directed to act as if they are in a particular situation
66
What are other tools used for psychological assessment?
Biofeedback devices (e.g. computers, physiological devices)
67
Steps in the Psychological Assessment Process
1. Determining the referral question
2. Acquiring knowledge relating to the content of the problem
3. Data collection
4. Data interpretation
68
Hit Rate
accurately predicts success or failure
69
Profile
narrative description, graph, table. Or other representations of the extent to which a person has demonstrated certain targeted characteristics as a result of the administration or application of tools of assessment
70
Actuarial Assessment
an approach to evaluation characterized by the application of empirically demonstrated statistical rules as determining factor in assessors’ judgement and actions
71
Mechanical Prediction
application of computer algorithms together with statistical rules and probabilities to generate findings and recommendations
72
How many levels of interpretation are there in psychological assessment?
There are 3 levels.
73
Level 1 (levels of interpretation)
+ there is a minimal amount of any sort of
interpretation
+ minimal concern with intervening processes
+ data are primarily treated in a sampling or correlate way
+ no concern with underlying constructs
+ found in large-scale selection testing
+ for psychometric approaches
74
Level 2 (level of interpretation)
a. Descriptive Generalizations
b. Hypothetical Construct
75
Hypothetical Construct
the assumption of an inner state which goes logically beyond the description of visible behavior
76
Level 3 (level of interpretation)
+ the effort to develop a coherent and inclusive theory of the individual life or a “working image” of the patient
+ the clinician attempts full scale exploration of the individual’s personality, psychosocial situation, and developmental history
77
Extra-Test Behavior
observations made by an examiner regarding what the examinee does and how the examinee reacts during the course of testing that are indirectly related to the test’s specific content but of possible significance to interpretation
78
What are the parties in psychological assessment?
1. Test Author/Developer
2. Test Publishers
3. Test Reviewers
4. Test Users
5. Test Takers
6. Test Sponsors
7. Society
79
Test Author/Developer
creates the tests or other methods of assessment
80
Test Publishers
they publish, market, sell, and control the distribution of tests
81
Test Reviewers
prepare evaluative critiques based on the technical and practical aspects of the tests
82
Test Users
uses the test of assessment
83
Test Takers
those who take the tests
84
Test Sponsors
institutions or government who contract test developers for various testing services
85
Test Battery
selection of tests and assessment procedures typically composed of tests designed to measure different variables but having a common objective
86
Assumptions about Psychological Testing and Assessment
Assumption 1: Psychological Traits and States Exist
Assumption 2: Psychological Traits and States can be Quantified and Measured
Assumption 3: Test-Related Behavior Predicts Non-Test-Related Behavior
Assumption 4: Test and Other Measurement Techniques have strengths and weaknesses
Assumption 5: Various Sources of Error are part of the Assessment Process
Assumption 6: Testing and Assessment can be conducted in a Fair and Unbiased Manner
Assumption 7: Testing and Assessment Benefit Society
87
Trait
+ any distinguishable, relatively enduring
way in which one individual varies from another
+ permits people to predict the present from the past
+ characteristic patterns of thinking, feeling, and behaving that generalize across similar situations, differ systematically between individuals, and remain rather stable across time
88
Psychological Trait
+ intelligence, specific intellectual abilities, cognitive style, adjustment, interests, attitudes, sexual orientation and preferences, psychopathology, etc.
+ exists as a construct
89
States
+ distinguish one person from another but are relatively less enduring
+ characteristic pattern of thinking, feeling, and behaving in a concrete situation at a specific moment in time
90
Construct
an informed, scientific concept developed or constructed to explain a behavior, inferred from overt behavior
91
Overt Behavior
an observable action or the product of an observable action
92
Assumption 1: Psychological Traits and States Exist
+ Trait is not expected to be manifested in behavior 100% of the time
+ Whether a trait manifests itself in observable behavior, and to what degree it manifests, is presumed to depend not only on the strength of the trait in the individual but also on the nature of the action (situation-dependent)
+ Context within which behavior occurs also plays a role in helping us select appropriate trait terms for observed behaviors
+ Definition of trait and state also refer to a way in which one individual varies from another
+ Assessors may make comparisons among people who, because of their membership in some group or for any number of other reasons, are decidedly not average
93
Do traits always manifest in behavior?
No. Trait is not expected to be manifested in behavior 100% of the time
94
A trait being observable is dependent on what factors?
Whether a trait manifests itself in observable behavior, and to what degree it manifests, is presumed to depend not only on the strength of the trait in the individual but also on the nature of the action (situation-dependent)
95
What is another factor that plays a role in helping professionals select appropriate trait terms for observed behaviors?
Context within which behavior occurs also plays a role in helping us select appropriate trait terms for observed behaviors
96
What does the definition of trait and state refer to?
Definition of trait and state also refer to a way in which one individual varies from another
97
What may assessors do in terms of traits and states?
Assessors may make comparisons among people who, because of their membership in some group or for any number of other reasons, are decidedly not average
98
Assumption 2: Psychological Traits and States can be Quantified and Measured
+ Once the trait, state or other construct has been defined to be measured, a test developer consider the types of item content that would provide insight to it, to gauge the strength of that trait
+ Measuring traits and states means of a test entails developing not only appropriate tests items but also appropriate ways to score the test and interpret the results
99
Cumulative Scoring
assumption that the more the testtaker responds in a particular direction keyed by the test manual as correct or consistent with a particular trait, the higher that testtaker is presumed to be on the targeted ability or trait
100
Assumption 3: Test-Related Behavior Predicts Non-Test-Related Behavior
+ The tasks in some tests mimics the actual behaviors that the test user is attempting to understand
+ Such tests only yield a sample of the behavior that can be expected to be emitted under nontest conditions
101
Assumption 4: Test and Other Measurement Techniques have strengths and weaknesses
Competent test users understand and appreciate the limitations of the test they use as well as how those limitations might be compensated for by data from other sources
102
Error
+ refers to something that is more than expected; it is component of the measurement process
+ refers to a long-standing assumption that factors other than what a test attempts to measure will influence performance on the test
103
Error Variance
the component of a test score attributable to sources other than the trait or ability measured
104
What are the potential sources of error variance?
1. Assessors
2. Measuring Instruments
3. Random errors such as luck
105
Classical Test Theory
each testtaker has true score on a test that would be obtained but for the action of measurement error
106
Assumption 6: Testing and Assessment can be conducted in a Fair and Unbiased Manner
+ Despite best efforts of many professionals, fairness-related questions and problems do occasionally rise
+ In all questions about tests with regards to fairness, it is important to keep in mind that tests are tools ꟷthey can be used properly or improperly
107
Assumption 7: Testing and Assessment Benefit Society
Considering the many critical decisions that are based on testing and assessment procedures, we can readily appreciate the need for tests
