Exam 1 Flashcards
You can have _________ without _________ but the opposite is not true
Reliability &; validity
Reliability is a prerequisite to validity
What an instrument measures and how well it does that task
Validity
The degree to which the evidence indicated that the items, questions, or tasks adequately represent the intended behavior domain
Content-related Validity
Concerns the extent to which the instrument may measure a theoretical or hypothetical construct or trait
Construct Validity
The degree to which test scores are consistent, dependable, stable, and repeatable
Reliability
a reliability coefficient is calculated simply by correlating the performance on the first administration with the performance on the second. It is expected that the variation in scores between the 1st and 2nd would relate to random error because everything else should be the same
Test-Retest
Individuals are given one form of the instrument initially and then are assessed with a second alternate form of the instrument; scores are correlated
Parallel forms
What was the original purpose for the MMPI? Were they successful?
To diagnose psychiatric illnesses; NO
Who created the MMPI?
Hatheway & Mckenley
4 Important Content areas to cover in an interview?
Identifying information, History of the problem, family background/history, Developmental history, age specific information, ethnic/cultural considerations
What is the difference between the content-related procedure & empirical criterion keying?
the rational approach basically looks at content relevance of the items
Items are selected based on their relationship to some external criterion rather than on their content; MMPI items selected that separated people who were considered normal from those who were diagnosed with some form of psychopathology
What does validity refer to in regards to MMPI?
Describes test taking attitude; whether or not they endorse the test in some distorted manner
MMPI I content scales
hostility scales
Alcoholism Scale
MMPI-2 content scales
ANX The anxiety Scale
DEP The Depression Scale
What is the Code type?
The 1 or two highest clinical scales at or above a T score of 65 with the most elevated one first
Scale 3T=69; Scale 6T=65 (3-6)
Scale 3T=70; Scale 6T=70 (3-6)
90 possible code types
Clinical Scale #1
Hs Hypochondriasis Scale: Indicates concern with physical symptoms and fear of bodily harm
Clinical Scale #2
D Depression: measures depressive symptoms and psychic energy
Clinical Scale #3
HY Hysteria: indicates awareness of problems and vulnerabilities, and social comfort
Clinical Scale #4
Pd Psychopathic Deviate: measures conflict, struggle, respect for society’s rules, anger, social poise, and exploitation
Clinical Scale #5
Mf Masculinity/Femininity: Male-Female, indicates interest in activities & behaviors that are stereo-typically masculine or feminine
Clinical Scale #6
Pa Paranoia: Indicates level of trust, sensitivity, and suspiciousness
Clinical Scale #7
Pt Psychasthenia: detects worry , anxiety, doubts, tension, discomfort, and obsessiveness
Clinical Scale #8
Sc Schizophrenia: Measures odd thinking and social alienation
Clinical Scale #9
Ma Hypomania: indicates level of excitability and psychic energy
Clinical Scale #0
Si Social Introversion: indicates people orientation
What are the general objectives of an interview?
- Gather information not otherwise easily obtained
- Establish a relationship conducive to obtaining the information
- Develop greater understanding in both the interviewer and interviewee of the specific problem behavior
- Provide direction and support in helping the interviewee deal with problem behaviors
What is Standard Deviation?
a measure of the variability or spread of scores in a group; it is the positive square root of the variance
variance = 25 SD= 5
In terms of correlation, what determines a strong relationship?
-/+ 1
What is a norm referenced instrument and what do you need to evaluate to determine if that instrument is appropriate?
what does norm refer to?
is the norm group appropriate?
size?
sample comparable in terms of sex, age, ethnic, SES, education level, and geographic representation
large samples are better
how did they select the norm group? (simple random sample/ stratified sample/ cluster sampling)
These forms of reliability examine the instrument “internally” to determine its consistency or lack of error
split-half reliability, spearman-brown formula, Kuder-Richardson formulas, coefficient alpha
______ instruments compare an individual’s performance with a standard or criterion
criterion referenced
need classification consistency for reliability evidence
refers to how consistently different raters evaluate the answers to the items on the instrument
INTERrater reliability
Provides an estimation of the range of scores that would be obtained if someone were to take an instrument over and over again
SEM Standard Error of Measurement
Reliability is related to results and the degree of error in those results
concerns the degree to which the instrument is unable to capture significant aspects of the construct
construct under-representation
refers to the degree to which scores or results are affected by factors that are extraneous to instrument’s intended purpose
construct irrelevance
concerned the extent to which an instrument was systematically related to an outcome criterion
Criterion-related validity
Validity evidence based on response processes
analyzing the cognitive processing of the clients while they are taking the assessment is important in some assessment situations
the goal; think out aloud; question examinees about their performance strategies; examine subgroup differences
Validity evidence based on internal structure
an instrument may be designed to measure one clearly defined construct or it may have subtests or scales withing the instrument
Factor analysis (a statistical technique used to analyze the interrelationships of a set or sets of data): exploratory (explore the factor structure of the instrument) & confirmatory (use existing structure of the instruments and examine whether the factor analysis matches or confirms that structure)
Differential Item Functioning (examines whether there are group differences in terms of the percentage of individuals who get the item correct
Validity evidence based on relations to other variables
Criterion-Related validity - correlate measure with a criterion measure known to be valid
- concurrent validity: when the criterion measure is collected at the same time as the measure being validated
- predictive validity: when the criterion is collected later. In order for a test to be a valid screening device for some future behavior, it must have predictive validity
The extent to which your items are tapping into the underlying theory or model of behavior. It’s how well the items hang together (_______) or distinguish different people on certain traits or behaviors (________)
convergent validity; discriminant validity
Is closely related to correlation, is commonly used to determine the usefulness of a variable, or set of variables, in predicting another important or meaningful variable
regression
Another method that examines the relationship between an instrument and a criterion or predictor variable is ______ or _______. this approach is sometimes referred to as _________ because instruments are often used in decision making & this approach evaluates the accuracy of the decision making.
group separation; expectancy tables; decision theory
_______ occurs when the instrument predicts that individuals have “it” when in fact they do not
false positive
_______ occurs when the instrument predicts that test takers do not have it when in fact they do
false negative
The statistical procedure used to examine whether an instrument’s validity can be generalized is meta-analysis
Validity Generalization
________ focuses on examining and evaluating each item within an assessment; examines the qualities of each item
Item analysis
______ is an index that reflects the proportion of people getting an item correct
item difficulty (.50 is optimum)
___________ provides an indication of the degree to which an item correctly differentiates among the examinees on the behavior domain of interest
Item discrimination
_________ builds on the concepts of item analysis; the focus is on arriving at one true score and our attempts to control error in the instrument
IRT Item Response Theory
a statistical technique used to analyze the INTERRELATIONSHIPS of a set or sets of data
Factor Analysis
What is intelligence influenced by?
Genes & Environment; we cant determine if one is more important than the other
What are the gender differences related to tasks on the intelligence test?
Male: better at visual/spatial tasks
Females: verbal fluency tasks
no general intellectual differences
What is the current thinking on ethnic differences between intellectual tests?
SES is a major determinant
What are the different types of achievement assessments?
screening tests and comprehensive tests; each type includes single subject tests and multiple subject tests
________ tests are brief tests containing one sub-test for each subject area covered; achievement
Screening
________ tests contain more than one sub-test per subject area and cover the subject area in depth; achievement
Comprehensive
Screening tests
- Usually simpler to administer and score
- Useful in determining whether a comprehensive test is necessary
- Results may be misleading; can overestimate or underestimate an individual’s skills
- Do not measure other important achievement areas
Wide Range Achievement Test-4, Wechsler Individual Achievement Test - screener
Comprehensive Tests
- Must assess 3 or more subject areas typically taught in schools
- At least 2 different sub tests for each subject area
- Assess both lower and higher levels of cognitive skills within each subject area
Wechsler Individual Achievement Test-Comprehensive Test, Wookcock-Johnson III Tests of Achievement
_______ provide information about what an individual has learned or acquired.
Achievement Tests
________ predict future performance or ability to learn
Aptitude Tests
___________ is particularly critical because some researchers have found that it has more influence on est performance than either ethnicity or instrument bias
SES
An important concept to consider in interpreting assessment results with racially and ethnically diverse individual s is __________
acculturation
An instructor wants to know if an item is too difficulty or too easy on a test he should look at the ______
Item difficulty
Comprehensive Tests: the ______ the cognitive level assessed, the _____ reliable one can predict performance on higher levels; the ____ the level assessed, the more reliable the results
lower; less; higher
Many screening tests only assess ______ cognitive abilities; the lower the level assessed, ___ ability to predict overall achievement
lower; less
Categories of Achievement TESTS
- Survey Achievement Batteries
- Individual & Diagnostic Achievement Tests
- Criterion-referenced Tests & Minimum-level skills assessments
- Subject Area Tests
Scale 4-5-6 configuration; 4&6 above T score of 65
Scarlett O’Hara V
Scales 6-7-8 configuration; 6&8 high; all above 65
Psychotic “V” or Paranoid Valley
Validity scale Configuration
LFK
Scale 1-2-3
Neurotic Triad Configuration
What is the mean of an IQ test? SD?
100 &; 15
What happens to the SEM as the reliability coefficient increases?
less error; decreases