Psychometrics/testing Flashcards

Question

coverting t to z to ss scores

Answer 1

-T→ z → SS Z = (T-50)/10 Z = (SS-100)/15 -T = 10z + 50 Example: Convert T = 63 to SS 63 – 50/10 = z = 1.4 1.4(15) + 100 = 121

Answer 2

statistic that describes the relationship between two variables, X and Y Pearson: Range = -1.00 to +1.00 (magnitude and sign) R = 0; X and Y not related If r> 0, then higher scores on X associated with higher Y scores If r< 0 then higher scores on X associated with lower scores on Y

Answer 3

r = 0…but still relationship

Answer 4

heteroscedasticity homoscedasticity restriction of range

Answer 5

distribution is spread out- dots all over Thus…correlation coefficient will not accurately reflect

Answer 6

evenly distributed – better

Answer 7

reduce the magnitude of r; not sampling full range When a variable has restricted range, put little confidence in r

Answer 8

goal is to simplify a complex amount of information …with an assumption that variables in the matrix correlate the way they do because there are one or more underlying themes or factors that link some of the variables together

Answer 9

extracting one or more factors

Answer 10

is what it originally looks like; then we rotate it to get rid of all the negative loadings

Answer 11

shifting the factors to see if they are correlated or not

Answer 12

represent both how the variables are weighted for each f actor but also the correlation between the variables and the factor; shows loading of variables on the new rotated factors ; utilizes… Turstone’s Criteria : to have a simple structure

Answer 13

we can detect and eliminate- i.e. multiple choice test graded wrong

Answer 14

cannot be fixed, inevitable, inescapable; most concerning b/c cant fix

Answer 15

your obtained score consists of your true score plus error Obtained score = true score + error → Xo = Xt + e

Answer 16

Error due to time sampling; same group of examinees test again after a period of time has elapsed ISSUES = Practice effect and time interval SOLUTION: Test/ Re-test – not too long or too short! Practice effect- 2nd admin never same as 1st

Answer 17

error resulting from the need to select a subset of the total universe (domain) of items alternate-form method o 2 forms of test - same process to select items for both o Is prohibitive because difficult to create twice as many good items

Answer 18

-Items divided into two half tests – often odd/even -Decreases reliability when shorter (reliability related to length) -Correlation must be adjusted by means of Spearman-Brown formula

Answer 19

are all the item on the scale equally good measures of the construct?; is the scale homogenous”

Answer 20

-Correlation between the scores on each pair of items on a test -the larger the mean correlation, the more homogeneous the items

Answer 21

-Correlation between each item and the total score of the test -The correlation between the score on a particular item and the total score on the test is also known as the discriminability of the item

Answer 22

reliability test for items that are either correct or incorrect

Answer 23

measure of internal validity, used for non-binary items; MOST COMMON -High value = homogeneity = interchangeable of construct -Can be too high!! – identical questions, don’t survey variety -Low value = might not be measuring the same construct

Answer 24

fixed with percent agreement calculating the percentage of the items they agree Likely to underestimate the actual amount of error due to inter-rater difference

Answer 25

better way of assessing inter rater reliability- corrects for chance agreement and corrects for it

Answer 26

1.00 minus reliability

Answer 27

confidence interval where your score, if retested, will fall The SEM is INVERSELY related to the reliability -If reliability is high, SEM is low -If reliability is low, SEM is high

Answer 28

confidence interval where your true score is Example: 95% CI using SEE = 95-105 Means that there is a 95% chance that the person’s true score falls b/w 95 and 105

Answer 29

used when you are comparing TWO scores and taking into account error associated with both scores to determine whether the scores are in fact different from each other; SED is always larger than then SEM of the two SEMs SED = square root of (SEMsquared one + SEMsquared two) … aka SED = √(SEM21 + SEM22)

Answer 30

does the test measure what it claims to measure? *Reliability is a necessary condition for validity

Answer 31

-Does it look like a test that measures what is says it will? Does it look like they belong -A cosmetic issue – not a requirement for validity -Possible for test to be a highly valid measure, but low face validity - Ex. MMPI

Answer 32

-Do items have sufficient coverage? Covering what supposed to? -A matter of professional judgment – no actual test for content validity -Addressed during process of test construction

Answer 33

Goal is to demonstrate that the test correlates with other criteria that are important elements of the construct be measured Ex. Self reports scale of depression compared to psychiatric diagnosis

Answer 34

recorded at same time ; now -To show that a test can be a substitute for more costly or inconvenient tests -ex. Freshman GPA and college admissions test

Answer 35

recorded at a later point in time - is our test an accurate predictor? -To show that the test can accurately predict future performance -Ie SATs and college GPA

Answer 36

-Basis measure manner in which the relationship between a test and criterion is expressed is via the correlation - The correlation between a test and a criterion is called the validity coefficient

Answer 37

those who assess the examinee should be blind to the test scores when assigning criterion Ex. Psychiatrist should not know high score on depression scale before diagnosing depressing

Answer 38

poor reliability of the criterion measure Correct for error with correction for attenuation – but might over correct

Answer 39

validity of test differs depending on subgroup – i.e. gender and criterion is validity coefficient

Answer 40

Ability of a test to measure more accurately or precisely than other measures -Does the test add to already existing measures?

Answer 41

-Comparison of the with the only existing measures and the after the new test has been added -Statistical measure that identifies the optimal battery

Answer 42

a regression formula applied to a new sample will always shrink; helps to estimate by cross validation-- concern

Answer 43

to eliminate how much shrinkage

Answer 44

Is the test adequately measuring the construct of interest

Answer 45

when a tests correlates highly with another test or observation with another test we would expect it to correlate with Ex. High IQ and grades

Answer 46

when a test does not correlate with observations it should not correlate with

Answer 47

multimethod matrix - Methodology to assess convergent and discriminant validity

Answer 48

-Used to look at construct validity -Examines the test’s factor structure to determine if it fits what is predicted or theoretically expected

Answer 49

No theories used – no hypothesis before - Let data tell you (hypothesis)

Answer 50

start with hypothesis and test data against that (no hypothesis)

Answer 51

statistics for evaluating the utility of a test in assigning a diagnosis statistics depend on the cutting score

Answer 52

the score that divides our scale into two parts If above the cutting score – positive for a diagnosis If below – negative for the diagnosis Increasing the cutting score will DECREASE the sensitivity and INCREASE the specificity Decreasing the cutting score with INCREAES the sensitivity and DECREASE the specificity

Answer 53

Base rate must also be taken into account Base Rate - Percentage of those in sample who actually have the disorder Relative “cost” of false positive vs false negative error – depends on setting What’s the risk of over or under diagnosing? How much do you adjust the cutting score?

Answer 54

test accurately identifies a person as having the disorder (A)

Answer 55

tests says that the person has the disorder when s/he actually does not (B)

Answer 56

test accurately identifies a person as NOT having the disorder (D)

Answer 57

tests says that the person does not have the disorder when s/he actually does have disorder (C)

Answer 58

(A + D) / N Percentage of ALL CASES correctly classified by the test

Answer 59

A / (A + C) Percentage of people who HAVE the dx and were accurately identifies

Answer 60

D / (B + D) Percentage of people who DO NOT HAVE dx and correctly identified as negative

Answer 61

Positive predictive power – A / (A + B) Percentage who received POSITIVE diagnosis that actually have it

Answer 62

Negative predictive power - D / (C +D) Percentage who receive a NEGATIVE diagnosis and do not have it

Answer 63

test result is positive and diagnosis is positive: A test result is positive and diagnosis is negative: B testing result is negative and diagnosis is positive: C test result is negative and diagnosis is negative: D

Answer 64

To make sure that you have the best possible items for the purposes to which we intend to use our test ; make sure have best possible items

Answer 65

proportion who get item correct (reverse of what you’d think); how hard the item is Example: item difficulty: .60 means 60% of examinees got the item correct Harder items have LOW values for the item difficulty parameter Easy item have HIGH value for the item difficult parameter

Answer 66

consists of item that all have the same value for item difficulty (low or high)

Answer 67

shows items are too difficult

Answer 68

shows items are too easy

Answer 69

correlation between item score and total test score Refers to the degree to which an item can differentiate among test takers on the trait or ability being measured

Answer 70

One common measure is the correlation by the item score and total score on the test, technically known as the point-biserial correlation (Special case of the Pearson Product Moment Correlation Coefficient) Examines the correlation between a dichotomous variable and a continuous variable

Answer 71

High Positive Values = good discriminability Zero (or close) = poorly discriminating item – does not help discriminate Negative = bad item! Those who get item right tend to get lower scores on the test

Answer 72

plots % passing item (Y-axis) against the total score on the test (X-axis) Looks at difficulty in discriminability Steepness of curve or slope, provides information about item discriminability Steeper = greater discriminability

Answer 73

Involves use of mathematical models to define the item characteristic curve (ICC) Attempts to construct ICC’s that reflect the relationship between the probability of passing an item and the person’s standing on a scale of ability that is invariant across different samples Theta (Ɵ) = ability or latent trait p (Ɵ) = probability that an individual of ability level = (Ɵ) will get item correct

Answer 74

bases the ICC on a single item parameter: difficulty

Answer 75

is defined as the ability level (theta; Ɵ) where the probability of getting the item correct [p (Ɵ)] = .50

Answer 76

test predicts criterion with different accuracy for two groups; poor choice for making decisions about people from diverse populations

Answer 77

if trait being assessed is not equal in all subgroups (i.e. never been pregnant)

Answer 78

test predicts criterion with different accuracy for two groups -Biased b/c not predict criterion as accurately for males as does for females -Different slope

Answer 79

one group obtains higher scores on the test than the other, even though the test predicts the criterion score equally well for both groups -Same slope but one group has lower mean -Solution for slope and intercept bias = employ a modified quota system; i.e. admission committee decided on 50% male and 50% female instead of another cutoff score

Answer 80

-Want to obtain consent -Assent (from child) -What is included in informed consent? -General description, nature and purpose of assessment, fee -Client given opportunity to ask questions

Answer 81

-Mandated by law or government regulations (detention center) -Informed consent is implied because testing is conducted as a routine educational, institutional, or organization activity -(assessment when applying for a job) -Evaluate decisional capacity

Answer 82

-Client decided to whom the test results are released -If client referred by 3rd party, must discuss limits before assessment begins

Answer 83

raw and scaled scores; actual responses to questions; any recordings -Cannot withhold test data -Make efforts to see that the recipient is informed about the meaning of data

Answer 84

copies of manuals or copies of test questions or stimuli -Make reasonable efforts to maintain security -Client not entitled to material

Answer 85

not advisable, but not forbidden – better in office with supervision

Answer 86

-WISC-V: ages 6-16 -WAIS: ages 16-90

Answer 87

- Verbal concept formation; Verbal reasoning - Knowledge acquired from ones environment; Requires ability to express ideas in words

Answer 88

- Perceptual and fluid reasoning; Spatial processing; Visual-motor integration

Answer 89

- Working memory abilities; Weaknesses in attention and concentration will negatively affect WM - temporarily retain information in memory while performing a mental operation on it to produce a result

Answer 90

- Processing speed - Ability to quickly and correctly scan, sequence, or discriminate simple visual information - Measures Short term memory, visual memory, attention, visual-motor coordination

Answer 91

-Similarities – measures abstract verbal reasoning -Vocabulary – facility with words, extent of vocabulary, and verbal concept formation; crystallized ability -Information – fund of general knowledge; crystallized supplemental -Comprehension – understanding of general principles in social situations; fluid knowledge WISC: -Word Reasoning – fluid reasoning in older children (8 and up); for 6-8 its more direct retrieval factual knowledge

Answer 92

Block design – ability to analyze and synthesize abstract visual information; visual-motor coordination Visual Puzzles – nonverbal reasoning; analyze and synthesize abstract visual information; no visuo-motor component Matrix Reasoning – fluid intelligence, spatial ability, perceptual organization supplemental -Picture completion – influenced by crystallize d ability; influenced by processing speed; visual analogy to information; involves retrieval of stored information -Figure Weights – quantitative ability visually (19-69 only) WISC: -Picture concepts – abstract reasoning, visual and nonverbal analogy to similarities; more concrete

Answer 93

Digit-span -Forward – passive encoding -Backward – encoding and more working memory Arithmetic - requires mental manipulation Supplemental -L-N sequencing – more complex than DS; if greater than DS by 3+ = attention and concentration or motivational issues (16-69 only)

Answer 94

Coding – more influenced by endurance and graph-motor skill than SS; perfectionism might hinder Symbol search – ability to sustain attention; careful visual discriminations rapidly Supplemental: -Cancellation – organization, strategy, arrangement and ability to sustain attention (16-69 only)-

Answer 95

-VCI & PRI differ by 20 points or more --VCI & WMI differ by 20 points or more -PRI & PSI differ by 20 points or more

Answer 96

GAI – general ability index -use VCI and PRI -(exclude WMI and PSI) -General Ability Index (GAI = VCI + PRI) -(focus more on reasoning abilities)

Answer 97

in comparison to the rest of the examinee’s scores (7 and 13)

Answer 98

in comparison to my peers (85 and 115)

Answer 99

VCI vs PRI: >9 VCI vs WMI: > 12 PRI vs. PSI: > 12 PSI vs. WMI: > 12

Answer 100

Verbal knowledge and reasoning, crystalized intelligence vs. Visual/spatial processing and fluid reasoning

Answer 101

Performance on verbal/auditory tasks when working memory and attention are more (WMI) or less (VCI) important

Answer 102

Performance on tasks involving visual stimuli when speed of processing is more (PSI) or less (PRI) important

Answer 103

Attention to verbal/auditory stimuli (WMI) vs visual stimuli (PSI)

Answer 104

-Differences between pairs of index scores in previous slide are not statistically significant -No subtests stand out as RS (relative strength) or RW (relative weakness) within indices -Maybe a few ups and down but nothing extreme THESE ARE the people you can confidently use FSIQ as their “general intelligence”

Answer 105

VCI and PRI are both interpretable and are significantly different from each other

Answer 106

-WMI is interpretable stands out as relative weakness among index scores -WMI significantly lower than VCI (assuming both index scores are interpretable) -Arithmetic might not cluster with DS and LN because of quantitative component being stronger (A > DS,LN) or weaker (A < DS,LN)

Answer 107

-PSI is interpretable and stands out as relative weakness among index scores -PSI significantly lower than PRI (assuming both index scores are interpretable) -Might also see BDN > BD

Answer 108

Attentional difficulties are suggested if VCI is significantly higher than WMI AND PRI is significantly higher than PSI -attention: WMI, PSI - Reasoning and Problem Solving : VCI, PRI (WMI < VCI) and (PSI < PRI) WMI requires attention to AUDITORY stimuli; PSI requires attention to VISUAL stimuli

Answer 109

100 average, 15 SD 130+ very superior 120-129 superior 110-119 high average 90-109 average 80-89 low average 70-79 borderline below 69 extremely low

Answer 110

16-19 very superior 14-15 superior 12-13 high average 9-11 average 7-8 low average 5-6 borderline 1-4 extremely low

Answer 111

MMPI-2: At least 6th grade reading level & 18+ MMPI-A: 14-18

Answer 112

variable response inconsistency inconsistent responding or random responding -VRIN > 79T indicates a level of inconsistent responding that invalidates the test. The profile should not be interpreted. -T score of 80 or higher on VRIN, it is invalid and NOT interpreted.

Answer 113

true response inconsistency tendency to answer T or F; acquiescence bias -True (yea saying or acquiescence) or False (nay saying) regardless of the content of the item. -TRIN > 79T suggests the profile is invalid and should not be interpreted.

Answer 114

Test underestimates the degree of problems or pathology

Answer 115

(T ≥ 65) 1. The person is deliberately trying to “fake good” and make themselves look unusually well adjusted 2. The person has very little insight into themselves, so that they actually believe they are as well adjusted as they are making themselves look, Naïve and believe it. 3. This one is rare: Certain members of the clergy will obtain high scores on L, possibly because they are unusually careful about not doing anything wrong. -not very good at picking up more sophisticated forms of defensiveness…K scale can do this

Answer 116

-Detect more sophisticated forms of defensiveness -elevate on K underreport problems or symptoms because of: limited insight, defensiveness, or desire to present themselves in a positive light -The effect of the underreporting bias that is picked up by K is to make the person look better adjusted than they actually are.

Answer 117

-Superlative” scale. -There are five major dimensions in the S scale: (1) belief in human goodness; (2) serenity; (3) contentment with life; (4) patience and denial of irritability and anger; (5) denial of moral flaws. -The S scale is particularly useful in detecting underreporting and “faking good” in non-clinical samples (if S ≥ 65).

Answer 118

person is reporting MORE problems or MORE SEVERE problems than they actually have. motivated by some kind of gain (financial or otherwise) or to call attention to their problem/ “cry for help”

Answer 119

Infrequency scale -items that were very rarely endorsed -Reasons: lack of cooperation, inadequate reading level, response bias -Correlated with severe psychiatric disturbances

Answer 120

(Infrequency-psychopathology) Attempts to separate out the effect of psychopathology from other influences on the elevation of the F-scale If F ≥ T score 90: -Fp ≥ 100: PROFILE IS INVALID -Fp < 100: Cautiously interpret profile

Answer 121

created for new content since all F items appear in the first 370 test questions, the Fb scale was created to assess over-reporting and/or inconsistent responding in the rest of the items -NO, if Fb ≥ 90→Do not interpret content and supplemental scales.

Answer 122

More than 30 items not answered VRIN ≥ 80T (inconsistent responding) TRIN ≥ 80T (bias towards answering T or F) F ≥ 90T (over-reporting) If Fp < 100T, may interpret profile with caution If Fb ≥ 90T, do not interpret content, supplemental, and PSY-5 scales (maybe interpret clinical scales) L ≥ 80T (extreme denial) S ≥ 75T (extreme defensiveness)

Answer 123

Between 10-29 items omitted VRIN between 70-79 (some inconsistency) TRIN between 70-79 (some T/F bias) F between 80-89 or Fp between 80-99 (possible over-reporting) L between 65-79: (effort to present self in unrealistically favorable light) K ≥ 65: (defensiveness and minimization of problems) – Check non-K-corrected profile if K is in this range

Answer 124

--general idea of how a person approached the test, if have these, then mention it! Open and non-defensive: F ≤ 80, L < 60, K between 40-60. Mild defensiveness: L or K between 60 and 64 K ≤ 40 (especially if F > 80): self-critical; possible exaggeration of problems

Answer 125

Problems are likely to be severe and characteristics associated with scale or code type are likely to be prominent (higher elevations = more symptoms and greater severity)

Answer 126

Most of the characteristics associated w/ the scale or code type are likely to be present and problems are moderate in severity

Answer 127

Some of the characteristics associated w/ elevation on the scale or code type are likely to apply but some others might not apply. Problems reflected by scale are likely to be mild.

Answer 128

Tendencies associated w/ the scale or code type are present but DO NOT IMPLY PATHOLOGY FOR SCORES IN THIS RANGE.

Answer 129

Within normal limits; scale should not be interpreted

Answer 130

Low Score. Some (not all) of the scales are interpretable at this range

Answer 131

hypochondriasis excessive bodily concern

Answer 132

depression depressive symptoms

Answer 133

hysteria somatization and denial

Answer 134

psychopathic deviate rebelliousness and non conformity

Answer 135

masculinity femininity non conformity to traditional traits and interests associated with ones gender

Answer 136

paranoia suspiciousness and oversensitivity

Answer 137

Psychaesthenia anxiety and self doubt

Answer 138

schizophrenia odd and eccentric thoughts and beliefs

Answer 139

mania overactivity and impulsivity

Answer 140

social introversion shyness and social avoidance

Answer 141

anxiety – generalized anxiety and excessive worry

Answer 142

fears – general fearfulness and specific phobias – condensation of anxiety

Answer 143

obsessiveness – indecisiveness and rumination – closer to axis I OCD

Answer 144

depression – full range of depressive symptoms – low scores do not r/o suicide

Answer 145

health concerns – preoccupation with bodily concerns – redundant to scale 1

Answer 146

bizarre mentation – reflects peculiar and unusual ideation, primarily of a paranoid nature, easily denied by someone who wishes to conceal

Answer 147

anger – high levels of anger and hostility – subscale help determine how anger is expressed and experienced

Answer 148

cynicism – bipolar scale – from naive optimism on low end to mistrustful and negative attitudes on high end

Answer 149

antisocial practices – antisocial behaviors

Answer 150

impatience, competitiveness and compulsivity – measures interpersonal hostility

Answer 151

low self esteem – bipolar dimension – high unrealistically self esteem to low self esteem

Answer 152

social discomfort – also bipolar dimension – extraversion to introversion

Answer 153

family problems – family tension and conflict

Answer 154

work interference – broad range of problems that are likely to interfere with work

Answer 155

negative treatment indicators – indicator as obstacles to effective treatment

Answer 156

MacAndrew’s alcoholism scale - Suspicion drugs are prominent in their life -High scores = extraverted, uninhibited, pleasure seeking -Low scores = timid, non-aggressive, submissive to authority

Answer 157

Addiction potential scale - More emotionally distress, oversensitive, guilt prone

Answer 158

addiction admission scale -High scores openly acknowledge substance abuse behaviors; Low – deny behaviors

Answer 159

If all 3 low – likelihood of substance abuse is low; If all MAC-R and/or APS elevate – substance abuse should be strongly considered; Rare but possible for only ASS elevated = possible substance abuse in the past, but nothing currently

Answer 160

-27/72 = depression -13/31 = Somatic Complaints -49/94 = antisocial features, acting out -68/86 = paranoid psychosis

Answer 161

Demoralization Provides an indication of the overall emotional discomfort the individual is reporting - may describe selves as discouraged, insecure, pessimistic, helpless.

Answer 162

Somatic Complaints Excessively preoccupied with bodily concerns and probably present a diffuse set of health concerns, such as fatigue, weakness, and chronic pain (high levels) - high scores are inclined toward somatization

Answer 163

Low Positive Emotions Individuals who produce elevated scores report a lack of positive emotional engagement in their lives - increased risk of experiencing depression - pessimistic, withdrawn, and passive in social situations - find it difficult to take charge and make decisions

Answer 164

Cynicism Core component of scale 3 (excessive avowal of trust) - high scores reflect higher level of cynicism and endorse that others are untrustworthy, untruthful, uncaring, and exploit others - those with low scores ay be naive, gullible, and overly trusting

Answer 165

Antisocial Behavior Assessment of antisocial tendencies as the questions allow people to acknowledge past and current antisocial behaviors and related family conflict.

Answer 166

ideas of Persecution Individuals with high scores may feel mistreated and picked on, and may have significant difficulties forming trusting relationships - scores above 75T suggest paranoid thinking that may be symptomatic of schizophrenia or delusional disorders.

Answer 167

Dysfunctional Negative Emotions Reflects a tendency to have negative emotional experiences, conceptualized as underlying anxiety, irritability, and other forms of aversive reactivity - high scores are at an increased risk for experiencing anxiety and/or developing anxiety disorders

Answer 168

Aberrant Experiences Describes a variety of sensory, perceptual, cognitive, and motor disturbances that indicate impaired functioning of the self - individuals who produce elevated scores report frank psychotic symptoms that may include visual or olfactory hallucinations, bizarre perceptual experiences, and delusional beliefs involving thought broadcasting - scores equal to or above 75T may meet diagnostic criteria for a schizophrenia, delusional, or schizoaffective disorder - more moderate (65-74) are suggestive of schizotypal characteristics

Answer 169

Hypomanic Activation Items describe a variety of emotions, cognitions, attitudes, and behaviors consistent with hypomanic activation, including thought racing, high energy, heightened mood and self-regard, excitement seeking, irritability, and pugnacity - those with scores equal to or above 75T may be experiencing hypomania or mania and may also meet criteria for a bipolar disorder.

Answer 170

pattern defined by the two (sometimes three) highest clinical scales in the profile, certain code types occur commonly and these have been found to have particular interpretative correlates

Answer 171

within normal limits

Answer 172

only one scale is > 65 and the highest scale is more than 5T higher than second highest scale

Answer 173

consider whether the profile fits guidelines for well defined two point or three point code type

Answer 174

the highest scale is less than 5T higher than the second highest scale, and the second highest scale is at least 5 points higher than the third highest

Answer 175

second highest scale is less than 5T higher than third highest and third highest is more than 5T higher than 4th highest

Answer 176

Correlation Matrix: calculate correlation between each pair of tests in a battery (find correlation between each combination of samples)

Answer 177

Factor loading = correlation between each of the original variables and each factor

Answer 178

factors are correlated with one another factors are not correlated with one another