SSR exam 5 Flashcards

1
Q

testlets

A

multiple items that have the same context

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

facet method

A

item construction by conceptual analysis of the construct (using exising theory/knowledge) every aspect (facet) of the construct is measured in a systematic way

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

validity

A

degree to which a test measures what it is supposed to measure.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

construct validity

A

the degree to which test cores can be interpreted as reflecting a particular psychological construct/ the extentto which relationships between observations are aligned with the nomoloigcal network

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

face validity

A

the degree to which a measure appears to be related to a specific construct, in the judgment of nonexperts such as testtakers and representatives of the legal system

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

content validity

A

degree to which the content of a measure truly reflects the full domain of the construct for which it is being used no more and no less

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

internal structure of a test

A

the way the parts of a test are related to each other

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

factors

A

items that are highly correlated with eachother form clusters of items

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

convergent evidence

A

the degree ot which test scores are correlated with tests of related constructs

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

discriminant evidence

A

the degree to which test scores are uncorrelated with tests of unrelated constructs

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

concurrent validity evidence

A

refers to the degree to whcih test scores are correlated with ther relevant variables that are measured at the same time as the primary test of interest

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

predictive validity evidence

A

refers to the degree to which test scores are correlated with relevant variables that are measured at a future point in time

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

criterion validity

A

extent to which a measure is related to an outcome

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

inductive approach

A

proceeds by examining the associations between test scores and a large set of potentially important and relevant psychological variables

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

deductive approach to validity

A

test evaluation proceeds by deducing a particular hypothesis from the theoretical basis of a construct and then empirical evaluating the accuracy of the hypothesis

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

item writing guidelines

A

dont ask two questions in one, use independent item content (no dependency between questions), avoid items that deliberately deceive test takers, use three options unless it is easy to write plausible distractors, avoid negatives, use one option that is unambiguously the correct or best option, avoid giving clues in the correct answer(long answer),

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

construct validity steps to follow

A
  1. define a construct in terms of a network of relationships
  2. operationalize every cnstruct with a measurement
  3. see if the relationships between measurements are aligned with the theory
  4. if yes : there is support for a theory in question
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

interpretationist account redefine validity

A

as an assessment of evidence with regards to proposed interpreations of the test scores

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

causal account of validity

A

test X is valid for attribute Y when differenes in Y cause differences in X. we have to establish a causal relationship between construct and the test only then can we conclude that a test is valid

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

response processes

A

the match between the psychological process that respondents actually use when completing a measure and the process that they should use

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

test utility

A

refers to using a test as a tool

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

criterion

A

what we want to predict (good employees or good students)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

predictor

A

the test we use to predict

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

predictive validit

A

defined as the correlation between the predictor/test score and the criterion

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
base rate
cut off for the criterion (when we truly know someone is competent). percentage of correct conlcusions when not selecting TP+FN/N
26
selection ratio
cut off for the predictor (at what predictor score we want to select someone)
27
hit rate
(percentage of correct conclusions) TP+TN/N
28
benefit
hit rate-base rate
29
sensitivity
the percentage of eligible candidates that will be selected = TP/(FN+TP)
30
specififcity
percentage of inapt candidates that will be rejected = TN/(TN+FP)
31
two factors influencing the cell values "the proportion satisfactory among those selected"
predictive validity, selection ratio
32
increase validity
higher reliability(more items deleting bad items), use better predictor, less restriction of range
33
principle cmponent analysis
statistical procedure that uses an orthogonal transofmration to converta set of observations of posssibly correlated variables into a set of values of linearly uncorrelated variables called principle components
34
nomonoligical network
network of "meaning" surrouring a construct
35
validity generalization
process of evaluating a tests validity coefficients across a large set of studies
36
shared method variance
the correlatin is positive because it is based on two measures derived from the same source
37
heterotrait-heteromethod correlations
based on measures of different constrructs measured thorugh different methods
38
heterotrait-monomethod correlations
based on measures of different constructs measure through the same method
39
monotrait-heteromethod correlations
based on measures of the same construct measured through different methods
40
monotrait-monomethod correlations
based on measures of the same construct measured through the same method
41
factor analysis
attempts to achieve parsimony by explaining the maximum amount of common variance in a correaltion matrix using the smallest number of explanatory constructs.
42
PCA
ties to explain the maximum amount of total variance (not just common variance) in a correlation matrix by transforming the originial variables that correlate highly with each other
43
factor loading
the coordinate of a variable along a classification axis
44
common factors
factors that explain the correlations between variables
45
unique factors
factors that cannot explain the correlation between variables
46
factor scores
sould be possible to estimate a person's score on a factor, based on their scores for the constituent variables
47
commn variance
some of it will be shared wiht other variables or measures
48
unique variance
some of it will be specific to that measure
49
random/error variance
variance that is specific to one measure but not reliably so
50
communcality
the proportion of common variance present in a variable
51
factor extraction
the process of deciding how many factors to keep
52
scree plot
plotting each eigenvalue (y-axis) against the factor with which it is associated (x-axis)
53
kaisers criterion
to retain factors with eigenvalues greater than 1
54
factor rotation
rotates axis with variables that can be plotted and rotates them such that tvariables are loaded maximally to only one factor
55
orthogonal rotation
we rotate factors while keeping the independent or uncorrelated
56
oblique rotation
allows factors to correlate
57
quartimax rotation
attempts t maximinze the spread of factor loadings for a variable across all factors
58
varimax rotation
it attempts to maximize the dispersion of loadings within factors
59
equamax
hybrid of the otehr two approaches
60
direct oblimin
determines the degree to which factors are allowed to correlate by the value of a constant called delta
61
factor transformation matrix
used to convert the unrotated factor loadings into the rotated ones
62
KMO
varies between 0 and 1. 0 means that the sum of partial correlations is large relative to the sum of correlations indicating diffusion in the pattern of correlations. 1 means indicates that patterns of correlations are relatively compact and so factor analysis should yield distinct and reliable factors.
63
multicollinearity
variables that are very high correlated
64
singularity
variables that are perfectly correlated
65
split-half reliability
splits the scale set into two randomly seleected sets of items
66
utility analysis
a family of tehcniques that entail a cost-benefit analysis designed to yield information relevant to a decision about the usefulness and or practical value of a tool of assessment
67
selection ratio
numerical value that reflects the relationship between the number of people to be hired and the number of peole available to be hired
68
base rate
refers to the percentage of people hired under the existing system for a particular position
69
top-down selection
process of awarding available positions to applicants whereby the higherst scoreri is awarded the first position, the next scorer get the next position
70
hit
correct classification
71
miss
incorrect classification; a mistake
72
hit rate
prop of people that an assdssment tool accurately identifies as possessing or exhibiting a particular trait ability behaviour or attribute
73
miss rate
prop of people that an assdssment tool inaccurately identifies as possessing or exhibiting a particular trait ability behaviour or attribute
74
false positive
specifc type of miss whereby an assessment tool falsely indicates that the testtaker possesses or exhibits a particular trait ability behaviour or atribute
75
false negative
specifc type of miss whereby an assessment tool falsely indicates that the testtaker does ot possesses or exhibits a particular trait ability behaviour or atribute
76
true positive
the test leads test users to a correct identifiction of a test taker who truly has the disorder
77
true negatvie
test leads test users to a correct identification of a test taker who truly does not have the disorder
78
item response theory
provide a way to model the probability that a person with X ablity will b able to perform at a level of Y
79
difficulty in a test item
refers to te attribute of not being easily accomplished, solved or comprehended
80
discrimination
signifies the degree to which an item differentiates among people with higher or lower levels of the trait ability or whatever it is that is being measured.
81
dichotomous test items
test items or questions with three or more alternative respones where only one is scored correct or scored as being consistent wiht a targeted trait or construct
82
polytomous test items
test items or questions with three or more alternative responses where only one is scored correct or scored as being consistent with a targeted trait or other construct.
83
unidimensionality assumption
the set of items measures a single continuous latent construct
84
theta level
reference to the degree of the underlying ability or trait that the testtaker is presumed to bring to the test
85
local independence
the latent trait explains all item correlations/ there is a systematic relationship between all of the test items and that relationship has to do with the theta level of the testtaker
86
assumption of monotonicity
the higher the trait, te higher the expected score / means that the probability of endorsing or selecting an item response indictive of higher levels of theta should increase as the underlying level of theta increases
87
item characteristic curve
the probalistic relationship between a testtaker's response to a test item and that testtaker's level on the latent construct being measured by the test is eexpressed in graphic form
88
maximum performance test
responsees are partly correct or incorrect
89
typical performance test
you respond in a way that s typical for you
90
classical test theory
body of related sychometric theory that predict outcomes of psychological testing such as the difficulty of items or the ability of test takers
91
psychometrics
measrement of phenomena like knowledge abilities attitudes emotions and personal characteristics