Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

Elsa AACN > Ch 8 Psychometrics, Test Design and Stats > Flashcards

Ch 8 Psychometrics, Test Design and Stats Flashcards

1
Q

Standard error of measurement

A

SD of random errors around the true score

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

Any test score of an individual consists of what?

A

a true score

a random error

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Random errors around true score have a what?

A

normal distribution and mean of 0 over infinite trials

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Are all errors in tests random?

A

No. Some are systematic

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Descriptive stats describe what

A

quantitatively main features of data collected

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What are measures of central tendency

A

mean, median, mode

interquartile range

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

What are measures of variability

A

SD and variance

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Tighter distribution of variability means?

A

High reliability

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

Kurtosis

A

see where there is a peaked or flat distribution of data

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Leptokurtic distribution means…

A

Peaked distribution of data

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Platykurtic distribution means…

A

Flat distribution of data

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Skew

A

tendency of scores to cluster to higher or lower end of distribution

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

Cluster at Higher end of distribution means positive or negative skew?

A

negative

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

cluster at Lower end of distribution means positive or negative skew?

A

positive

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Examples of generalized linear model (GLM)

A

logistic regression

maximum likelihood

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Examples of general linear model are

A

ANOVA
ANCOVA
linear regression

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

DV follows what distribution in general linear model

A

normal distribution

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

DV follows what distribution in GLM (generalized linear model)?

A

error distribution other than normal

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

Item Response Theory

A

latent response theory
refers to models that explain the relationship between latent traits (unobservable attribute) and manifestations (i.e. observed outcomes)
focuses on item level characteristics rather than on test level characteristics

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

Simplest IRT model - Rasch model

A

built on assumption that the most parsimonious and effective predictor of a trait is the relationship between the difficulty of an item and the ability of a person

used to measure latent traits like attitude or ability; It shows the probability of an individual getting a correct response on a test item.

uses item characteristic curve (ICC)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

Bayesian model has which 3 elements

A

prior probability distribution
likelihood function
available new data

uses probability to represent all uncertainty within the model

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

The 3 elements according to Bayesian model can produce what?

A

posterior probability

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

What makes Bayesian models unique

A

incorporate prior info into a statistical model

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

Normal Distribution is also known as

A

bell curve

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
Q

normal distribution

A

the classic way scores are expected to fall

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
26
Q

Central tendency include

A

mean median mode

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
27
Q

What will alter the rank of mean, median, mode

A

Skewed distributions

if skewed to the left (positively skewed), mode< mean < median

if skewed to the right (negatively skewed), mean < median < mode

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
28
Q

What is SD?

A

square root of variance

spread/dispersion of a dataset relative to its mean

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
29
Q

What is variance?

A

average of squared differences of each observation in a distribution from the mean

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
30
Q

First moment of distribution?

A

Mean

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
31
Q

Second moment of distribution?

A

Variance

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
32
Q

Third moment of distribution?

A

Skewness

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
33
Q

Fourth moment of distribution?

A

Kurtosis

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
34
Q

When are transformations used?

A

to change overall shape of underlying data to address issue of non normal distribution

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
35
Q

Mean of Standard Score

A

100

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
36
Q

SD of standard score

A

15

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
37
Q

Mean of T score

A

50

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
38
Q

SD of T score

A

10

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
39
Q

Mean of Scaled Score

A

10

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
40
Q

SD of scaled score

A

3

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
41
Q

Mean of Z score

A

0

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
42
Q

SD of Z score 1

A

1

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
43
Q

Mean of Stanine

A

5

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
44
Q

SD of stanine

A

2

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
45
Q

Mean of percentile

A

50

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
46
Q

1SD in percentile?

A

34.13%

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
47
Q

2SD in percentile?

A

+13.59%

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
48
Q

3SD in percentile

A

+2.14%

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
49
Q

4SD in percentile

A

+0.13%

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
50
Q

Reliability

A

consistency of results

tells to what degree that individual differences in test scores can be attributed to true differences

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
51
Q

Reliability can be expressed in terms of

A

reliability coefficient 0-1

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
52
Q

Reliability can be expressed as the ratio of…

A

true variance to total variance

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
53
Q

Relationship between reliable measures and sensitivity to change

A

perfectly reliable measures cannot detect change

TRADE OFF between the 2

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
54
Q

Name 5 types of reliability

A 
Test-retest reliability
Alternate forms reliability
Split-half reliability
Inter-item reliability
Interrater reliability
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
55
Q

Test-retest reliability looks at

A

stability of scores on repeated administrations

can be affected by test-retest interval and practice effects

56
Q

Error variance means

A

random fluctuation in performance from one administration to another

57
Q

alternate forms reliability

A

stability of test over time

consistency of response to diff sample of items tapping the same knowledge

58
Q

split half reliability

A

internal consistency
split test in diff ways using a single admin
correlation between half of the test scores and the other half

59
Q

inter item reliability

A

Kuder richardson formula

consistency between multiple items measuring the same construct

60
Q

interrater reliability

A

scoring of same test material by different scorer

61
Q

4 validity types

A

content validity
predictive validity
concurrent validity
construct validity

62
Q

content validity

A

The extent to which a measure is a representative sample of the subject matter or behavior under investigation.

63
Q

construct validity

A

The extent to which a measure accurately assesses the construct or latent attribute that it is intended to measure.

64
Q

concurrent validity

A

The extent to which the results of a test or measurement correspond to those of a previously established and related measure, collected at the same point in time

65
Q

predictive validity

A

The degree to which a test score predicts future behavior or performance on an accepted criterion measure

66
Q

structured equation modeling SEM

A

Any of a range of multivariate statistical analysis methods which examine the structural relationship between measured and latent variables.

67
Q

test sensitivity

A

The ability of a test to correctly classify an individual as having a disease or condition.

68
Q

test specificity

A

The ability of a test to correctly determine the absence of a disease or condition.

69
Q

item response theory

A

A statistical theory and a set of related methods which model the relationship between test item performance, test taker ability, and test item characteristics.

70
Q

principal component analysis

A

A statistical method for reducing the dimensionality of a data set of interrelated variables into its underlying dimensions, or principal components, using orthogonal rotation.

71
Q

exploratory factor analysis

A

A form of factor analysis used to explore the possible underlying factor structure and latent constructs of a set of observed variables, without a predetermined model.

72
Q

Confirmatory Factor Analysis

A

A form of factor analysis which is used to verify the fit of a hypothesized factor structure of observed variables and their underlying latent constructs.

73
Q

Threats to validity

A 
history 
testing interval
order of test admin
regression to the mean
multiple comparisons
situational variables
74
Q

Threat to internal validity - History

A

education, reading, age, handedness, gender, race

75
Q

Threat to internal validity - testing interval

A

artifact from duration of testing or practice effects

76
Q

Threat to internal validity - order of test admin

A

fatigue

exposure to test before

77
Q

Threat to external validity - regression to the mean

A
  • samples far from the mean on the first set of scores will be closer to the mean on the second set
  • random variance affecting the samples in the second measurement is independent of the random variance affecting the first
78
Q

Threat to external validity - multiple comparisons

A

comparing time 1 and time 2; differences may be due to chance and not a clinically relevant finding

79
Q

Threat to external validity - situation variable

A

medication, mood, effort, sleep etc

80
Q

Construct validity evaluated via which 2 techniques?

A

convergent

discriminant

81
Q

convergent validity

A

when 2 or more approaches to measurement of some trait are positive correlated

82
Q

divergent validity

A

low correlation b/w 2 similar approaches to measurements of different traits

83
Q

multitrait multimethod matrix

A

composition of correlation coefficients of 2 or more traits and 2 or more methods
contains 4 types of correlation

purpose is to measure construct validity

84
Q

positive predictive power

A

proportion of the time we are right when we state that a condition is present based on a test result

85
Q

negative predictive power

A

proportion of time we are right in stating on the basis of our test that someone does NOT have a condition

86
Q

likelihood ratio

A

likelihood that a given test result would be expected in a patient with the target disorder compared to the likelihood that that same result would be expected in a patient without the target disorder.
used for interpreting diagnostic tests

LR tells you 1) how likely a patient has a disease or condition, 2) the utility of a diagnostic test

The higher the ratio, the more likely they have the disease or condition.

87
Q

likelihood ratio of 1

A

test result is just as likely in those with and without condition - useless test result

88
Q

positive likelihood ratio values > 1

A

positive test result is indicative of the presence of the condition

89
Q

what is a desired negative likelihood ratio?

A

between 0 and 1

90
Q

as the likelihood ratiom moves further away from 1, it means that…

A

the test provides more useful information in detection of a specific condition

91
Q

what does an optimal likelihood ratio depend on?

A

the test or measurement characteristics

92
Q

benefit of using likelihood ratios over predictive values?

A

prevalence of the condition does not affect the statistic

93
Q

pre-test probability

A

the estimated probability that a patient has a condition prior to knowing a test result
- base rate of a condition

94
Q

post-test probability

A

probability that the patient has the condition given a positive test result
how well the test rules in the condition

95
Q

incremental validity

A

the extent to which the use of the test improves post-test probability with respect to the pre-test probability

96
Q

Goal for performance validity test

A

maximize specificity

97
Q

Goal for identifying persons showing any degree of impairment in a domain of function

A

maximize sensitivity

98
Q

Receiver operating characteristic (ROC)

A

visualize the performance of a test by creating a plot of sensitivity and 1 - specificity

The ability of a test to discriminate diseased cases from normal cases

99
Q

Area under ROC curve

A

AUC - measure that reflects overall accuracy of test’s predictions and can compare detection accuracy of assessment tool

100
Q

SEM

A

measure of error variance around a single true score

101
Q

how to calculate SEM

A

SD of the error distribution around true score

takes into account the SD of the test and test’s reliability coefficient

102
Q

What happens to SEM and SEE when reliability increases?

A

SEM and SEE decrease

103
Q

What happens to CI if reliability is very poor

A

it is very large

104
Q

2 things to determine whether norms are psychometrically sound

A

is it normally distributed in the population

is the standardization sample representative of the population that I am comparing my pt’s performance to?

105
Q

low score interpretation for a specific domain is based on the assumption of

A

central limit theorem

106
Q

reliable change index

A

minimum magnitude of change required for psychometric certainty that 2 scores actually differ

107
Q

diff b/w efficacy and effectiveness

A

parallels the difference between statistical significance and clinically meaningful difference

108
Q

efficacy

A

stat difference - whether an intervention produces expected result under ideal circumstances

109
Q

effectiveness

A

clinical meaningfulness - benefit of an intervention under real world conditions

110
Q

calculation of reliable change index

A

uses SE of difference and computes z score for the difference between the individual’s tests based on normal probability distribution

111
Q

RCI needs to fall within what range to reflect significant difference?

A

+/- 1.96

112
Q

discriminant analysis

A

the process of using a score profile to decide whether a patient belongs to one group or another

113
Q

descriptive discriminant analysis

A

to describe differences between 2 or more groups on a set of measure

114
Q

predictive discriminant analysis

A

to classify subjects into groups on the basis of a set of measures

115
Q

predictive discriminant analysis

A

to classify subjects into groups on the basis of a set of measures

116
Q

How is regression used in interpreting test scores?

A

estimate premorbid levels

assess change in functioning (predict retest score)

117
Q

logistic regression in test interpretation

A

allows one to determine the probability that a score belongs to one group or another
generate formulas using multiple variables from one test to differentiate between groups

118
Q

prevalence is also known as

A

base rate

119
Q

what is base rate

A

total no of cases of a particular phenomenon that develop within a given period

120
Q

signal detection theory

A

use in characterizing response styles in recognition memory testing

121
Q

standard error of measurement

A

measure of variability of scores obtained on a test relative to true score
reliable test has small standard error of measurement

122
Q

standard error of the mean (SEM)

A

whether sample mean varies from sample to sample around the true mean of the population
SD of the sampling distribution sample mean
larger sample -> smaller SE of the mean
SD divided by sq rt of sample size

123
Q

difference between SD and SE

A 
SD is used to figure out how “spread out” a data set is
Standard error (SE) or Standard Error of the Mean (SEM) is used to estimate a population's mean.
124
Q

Difference between generalized linear model and general linear model

A

The general linear model requires that the response variable follows the normal distribution whilst the generalized linear model is an extension of the general linear model that allows the specification of models whose response variable follows different distributions

125
Q

difference between SD and variance

A

Variance is a measure of how data points vary from the mean

SD is the measure of the distribution of statistical data.

126
Q

Sn-Nout

A

Negative test score on a high Sensitivity test rules OUT a diagnosis

127
Q

Sp-Pin

A

Positive test score on a high Specificity test rules IN a diagnosis

128
Q

PPV tells us

A

the likelihood that someone has the condition when the test score is positive for the condition

129
Q

NPV tells us

A

the likelihood that someone does NOT have the condition when the test score is negative for the condition

130
Q

Prevalence rates affect

A

PPV and NPV

131
Q

Prevalence rates do not affect

A

sensitivity and specificity of a measure

132
Q

When prevalence rate of a disease decreases

A

PPV also decreases

133
Q

post test probability of a positive test is equal to

A

PPV

134
Q

Pre test probability is the likelihood of…

A

having the disease before performing the test

135
Q

Parametric statistical modeling

A

define SOC based on central limit theorem
estimate performance of an individual relative to a group
norm referenced

136
Q

Bayesian statistical modeling

A

define SOC to develop individual comparison standard

137
Q

Pathognomonic sign

A

a sign whose presence means that a particular disease condition, or impairment is present beyond any doubt
(e.g. apraxia, aphasia, agnosia, hemiparesis, spatial neglect)

Elsa AACN (35 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions