Test Construction

Question 1

What is the shape of a percentile rank distribution

Answer 1

rectangular

Question 2

What is the KR-20 method

Answer 2

used for assessing consistency reliability

for test items that are scored dichotomously (right and wrong)

benefit: good for measuring unstable characteristics

Question 3

when using item response theory, the point when the curve intercepts the y axis indicates…

Answer 3

indicates the probability of answering correctly by guessing

Question 4

A tests validity coefficient cannot exceed

Answer 4

the square root of the reliability coefficient

Question 5

_______refers to the proportion of individuals with a disorder that are correctly identified by a test as having the disorder

Answer 5

test sensitivity (true positives)

Question 6

______refers to the proportion of individuals without the disorder who are correctly identified by the test as not having the disorder

Answer 6

test specificity (true negatives)

Question 7

refers to the proportion/percent of individuals who test positive for the disorder and actually have the disorder

Answer 7

positive predictive value

Question 8

refers to the proportion/percent of individual who test negative for the disorder and actually do not have the disorder

Answer 8

negative predictive value

Question 9

Flynn effect

Answer 9

IQ scores have generally increased in developed nations

Question 10

Leiter-3 is best at measuring the intelligence of

Answer 10

children with hearing impairments

Question 11

“testing the limits” refers to what

Answer 11

• Is done to obtain additional qualitative information about the examinee’s test taking/decision makeing
• readministering the test (or portions of the test) to an examinee while modifying the standardized procedures

Question 12

Raven’s Progressive Matrices

Answer 12

a nonverbal measure of general intelligence that is culturally fair

Question 13

Wonderlic Personnel Test measures

Answer 13

brief measure of cognitive ability

Question 14

P .01 means

Answer 14

there is a 1% chance that the null hypothesis will be incorrectly rejected (Type 1 error-false positive)

Question 15

A within-in subjects research design in which you evaluate the effects of an intervention by comparing multiple quantitative observerations of participants before an dafter they are exposed to an intervention

Answer 15

time series

Question 16

A research design that is counterbalanced to control carry-over effects

Answer 16

Latin Square

Question 17

a research design where one group receives a treatment and the other does not

Answer 17

static-group

Question 18

a single case research design that involves comparing the effects of an intervention across two or more settings, behaviors, or participants

Answer 18

multiple baseline design

Question 19

A modeling technique that is used to test causal hypotheses about relationships among measured variables and the latent traits those variables are believed to measure

Answer 19

LISREL structural equation model

Question 20

a causal modeling technique that predicts the causal relationship among measured attributes only

Answer 20

Path analysis

Question 21

sampling that involves first dividing the population into segments and randomly selecting from those segments

Answer 21

Stratified random sampling (gender, ses, race)

Question 22

sampling that involves selecting units rather than individuals

Answer 22

cluster sampling

Question 23

sampling that involves selecting every nth participant from a list of individuals in the population

Answer 23

systematic sampling

Question 24

single subject research design

Answer 24

includes at least one baseline (no txt) phase and one treatment phase

the dv is measured a regular intervals during each phase

Question 25

multivariate technique used when two or more predictors (interval or ratio) are used to predict the status on two ore more continuous criteria

Answer 25

canonical correlation

Question 26

occurs when participants in one group inadvertently receive the intervention that has been applied to participants in another group

Answer 26

Treatment diffusion/spillover

Question 27

when cues in the environment inform participants about the purpose of the study or indicate what behavior is required of them

Answer 27

demand characteristics

Question 28

useful to record behaviors that occur infrequently, have a long duration, and leave a permanent record or product

Answer 28

event recording

Question 29

used to determine how long it takes for a behavior to begin after a specific event has occurred

Answer 29

latency recording

Question 30

useful to record behaviors that have no clear beginning or end

Answer 30

interval recording

Question 31

useful for behaviors that have a clear beginning and end

Answer 31

duration recording

Question 32

what is inter-rater reliability and what stats are used

Answer 32

``` benefits: good for subjective tests assessed by: kappa statistic coefficient of concordance percent agreement-can be inflated ```

Question 33

what indicates the relationship between the likelihood that an examinee will endorse the item and the examinee's level on the attribute measured by the test

Answer 33

item characteristic curve

Question 34

how can you calculate the 95% confidence interval

Answer 34

multiply the standard error by 2

Question 35

an item discrimination index of 0 indicates

Answer 35

the item was answered correctly by the same number of low and high achieving students

Question 36

type II error

Answer 36

failure to reject a false null hypothesis (a "false negative"

Question 37

sources of error for inter-rater reliability

Answer 37

lack of motivation rater biases characteristics of the measurement consensual observer drift-when tow or more observers work together which influences each others ratings

Question 38

what is the most thorough method for estimating reliability

Answer 38

alternate forms

Question 39

factors that affect reliability

Answer 39

test length range of test scores guessing (test content) overall want it to be longer test with a heterogeneous sample

Question 40

what is test-retest reliability

Answer 40

when you test the same group at 2 different times coefficient of stability appropriate for stable attributes (personality)

Question 41

what is alternate forms

Answer 41

similar forms of the test to the same sample correlate versions of the tests appropriate for stable attributes

Question 42

3 ways to measure internal consistency

Answer 42

split 1/2 - test is divided into 2 halves so the examinee obtains two scores Cronbach's coefficient alpha-administering one test to a group; a formula is used to determine the average reliability that would be obtained from all possible splits of the test Kuder-richardson formula 20

Question 43

content validity

Answer 43

test is representing the domain associated with well defined behaviors

Question 44

construct validity

Answer 44

the extent to which an examinee possesses a particular hypothetical trait

Question 45

criterion related validity

Answer 45

test is used to estimate an exmainees performance on an external criterion

Question 46

convergent validity

Answer 46

comparing your test to another test that is know to measure the same construct want convergent validity to be high montrait - heteromethod

Question 47

divergent/discriminant validity

Answer 47

compare your test to another test that doesn't measure the same construct want to be low heterotrait - monomethod

Question 48

monotrait-monomethod

Answer 48

a measure of reliability

Question 49

orthogonal

Answer 49

factors are uncorrelated test communality can be calculated by squaring and adding the test's factor loadings

Question 50

oblique

Answer 50

factors are correlated

Question 51

a squared factor provides a measure of _____

Answer 51

shared variability

Question 52

concurrent and predictive validity are related to what type of validity

Answer 52

criterion related validity

Question 53

concurrent validity

Answer 53

when data are collected prior to or about the same time as the predictor

Question 54

predictive validity

Answer 54

when the criterion is measured after the predictor is administered

Question 55

correction for attenuation formula

Answer 55

leads to overestimation of actual validity used to evaluate predictor validity when reliability of predictor and/or criterion are are perfect

Question 56

criterion of contamination

Answer 56

contamination by the way the criterion was determined inflates the relationship between the predictor and criterion

Question 57

cross validation is associated with ____

Answer 57

shrinkage

Question 58

incremental validity

Answer 58

increase in correct decisions that can be expected if the predictor is used as a decision making tool use a scatterplot

Question 59

norm-referenced interpretation

Answer 59

comparing a test score to scores obtained by people in the normative sample percentile ranks and standard scores

Question 60

disadvantages of percentile ranks

Answer 60

represent an ordinal scale indicate relative position but don't provide info about absolute differences (differences between them vary) may maximize the differences between examinees raw scores who are in the middle of the distribution and compresses those at the extremes

Question 61

standard scores

Answer 61

advantage-allows to make comparisons among different tests

Question 62

criterion referenced score

Answer 62

a percentage score-percentage of test content person answered correctly used for specifying the terminal level of performance