Week 3 (Reliability)

Question 1

Psychological test definition

Answer 1

A systematic procedure for obtaining samples of behavior, relevant to cognitive, affective, or interpersonal functioning, and for scoring and evaluating those samples according to standards

Question 2

Why is validity/reliability important

Answer 2

Eliminating error

Question 3

Why is standardization important

Answer 3

Ability to generalize findings

Question 4

Frequency distribution

Answer 4

A way to organize a set of data by displaying the frequency of various outcomes in a sample

Grouping scores by intervals

Question 5

Central tendency

Answer 5

Where does the bulk of the data lie? The average (mean, median, and mode)

Question 6

Variability

Answer 6

Way of understanding how scores are different, or how much scatter there is in a set of scores

Ways to measure are range, variance, and standard deviation

If scores vary greatly from the mean, variance is big and vice versa.

Question 7

Standard deviation

Answer 7

Numerical value used to indicate how widely individuals in a group vary. If scores vary greatly from the mean, standard deviation is big and vice versa

Square root of variance.

Question 8

Continuous variables

Answer 8

Example IQ

Question 9

Categorical variables

Answer 9

Example T/F, gender

Question 10

Confidence intervals

Answer 10

Measurement error, have standard error of measurement to account for level of unavoidable testing error

Question 11

Levels and scales 4)

Answer 11

Nominal (numbers that don’t actually have to do w scale, goes w categorical variables)

Ordinal (a ranking of variables)

Interval (difference between #’s should be the same)

Ratio (has natural 0 point on scale)

Question 12

Correlation

Answer 12

How something relates to something, NOT CAUSATION

Question 13

Normal curve

Answer 13

Bell shaped

Limits extend to infinity

Happens when mean=median=mode

Unimodal and symmetrical

A particular shape of a frequency distribution

Question 14

IQ curve

Answer 14

Standard deviation for IQ is 15 (this is 1 standard deviation point)

95% of population is within two standard deviations of the mean

68% of population within one standard deviation of the mean

LOOK AT SLIDE 16

Question 15

Standardization

Answer 15

Uniformity of procedures and uniformity in how tests are scored

Components are administration, content, and scores

Makes sure test is given and scored the same

Question 16

Standardization of content

Answer 16

Inter item consistency (do items meant to measure a similar thing yield similar responses?)

Item characteristics (level of difficulty, ability of items to discriminate)

Question 17

Standardization of scores

Answer 17

The meaning of the responses, how to convert raw score to something clinically meaningful, how to reflect performance

Norm referenced testing (group of representative individuals who take a test, represent typical behavior

Interpreting scores is comparing individual scores to the norm sample

Question 18

Norm reference testing score interpretation

Answer 18

Within group norms are expressed by percentiles, standard scores (deviation of scores from the mean)

Question 19

Flynn effect

Answer 19

Gradual improvement in intelligence test scores over last several decades

Question 20

Criterion reference testing

Answer 20

Comparing a persons performance to a predetermined criterion or standard

Goal: evaluate competence in terms of pre established standard

Question 21

Psychological test

Answer 21

Objective and standardized measure of a sample of behavior

Question 22

Reliability

Answer 22

Way to document a tests standardization and absence of error

Refers to consistency of the data or results

Refers to the variation among scores that has to do w the trueness of the scores (how close is obtained score to true score?)

Question 23

Testing reliability, classical measurement (test) theory

Answer 23

Theory of testing based on the idea that a persons observed or obtained score on a test is the sun of a true score (error free score) and an error score

X0= X (true) + X(error)

Reliability coefficient (r) is the ratio of true score variance to total test score variance

If test had no error reliability would be 1

Question 24

Sources of error (3)

Answer 24

Context (location, distractions, tech issues, what is the person there for)

Test taker (language barrier, disability, sleep, hunger, health, cultural barrier

Test itself (unclear language, consistency of administration, scoring, items themselves, group norms)

Question 25

Sources of error within test itself

Answer 25

Time sampling error (variability due to time/natural changes) ex time of day

Content sampling error (selection of items does not adequately cover the content that it is supposed to be testing) ex cultural barrier

Inter item inconsistency (error in scores that results from fluctuations in items across the test) ex consistency of testing the same thing

Content heterogeneity (results from inclusion of items or sets of items that tap content knowledge that differ from those tapped by other items in the same test)

Question 26

How to test for time sampling error

Answer 26

Test retest reliability

Question 27

How to test for content sampling error

Answer 27

Alternate form reliability

Question 28

How to test for inter item inconsistency and heterogeneity error

Answer 28

Internal consistency (split half reliability, coefficient alpha)

Question 29

Split half reliability

Answer 29

Splitting the same test in half and comparing the two halves

Measure of content sampling and internal inconsistency

Question 30

K-R-20 used …

Answer 30

To measure internal consistency, only used when the choices are two (t/f)

Question 31

Measuring internal consistency with kr20 and coefficient alpha

Answer 31

The correlation between performance on all the items within the test

Looks at individual items in test to see correlation between all items. Correlations then get averaged for the score

Question 32

Standard error of measurement

Answer 32

Way to apply what we know about the reliability of a test to someone’s test score

Creates confidence interval around test score

Question 33

As standard error of measurement goes down, reliability goes

Answer 33

Up