Midterm 2

Question 1

Reliability - Definition

Answer 1

We can trust to measure each person in approximately the same way each time.

Question 2

What is a reliability coefficient + example?

Answer 2

A correlation between two sets of scores (statistical evaluation of reliability), f.ex. test-Retest, alternate forms, internal consistency

Question 3

What is the practice effect?

Answer 3

In relation to Test-Retest: the person scores quicker and more efficiently the second time (when close to real trial)

Question 4

What is the order effect?

Answer 4

Changes in test scores might occur, resulting from the order in which the tests were taken (f.ex. social desirability the 2nd time)

Question 5

What is scorer reliability/interscorer agreement?

Answer 5

The amount of consistency among scorer’s judgement

Question 6

What is intrascorer reliability?

Answer 6

Clinicians being consistent in way he or she assigned scores from test to test

Question 7

What is the confidence interval?

Answer 7

Range of scores that we feel confident will include the test taker’s true score (95% sure f.ex.)

Question 8

What is frequency distribution?

Answer 8

Orderly arrangement of a group of numbers (represented by histogram f.ex.)

Question 9

What does a test-retest method assume?

Answer 9

That a trait stays stable over time (test taker’s skills and qualities), but mood, fatigue, problems can change.

Question 10

What are the disadvantages of the test-retest method?

Answer 10

People can check answers, practice effects, and it lowers correlation

Question 11

What are the disadvantages of the alternate form method?

Answer 11

It is hard to construct, time consuming

Question 12

What are three examples of internal consistency measures?

Answer 12

Split-half method, Kuder Richardson Method KR 20, Cronbach’s Alpha

Question 13

What are advantages and disadvantages of the Split half methods?

Answer 13

ADV: time efficient (one sitting), DISADV: not applicable in heterogenous tests, reliability only based on 50% of test

Question 14

What is KR-20 + DISADV?

Answer 14

Measures consistency of objective right/wrong tests. DISADV: does not work when items = unequal difficulty, and does not work for non-objective tests

Question 15

What is Cronbachs’ Alpha?

Answer 15

Uses all possible half test version s for Likert-type (non-objective tests) to measure reliability

Question 16

Validity - Definition

Answer 16

Are the inferences I am going to draw from a person’s score on a test appropriate? (does it measure what it’s supposed to measure)

Question 17

What is validity as a process?

Answer 17

1. Built from the start/test development (existing lit)
2. Evidence based on content (contact experts, ppl in the situation)
3. Evidence based on internal structure (similar variables relation)
   4.Evidence based on consequences of testing

Question 18

Factor analysis - Definition

Answer 18

A statistical method that allows us to show that scale has number of dimensions that you hypothesized (items in related dimensions correlate with each other)

Question 19

What are the level of measurements?

Answer 19

1. Nominal (categories)
2. Ordinal (rank-order, distances vary)
3. Interval (rank-order, equal distance, no zero point, always agree sometimes agree f.ex.)
4. Ratio (rank order, distance same, has zero point, how many children f.ex.)

Question 20

What is a norm group?

Answer 20

We can compare converted scores with those from what we call norm group (previously tested)

Question 21

Variance - definition

Answer 21

Measures if individual scores tend to be similar or different from mean.

Question 22

What is the most common correlation coefficient & explain

Answer 22

Pearson product-moment correlation = Measures linear association between 2 variables measured on interval/ratio sclaes

Question 23

What is reliability according to the classical test theory?

Answer 23

The true-score variance divided by total observed score variance.

Question 24

What are parallel forms?

Answer 24

Refers to two tests that have certain identical and hard to achieve statistical properties.

Question 25

What is Cohen’s Kappa?

Answer 25

For nominal/ordinal data, it is an index of how consistent scorers are (interrater agreement)

Question 26

What is the standard error of measurement?

Answer 26

SEM = index of amount of inconsistency or error expected in observed test score (how much it differs from T)

Question 27

What factors influence reliability?

Answer 27

Test itself, test administrators, test scoring, test taker.

Question 28

What is the generalisability theory for reliability?

Answer 28

Theory of how well and when we can generalize estimation of reliability from on test to another, or same test in different circumstances.

Question 29

What are the 5 sources of evidence for validity?

Answer 29

Evidence based on test content, response processes, internal structure, relations with other variables , consequences of testing.

Question 30

How can we demonstrate evidence of validity based on test content DURING test development?

Answer 30

By defining the testing universe (body of knowledge), developing test specifications, carefully constructing test questions.

Question 31

How can we demonstrate evidence of validity based on test content AFTER test development?

Answer 31

By examining the extent to which experts agree on relevance of test item content (f.ex. calculating content validity ratio, 1.00- -1.00)