test 2

Question 1

what is a latent variable?

Answer 1

another word for construct. Something we are interested in that is hopefully influencing our measurement that get on the basis of our operational definition for our research.

Question 2

Compare the difference between having your data in a circle versus a square.

Answer 2

circle- represents something that we are not directly observing/interested in
square- represents something that is an actual score that we observe.

Question 3

true score vs error

Answer 3

true score; the real, expected influences (hope to have much more of this)
error: undefined/unexpected influences, much less of this

Question 4

explain the idea of the tennis ball

Answer 4

Take 3 flawed measurements and take average. Even if they are all wrong but make them work together. Error is randomly distributed, even if the measurement are wrong.

Question 5

reliability

Answer 5

the degree to which the result of a measurement, calculation, or specification can be depended on to be accurate. CONSISTENCY

Question 6

How do we figure out what is error & what is true score?

Answer 6

We are interested in reliability of a questionnaire as a whole not the reliability of an individual question. Having multiple questions in our questionnaire we can broad our ideas of how to have true score.

Question 7

The classical measurement model assumption 1

Answer 7

The individual items of a questionnaire each have error and true score.

• The amount of error varies randomly
• The mean error across items of 0 (sample sizes)

Question 8

The classical measurement model assumption 2

Answer 8

the error in one item is not correlated with the error in any other item
- why must this be true based on previous assumptions

Question 9

The classical measurement assumption 3

Answer 9

the error in the items is not correlated with the true score

Question 10

Parallel test model

Answer 10

• extends the classical measurement model
  1) the latent variable influences all items equally
• all items/construct correlations are the same
  2) Each item has the same amount of random error
• the combined influences of all other factors are the same

Question 11

Parallel test model assumptions

Answer 11

1) only random error
2) errors are not correlated with each other
3) errors are not correlated with true score
4) latent variable affects all items equally
5) amount of random error for each item is equal

Question 12

to achieve perfect reliability you have to…

Answer 12

eliminate error in your measurement

• extreme measurement error prevents you from observing any associations
• the degree of error in a measure can be estimated by correlating that measure with itself

Question 13

what are the forms of reliability

Answer 13

cronbach’s alpha- most commonly reported. easiest to use with SPPS
Split half- less common, also easy to use
test-retest- optimal, when possible, requires 2x resources
alternate form- even less common, requires 2 identical measures
omega- newest form, doesn’t require tau equivalence

Question 14

Cronbach’s alpha relies on several key assumptions

Answer 14

single factor model

• essential tau-equivalence
• error is random
• equivalent influences of true score
• equivalenet inter-item correlations
• inter-item correlations would all be equal in a large enough sample
• items would have equal variability in a large enough sample.

Question 15

Dropping bad items

Answer 15

If an item is not “equivalent’’ to the others then dropping it will affect alpha

• if it has a low inter-item correlation, Alpha goes up
• if it has a high inter-item correlation, alpha goes down

Question 16

identifying good items

Answer 16

inter-item correlations do not have to be very large

-there can be a lot of error in good items

Question 17

problems with alpha

Answer 17

because of its rigid assumptions, its often an underestimation of reliability
-its just internal consistency
The correlation coefficient
It also assumes normality

Question 18

Deviations of normality

Answer 18

they both threaten the validity of alpha

• Skewness: a longer than normal tail
• Kurtosis: a taller/wider than normal spread

Question 19

Kurtosis vs. Normal

Answer 19

Platykurtic (negative) wider ones with a pulled down peak

Leptokurtic (positive) distributions are tall and skinny

Question 20

Skewness vs. Normal

Answer 20

Positive means right tail is pulled out

Negative means left tail is pulled out

Question 21

three types of validity

Answer 21

a test’s ability to measure what it is suppose to

1) content validity
2) criterion validity
3) construct validity

Question 22

Content validity

Answer 22

when its items cover all aspects of the construct being measured.Key issue: representation of range
What are all the key thoughts, behaviours, skills, etc.
Representation by importance
How much of the total score is represented by each?
Success of representation is usually determined by:
1) advanced with prior literature review
2) Expert review

Question 23

Criterion Validity

Answer 23

refers to the degree to which a measurement can accurately predict validity
evaluation of your measures through comparison with an important “gold standard” outcome;
-delinquency assessment: correlation with numbers of offences
-Job suitability: employees rating on jobs
-Graduate records exams: grad school GPA

Question 24

Construct Validity

Answer 24

is the degree to which a test measures the construct that it is supposed to measure.
Two definitions:
1) An overall category encompassing all other forms of validity
2) The relationship of your measure with others theoretically- relevant measures
Overall category;
-A consideration of all evidence to indicate validity
-Everything we have, and will discuss
- Contains second definition
-Establishes wether we measured the ‘thing’ we wanted
relationship to other measures: convergent validity & discriminant validity

Question 25

Convergent validity

Answer 25

Shows appropriately strong correlation with related constructs

• not facets or the construct you’re measuring
  ex. Anxiety should show positive relations with depression and stress

Question 26

Face validity

Answer 26

does my measure seem to measure the right thing?
-Must consider whether there are other reasonable interpretations.
Very commonly argued for, as its the easiest
-potentially least useful.

Question 27

Internal validity

Answer 27

• It is the extent to which you can account for other plausible explanations
• You need to either rule them out or show they’re actually implausible
• error will again be a concern

Question 28

Error and validity

Answer 28

• We use correlations between our measure & another measure to quantify validity.
• We expect our correlations to be more modest than for reliability assessments
• An R of about .10 is only good for showing discriminant validity
• An R of about .60 is very good for showing convergent validity
• An R of about .85 or higher is probably bad

Question 29

Threats to validity

Answer 29

-Several common sources of error can cause problems (especially when systematic)
Motivation:
-We could end up with incomplete data
-we could have different context effects
-we could end up with only random data
Distractions
- we need our participants to be able to concentrate
Sampling bias
-We need to ensure we’ve sampled the right people
-Criterions validity is often particularly sensitive to maturation threats
-Both reliability and validity can suffer from instrumentation threats
-Order effects: practice, fatigue and boredom