Poli399 Quiz 2 Flashcards
Study
What is validity?
Validity – are we measuring what we are measuring, measurement validity
What is reliability
Reliability – Does our measurement process assign values consistently
What is a measurement error?
Measurement error – difference in the values assigned to observations, attributable to flaw in the measurement
What are systematic errors?
Systematic errors – occurs when our indicator is picking up some other property it is supposed to measure. This type of error systematically biases our results
Systematic –> Validity based
What are random errors?
Random errors – chance fluctuations in the measurement results that do not reflect true differences in the property being measured.
Random error –> Reliability and validity
What is content validity?
Content validity is the measure of how relevant or a measurement is actually what the information actually is.
What is face validity?
The degree to which a procedure, a test or assessment, appears effective in terms of its stated aims.
What is sampling validity?
The degree to which the measurement represents the full range of meaning to its target. And if this measure is complete.
What are some potential problems with face validity?
Relies on subjective judgement.
No replicable rules for evaluating the measure.
What are some potential Problems of sampling validity.
Relies on subjective judgement.
No replicable rules for evaluating the measure.
Difficult to specify the universe of content of abstract concepts.
Harder to represent that content completely.
What is criterion validity?
Is an indicator is valid is there is an empirical correspondence between the results obtained using the indicator and the results obtained using another indicator of the same concept that is already known to be valid.
How does the convergent discriminant approach work?
Requires indicator of at least two different concepts, each measured using at least two different methods.
(Criterion related validity) This form of validation(convergent discriminant approach) raises three questions.
- Why not use the criterion instead?
- How do we know the criterion is valid?
- What if we lack a valid criterion?
I don’t know…
What is construct validity?
And what is this known as?
The degree to which a test measures what it claims, to be measuring.
Does our indicator produce relationships with indicators of other concepts that our theoretical understanding of the target property would lead us to predict?
Construct validity has an acronym for its approach. What is the acronym?
AHEM.
Assume Hypothesis,Evaluate the Measure
What are some potential problems of Construct validity?
Our indicator is not valid.
The theoretical framework is flawed.
The indicators of other concepts were not valid.
What is the solution to the problems of construct validity’s problems?
Conduct Multiple tests.
What are the four ways to assess reliability?
The Test retest method.
Alternative forms/ Split half method.
Coefficient Alpha
Sub sample Method.
What is test retest method?
What are the advantages and disadvantages of the test retest method.
You literally do multiple sets of the method to see if you get the same findings
Best when, your data does not react.
Bad when, feasibility, reactivity, real change in the cases.
What is Alternative forms method, and what are its advantages and disadvantages?
Two forms on the same data.
Best because, no reactivity, no time elapse, no confounding effect from possible changes in the cases
Worst cases, difficulty of ensuring that the two forms
What is the Split half method and what are its advantages and disadvantages?
Reliability is assessed by random dividing the items in half and comparing the results.
Advantages, avoids the problem of having to come up with two parallel forms
Disadvantages
Difficulty coming up with sufficient items
Are two halves really are equilivent
Different splits may lead to different assessments
What is the internal consistency method (Alpha coefficient) and what are its advantages and disadvantages?
The alpha coefficient is based on the average correlation for every possible combination of items into two half tests. Items that produce low correlations are deleted
Advantages
No reactivity
No time elapse
No confounding effect from possible changes in the cases
Feasibility
What is the subsample method and what are its advantages and disadvantages?
Divide the sample randomly into several subsamples. the same items are administered to each subsample and reliability is assessed by the similarity of responses across the subsamples.
No reactivity No time elapse No need to come up with twice as many items as needed Feasibility a large sample size is required
Why is research design so important?
Purpose – to impose controlled restrictions on our observations of the empirical world
Allows the researcher to draw causal inferences with confidence
Defines the domain of generalizability of those inferences