What is in a dataset? Flashcards
Name 3 datasets and their characteristics
Why Allbus?
In panel, same people get the question –> learning effects, drop out problems if not random, in the end, sample might get less and less representative
Why are they correct?
Mind the sample, if u ask a highly religious group and no longer believe it might be problematic
Validity vs reliability
Are student evaluationof teachers valid?
Can I trust a dataset? Possible errors:
What is internal validity of research design?
What is external validity of research design?
Most common threat of external validity to research design?
Unrepresentative samples
What is construct validity?
- refers to how well you translated a construct into a functioning and operating reality, the operationalisation
- can be sperated into 2 categories (translation validity and criterion-related validity) and six validity types: face validity, content validity, concurrent and predictive validity, and convergent and discriminant validity
What does Translation Validity mean?
Translation validity centres on whether the construct is accurately translated and operationalisation is reflects the true meaning of the construct –> qualitative focus // does it SEEM to measure our construct
Using 1) Face Validity
- Subjective judgment on the
operationalisation of a construct “seems like a good measure of ..” - Even though subjective judgment is needed throughout the research process + liked by testers, face validity weak form of construct validity, we still could be measuring the wrong thing
and 2) Content validity
- Measuring the entire realm of the construct with all its dimensions and aspects (i.e. bad would be covering only psychological effects not also cognitive and physical effects of depression)
- There are basically two ways of assessing content validity: (1) ask a number of questions about the instrument or test; and/or (2) ask the opinion of expert judges in the field
What does Criterion-related Validity entail?
“…degree of correspondence between a test measure and one or more external referents (criteria), usually measured by their correlation.”
1) Concurrent Validity and Predictive Validity
Concurrent validity shows you the extent of the agreement between two measures or assessments taken at the same time. Example: New math test highly correlates with student’s math grades = high concurrent validity
Predictive validity refers to the ability of a test to measure some event or outcome in the future –> selection tests for job performance
2) Convergent and Discriminant Validity
Testing for convergence: Measures that are related to same construct should have high correlation
Testing for divergence: Measures that are not related to same construct should have low or no correlation
What is the sampling error?
& how to reduce it
Even best random sample will be a little different from population –> unavoidable error but we can reduce with good sample design & larger sample
Example: Income of American population and just by pure chance you get Oprah and Zuckerberg –> too many rich people
Errors related to Samling Procedure
≠ Sampling Error
1) Sampling Bias: inaccurate sample frame (might be out of date) OR systematic error due to systmatic under/overrepresentation of certain elements of population, usually only due to non-random sampling i.e. interviews on the street during working hours -> no working population in sample
2) Non-response: Lack of response or specific refusal, also dropout or invalid data –> when random not as bad bc only reduces sample size but becomes more problematic when non-random (i.e. certain subgroups refuse systematically more often) as estimates gets biased
Is Translation Validity enough?
No, we need to establish a nomological network, which is a theoretical model of the linkages between related variables –> Criterion-related Validity
