What is an indicator?
An Indicator is an aspect of the construct measured.
What is a measurand?
A measurand is the signal the instrument actually detects.
What is a measure?
A measure is the actual procedure used.
What is a measurement?
A measurement is the reading you take.
What is a measurement range?
A measurement range is conditions within which a measure works.
What is an observation unit?
An observation unit is a unit actually measured.
What is a research unit?
A research unit is the unit that you say things about.
What are the types of reliability?
- internal consistency: if I have parallel items, do they return the same values?
- External consistency: if I have the same measurements under the same circumstances, are my results the same?
What is measurement validity?
- Does the measurement really measure what it is supposed to measure?Does it adequately cover the construct?
- Does it measure all relevant component indicators without systematic bias?
What is face validity?
Face validity is when at first sight there appears to be a logical link between measurement instrument (question, operalisation) and the objective.
What is content validity?
Content validity is when different items of the research instrument and the research instruments themselves each cover all aspects of your constructs. Content validity is judged by experts.
What is predictive validity?
Does it predict a future outcome that theory says it should predict?
What is concurrent validity?
How well an instrument compares with a second assessment.
What is construct validity?
Is it statistically related with other constructs that it should be strongly related with?
Can concurrent validity, predictive validity and construct validity be empirically tested?
Yes
Can you explain regression towards the mean?
It occurs whenever what is actually measured is imperfectly correlated with the underlying trait we’re trying to measure, which results in an imperfect correlation between expressions of this trait over time, even if the trait itself is stable over that time period. For example, I want to measure baseball skills. My baseball skills may be the same this year and next year, but this year all the stars were aligned and I scored 30 home runs. Next year, I turn back to a more typical performance for my skill level, but you may wrongly conclude that either my skill has dropped or your measurement of my baseball skill isn’t reliable. This phenomenon explains the typical disappointing results of stars in whatever discipline after a top year.
What are the common problems with regard to determining reliability of your instrument?
- Regression towards the mean
- Testing/instrument effects: testing a unit influences how it will react to subsequent measurements
- Assumption of equal circumstances may not hold (we may confuse inconsistency of measurement with true change/variation)
- Assumption of equivalence of measures may not hold (parallel items aren’t truly parallel)
What is a hypothesis?
A hypothesis is an idea about what how the is world works, that can be empirically tested.
Why do we use hypothesis?
It gives a focus to our research. It provides a link to theory. And it is a target for falsification (experimental and exploratory).
Where do hypothesis come from?
Research often starts from or generates hypotheses.
What are the steps in hypothesis testing according to CIA?
- Identify the possible hypotheses to be considered
- List significant evidence and arguments for and against each
- Identify which evidence/arguments are most helpful judging each
- Reconsider hypotheses and delete evidence and arguments that have no diagnostic value.
- Draw tentative hypothesis. Proceed empirically by trying to disprove them
- Test sensitivity of conclusion to a few critical items of evidence. Consider the consequences for your analysis if that evidence were wrong, misleading, or subject to a different interpretation.
- Report relative likelihood of all the hypotheses, not just the most likely one. (context +)
- Identify milestones for future observation that may indicate events are taking a different course than expected. (decision support vs. description)
Be very worried if your initial bias is the same as your final conclusion.
What are the different types of hypotheses?
- Non-relational: states the existence/level/condition. E.g. Soil salinity in Wageningen is 1500 ppm
- Correlational: states a relation between variables. E.g. Soil salinity is related to plant growth
- Developmental: states a development of one/more variables in time. E.g. Soil salinity in Wageningen is increasing
- Causal: states a causal relation between variables. E.g. Soil salinity affects plant growth
What are the components of causal hypotheses?
- Independent variable (the cause) = x
- Dependent variables (the effect) = y
What are the types of extraneous variables?
- We distinguish between:
- (Other) Independent variables
- Intervening variables
- Moderating variables
- Confounding variables (big headache for causality)
