Validity Flashcards
Statement 1: Construct irrelevant variance refers to the variance in a CONSTRUCT that does not covary with TEST scores.
Statement 2: If the variances of a test and the construct it is attempting to measure only overlap by a small degree then the test is likely to have low reliability.
(a) Both statements are true.
(b) Statement 1 true; Statement 2 false.
(c) Statement 1 false; Statement 2 true.
(d) Both statements are false.
The answer was d. See Lecture 4. Statement 1 is false. Construct irrelevant variance refers to the variance in the TEST that does not covary with the CONSTRUCT (not the other way around, as stated in the question). Statement 2 is false. If the variances of a test and the construct it is attempting to measure only overlap by a small degree then the test is likely to have low VALIDITY (not RELIABILITY as it says in the question).
Factors that may affect a predictive validity coefficient do NOT include:
(a) The internal consistency of the criterion.
(b) Certain types of people dropping out of the sample between the original test and when the criterion is measured.
(c) The mean score on the test (assuming no ceiling or floor effects).
(d) Certain types of people agreeing to be in the sample.
The answer was c. See Lecture 4. The value of the mean score won’t affect the magnitude of the correlation (as calculating the correlation coefficient involves standardizing the variables anyway).
Statement 1: If the reliability of both a test and a criterion measure are high then this means the correlation between them should also be high.
Statement 2: Content validity is empirical data supporting the hypothesis that the content of a test is appropriate
(a) Both statements are true.
(b) Statement 1 true; Statement 2 false
(c) Statement 1 false; Statement 2 true.
(d) Both statements are false.
The answer was d. See Lecture 4. Statement 1 – false. If the reliability of both a test and a criterion measure is high then this means the correlation between them is not restricted – however, this doesn’t mean it can’t be small (the correlation can be high or low, depending on the validity of the test). Statement 2 – false. Content validity involves opinions and is not generally based on empirical data.
Statement 1: You can test the incremental validity of a test by seeing whether it can predict some relevant criterion measure in isolation from other measures.
Statement 2: If we had an established intervention known to reduce state anxiety then we potentially could use this to test the validity of a new measure of state anxiety.
Selected Answer:
(a) Both statements are true.
(b) Statement 1 true; Statement 2 false.
(c) Statement 1 false; Statement 2 true.
(d) Both statements are false.
The answer was c. See Lecture 4. Statement 1 is false because incremental validity is about whether a test contributes to predicting some outcome IN ADDITION TO the effect of other measures. Statement 2 is true as we can use the intervention as part of an experiment in which we see if the intervention reduces scores in the new test in the way we would predict if the test is valid (compared with some placebo intervention).
When students complain that, in a course examination, a lecturer did not ask any questions from a particular lecture, they are effectively complaining about the examination having:
(c) Potentially poor content validity.
(a) Potentially poor internal consistency.
(b) Potentially poor convergent validity.
(c) Potentially poor content validity.
(d) Potentially poor criterion validity.
The answer was c. See Lecture 4. The students are effectively arguing that the examination did not cover the content adequately and hence lacked content validity.
Statement 1: Evaluating a test by seeing if it does not correlate highly with a construct it is not supposed to be measuring is an example of deviating validity.
Statement 2: A factor analysis involves mathematically grouping items according to the similarity of their content.
Selected Answer:
(a) Both statements are true.
(b) Statement 1 true; Statement 2 false.
(c) Statement 1 false; Statement 2 true.
(d) Both statements are false.
The answer was d. Statement 1 – false. It’s an example of discriminant or divergent validity. “Deviating validity” is something I just made up and so isn’t an example of anything. Statement 2 – false. A factor analysis involves mathematically grouping items according to their inter-correlations not the similarity of their content (there’s no way the factor analysis can “know” what the content of the items is).
Statement 1: Non-random attrition between two time points in a longitudinal validation study is one of the factors that could potentially compromise the evaluation of the CONCURRENT validity of a test (assuming the test is administered during the initial time point).
Statement 2: In a validation study for a behavioural measure, you discover that self-selection biases in your sample are influencing the spread of scores for the measure. This could compromise the evaluation of the CONCURRENT validity of the test.
(a) Both statements are true.
(b) Statement 1 true; Statement 2 false.
(c) Statement 1 false; Statement 2 true.
(d) Both statements are false.
The answer was c. See Lecture 4. Statement 1 is false because CONCURRENT validity involves administering both the test and criterion measures AT THE SAME TIME. That means if people drop out after the initial time point then it won’t matter as we would have already have collected all the data we needed (though it may affect any evaluation of the test’s PREDICTIVE validity). Statement 2 is true because anything that affects the spread of scores in a test may affect its correlations with other variables (which is what we’re analysing when we evaluate the concurrent validity of a test).
If the validity coefficient between a test and its criterion measure (where a high test score should predict a high criterion score) is -.97 (minus point nine seven) and is statistically significant then this probably indicates:
(a) The test is unlikely to be either valid or reliable.
(b) The test could be valid, but is unlikely to be reliable.
(c) The test could be valid and is also likely to be reliable.
(d) The test could be reliable but is definitely not valid.
The answer was d. The negative correlation indicates that the test has an inverse relationship with the criterion - when it ought to have a positive correlation if it was valid. However the high magnitude of the correlation suggests that the test is probably reliable (if the test was unreliable, the correlation would be limited to something much closer to zero).
Statement 1: It is possible for a test to have excellent reliability but poor validity.
Statement 2: It is possible for a test to have excellent validity but poor reliability.
(a) Both statements are true.
(b) Statement 1 true; Statement 2 false.
(c) Statement 1 false; Statement 2 true.
(d) Both statements are false.
The answer was b. See Lecture 4. Statement 1 is true but statement 2 is false. You don’t need validity to have good reliability (your test can be consistent in the scores it produces without measuring what you want it to). However you do need good reliability to stand a chance of your test being valid (because the level of reliability places a ceiling on how high your validity coefficient can be). To put it another way - if your measure is producing wildly inconsistent scores then it’s probably not measuring anything (let alone what you want it to measure).
Statement 1: If a test has poor face validity then this may have implications for the data that the test yields.
Statement 2: Content validity is not important for a university examination as long as that examination is supported by empirically-based validity evidence.
Selected Answer:
(a) Both statements are true.
(b) Statement 1 true; Statement 2 false.
(c) Statement 1 false; Statement 2 true.
(d) Both statements are false.
The answer was b. See Lecture 4. Statement 1 is true. Poor face validity can lead to things like missing data. Statement 2 is false. Content validity is considered important for university exams, as they’re supposed to be testing knowledge of course content directly. This means that even if we could create a test that discriminated between good and poor students in the course (i.e. it had empirically-based validity), it would still be a problem if it did not do this by measuring knowledge of course content directly.
