validity Flashcards
what is validity?
refers to whether or not a test measures what it intends to measure - indicates the usefulness of the test
what is the aim of validity?
the ability to make accurate inferences from scores on a test, giving meaning to test scores
how are validity and reliability related?
if a test is not valid, the reliability does not matter, if a test is not reliable then it is not valid
what is the difference between reliability and validity?
validity tells you how good a test is for a particular situation, reliability tells how trustworthy a score on a test will be.
what do reliability and validity refer to?
reliability refers to the consistency of a measure
validity refers to the accuracy of the measure
what are the types of validity?
face validity, content validity, criterion validity, construct validity
what is face validity?
when a test on the surface (its face, so to speak) seems to measure what it is supposed to measure
which form of validity is the least scientific and why?
face validity - the least scientific of all measures of validity as it is just the researcher’s opinion if the items look valid or not
what is the main issue with face validity relating to validity?
a test can have good face validity but not really a valid test (doesn’t actually measure what its intended to measure) - test must feel authentic to participants
why is face validity important?
test takers are interested in taking it because it is relevant to them - if participants have doubts about the test, it effects the scores, tests with low face validity usually have low reliability
how does face validity effect reliability?
tests with low face validity usually have low reliability, also important to have good face validity so that those who would like to use the test think that it will measure what it is said to measure
what are the issues with face validity?
doesn’t refer to what is actually being measured, rather what it appears to measure - determined by a review of items, not statistical analysis - insufficient for claiming a test is valid
what is content validity?
the degree to which a test measures an intended content area
how does content validity relate to the domain sampling model?
do the items on the test make up a representative sample of the attribute the test is supposed to measure?
what does content validity aim to do?
ensure correspondence between items on a test and the content domain
how is content validity created when developing a measure?
specifying the content areas covered by the phenomenon when developing the construct definition, writing questionnaires or scale items that are relevant to each of the content areas, developing a measure of the construct that includes the best (most representative) items from each content area
why is content validity important?
content validity is the core of a test. If you do not get this right, your test is not useful since it wouldn’t measure what it says it measures. It’s important to specify the content areas covered, and writing questions/items that are relevant in these content areas
what are the aspects of content validity?
whether the construct is fully represented - if not = construct under-representation, construct irrelevant-variance
what is construct under-representation in content validity?
the test does not capture important components of the construct
what is construct irrelevant-variance in content validity?
when test scores are influenced by things other than the construct the test is supposed to measure
how is content validity established?
judgement by expert judges, use of statistical methods
how is content validity established by expert judges?
judges independently examine the items and decide whether each of the items is weakly relevant or strongly relevant to the content domain of the construct - The value would range from 0 to 1, with higher values indicating better content validity.
what statistical methods of analysis can be used to establish content validity?
factor analysis to assess whether items said to relate to each content area fit well together statistically
what is criterion validity?
how well the test score predicts or estimates the criterion behavior or outcome, now or in the future
what is concurrent criterion validity?
refers to the extent to which the test scores can correctly identify the current state of individuals
how can concurrent validity be established?
give two tests to participants - one a known test and the other the new test and measuring the new test results against the known test results to see the correlation between scores
what is predictive criterion validity?
how well the test performance estimates or predicts current and future performance on some valued measure
how can predictive criterion validity be established?
Predictive criterion validity asks if that score on one test can predict how well you will do in the next test or how well you will handle a task. If you correlate the two scores, a high correlation will indicate high predictive validity.
what is a construct?
A construct is a postulated, hypothetical attribute. something we think exists, but it is not directly observable or measurable
how is construct validity measured?
by looking at the relationship between the construct and other constructs. We ask to what other constructs is this construct similar and to what other constructs is this construct different?
what evidence do we look for in the relationship between one construct and another relating to construct validity?
convergent validity and divergent/discriminative validity - For a test to have good validity, it needs to have evidence for both convergent and discriminant validity
what is convergent validity?
scores on the test must have high correlations with other tests that measure the same construct
what is divergent/discriminative validity?
scores on a test have low correlations with other tests that measure different constructs.
what factors effect validity?
reliability: Any form of measurement error can reduce validity. Importantly, you can have reliability without validity. must demonstrate reliability before validity.
social diversity: Tests may not be equally valid for different social/cultural groups.
