Psych testing quiz 2 Flashcards
Reliability
Reliability is the extent that the results of a test are reproduced. Other similar terms are
consistency, dependability, reproducible, and stable.
Reliability is _____, but not sufficient for validity
Reliability is necessary, but not sufficient for validity
reliability
reliability = the test gives consistent results
Validity
test is actually assessing what it says it will test
usability
usability = it is a test that pragmatically works
Validity and reliability indicate __
Validity and reliability indicate the quality of the instrument
Validity and reliability ___ is most important
Of the two, validity is the most important because if the instrument doesn’t measure what it is suppose to, it does not matter if it does it accurately.
There must be ___ in order to have ____, but a test may be ___ without being ____
There must be RELIABILITY in order to have VALIDITY, but a test may be RELIABLE without being VALID
Validity is
Validity is the extent the test measures what it purports to measure and the
appropriateness with which inferences can be made on the basis of the test results.
Tests are not intrinsically valid or not, rather validity depends upon…
Tests are not intrinsically valid or not, rather validity depends upon the intended
PURPOSE of the test
____ is the single most important attribute of a test.
Validity is the single most important attribute of a test.
____ is a
necessary, but not sufficient condition for validity.
Reliability is a
necessary, but not sufficient condition for validity.
If validity of a test is in question, such doubts should be clearly expressed in the report
true
4 Kinds of validity
face validity
content validity
criterion validity
construct validity
____ validity; a. Items LOOK appropriate for the stated purpose of the test.
b. Least important kind of validity
= Face validity
a. Items LOOK appropriate for the stated purpose of the test.
b. Least important kind of validity.
____ validity;
a. is important when there is no obvious or agreed upon criterion to use in correlation studies.
b. An example would be a study on anxiety. In this case the ___ validity would
be developed by comparing all know information about anxiety with the results of the test.
= construct validity
a. Construct validity is important when there is no obvious or agreed upon criterion to
use in correlation studies.
b. An example would be a study on anxiety. In this case the construct validity would
be developed by comparing all know information about anxiety with the results of the test.
____ validity;
a. Items cover specific content that the test is looking at.
b. is necessary in order to establish Criterion validity.
= content validity
a. Items cover specific content that the test is looking at.
b. Content Validity is necessary in order to establish the next kind of validity, Criterion validity.
____ validity;
a. ____ validity is approximately the same as empirical validity (relationship
shown to be statistically significant).
b. It looks at the “relationship” of the items on a test to an external standard.f
= criterion validity
a. Criterion validity is approximately the same as empirical validity (relationship
shown to be statistically significant).
b. It looks at the “relationship” of the items on a test to an external standard.
Criterion validity is affected by the:
ii. Specific VARIABLES in question as some variables are easier to validate.
iii. Specific CRITERION in question. Some criterion is more important and/or
relevant to purpose of text.
iv. Specific GROUPS taking the test having different characteristics which
result in different validity.
v. Specific SCORES of examines. Increased variability of scores will result in
increased validity.
KindsofCriterionvalidity
i. Concurrent validity is when the scores on a test and the external criterion are compared with data gathered at the same time.
ii. Predictive validity is when the scores on a test and the external criterion are compared with data gathered at different times. The data for the test is gathered at one point and the data for the criterion is something that will happen in the future.
reliability
- Reliability is the extent that the results of a test are reproduced. Other similar terms are
consistency, dependability, reproducible, and stable. - Reliability is necessary, but not sufficient for validity.
The reason a test is not 100% reliable is because there is always some amount of ___
The reason a test is not 100% reliable is because there is always some amount of error.
A test would have perfect reliability (or give same results every time it is used) if it did not
have any errors in the process of creating, administering, scoring and/or interpreting the
test.
_____refers to the extent that the obtained test scores are affected or
how much they vary from the theoretical accurate score.
Error Variance refers to the extent that the obtained test scores are affected or
how much they vary from the theoretical accurate score.
Kinds of reliability
- Test content reliability indicates if each question of the test gives similar answers.
a. Test content reliability is established by comparing test items with other items or with the entire test.
b. Specific names: Alternate forms and Split-half.
c. Reliability increases as the length of the test increases. - Time based reliability indicates if the test gives the same results over a period of time.
a. Specific kind: Test-Retest - Situation based reliability indicates if changes in the environment impact the test
results
a. Environmental or situational reliability looks at things such as temperature, noise,
motivation, etc. Cheating by the examinee can impact the situational reliability.
- ___ reliability indicates if each question of the test gives similar answers.
a. is established by comparing test items with other items or with the entire test.
b. Specific names: Alternate forms and Split-half.
c. Reliability increases as the length of the test increases.
- Test content reliability indicates if each question of the test gives similar answers.
a. Test content reliability is established by comparing test items with other items or with the entire test.
b. Specific names: Alternate forms and Split-half.
c. Reliability increases as the length of the test increases.
- ___ reliability indicates if the test gives the same results over a period of time.
a. Specific kind: Test-Retest
- Time based reliability indicates if the test gives the same results over a period of time.
a. Specific kind: Test-Retest
- ____ reliability indicates if changes in the environment impact the test
results
a. ___ reliability looks at things such as temperature, noise,
motivation, etc. Cheating by the examinee can impact the situational reliability.
- Situation based reliability indicates if changes in the environment impact the test
results
a. Environmental or situational reliability looks at things such as temperature, noise,
motivation, etc. Cheating by the examinee can impact the situational reliability.
Reliability Coefficients
Reliability Coefficients
1. Reliability is expressed by reliability coefficients. The coefficient expresses how likely the
same score is to be repeated.
a.Coefficients range from 0.00 to 1.00
i. Coefficients of .80 or higher are acceptable for most tests of cognitive and special abilities.
iii. Even with the highest possible reliability coefficient, it’s impossible to be certain of a true score due to error.
confidence interval
Confidence Interval
1. Since it is impossible to be certain of the accuracy of a score, results are often given in
ranges, rather than one specific score.
2. This range is called a Confidence Interval for a score. It is a a band or range of scores
a. The Examinee
(The one taking the test)
that has high probability to include the true score.
a. For example if an obtained score is 118, it could be reported as a score likely to fall in between 114 and 122. Obviously this increases the chances that the true score is included somewhere in the range.
b. It is common to report such as score as “118+/-4.”
Norms
C. Once the test has established norms, an individual’s score can then be interpreted by comparing it with others who have taken the test. Norms will be in test manual.
D. The term “norm” means the normal, or average performance. Without a norm or something to compare to, any one score is meaningless.
