Research Methods and Test construction Flashcards
In the context of psychological assessment, the terms “floor” and “ceiling” refer to:
A. the lowest and highest true scores an examinee is likely to have, given his or her obtained predictor score.
B. the lowest and highest scores an examinee is likely to obtain on a criterion, given his or her predictor score.
C. the degree to which a test can discriminate among examinees who have very low levels or very high levels of the characteristic measured by the test.
D. the degree to which a test accurately predicts the criterion scores of examinees who obtain very low scores or very high scores on the test.
A test has limited floor when it cannot discriminate well among examinees who have a low level of the characteristic measured by the test because the test does not include a sufficient number of easy items. In contrast, a test has limited ceiling when it cannot discriminate well among examinees who have a high level of the characteristic measured by the test because it does not include a sufficient number of difficult items.
A psychologist is planning a research study to evaluate the effects of a two-hour online lecture on statistics for improving the statistics knowledge of 35 psychologists who have just started studying for the EPPP. All participants will (1) take a pre-test consisting of 50 multiple-choice statistics questions on Monday, (2) attend the online lecture on Wednesday evening, and (3) take a post-test consisting of 50 multiple-choice statistics questions that are equivalent to the pre-test questions on Friday. To analyze the data she obtains in her study, the researcher will use which of the following?
A. t-test for a single sample
B. t-test for correlated samples
C. two-way ANOVA
D. single-sample chi-square test
The first steps in identifying the appropriate statistical test are to identify the independent and dependent variables and the scale of measurement of the dependent variable. This study’s independent variable is the lecture on statistics and the dependent variable is statistics test score. The dependent variable is measured on a ratio scale, which means that the statistical test will be used to compare the mean scores obtained by the psychologists on the pre- and post-tests. The t-test and ANOVA are both used to compare mean scores, but because there are only two means, the t-test is the appropriate test. To determine which t-test to use, you determine how the means will be obtained: In this study they will be obtained from a single group of subjects, and the t-test for correlated samples is used when two means are obtained from the same group or from two groups that are related in some way.
Which of the following is a culture-reduced measure of fluid intelligence?
A. Kuhlmann-Anderson
B. Raven’s Progressive Matrices
C. Woodcock-Johnson
D. Slosson Intelligence Test
The Raven’s Progressive Matrices (RPM) tests are measures of fluid intelligence and are considered to be culture-reduced because they do not use language and performance does not depend on specific cultural or academic learning. There are three RPM tests: Standard Progressive Matrices, Colored Progressive Matrices, and Advanced Progressive Matrices.
Which of the following is a type of counterbalanced design?
A. Solomon four-group
B. Latin square
C. factorial
D. multiple-baseline
Counterbalancing is used to control order effects that may occur when a within-subjects design is used – i.e., when subjects in each group will receive or participate in all levels of the independent variable. The Latin square is a type of counterbalanced design that ensures that the different levels of the independent variable are assigned to the groups of subjects so that each level appears an equal number of times in each ordinal position.
Which of the following best describes ethical requirements regarding sexual relationships with students?
A. Psychologists are prohibited from becoming sexually involved with current and former students in their department.
B. Psychologists are prohibited from becoming sexually involved with students who are “vulnerable to undue influence.”
C. Psychologists are prohibited from becoming sexually involved with students when doing so creates an unacceptable multiple relationship.
D. Psychologists are prohibited from becoming sexually involved with students over whom they have or are likely to have evaluative authority.
Correct answer
D. Psychologists are prohibited from becoming sexually involved with students over whom they have or are likely to have evaluative authority.
This is the best answer because it’s closest to the language of Standard 7.07 of the APA Ethics Code and Standard II.29 of the Canadian Code of Ethics, which prohibit psychologists from becoming sexually involved with students who are in their departments or over whom they have evaluative authority.
The Generations Study (Bishop et al., 2020) compared the timing and pacing of sexual identity development for lesbian, gay, and bisexual individuals ages 18 to 26, 32 to 43, and 50 to 60. The results of the study found that:
A. members of the three age cohorts reported similar ages of onset for all milestones and similar lengths of time between milestones.
B. members of the youngest age cohort reported the earliest ages of onset for all milestones, but members of all three age cohorts reported similar lengths of time between milestones.
C. members of the youngest age cohort reported the earliest ages of onset for all milestones, but members of the oldest age cohort reported the shortest lengths of time between milestones.
D. members of the youngest age cohort reported the earliest ages of onset for all milestones and the shortest lengths of time between milestones.
Correct answer
D. members of the youngest age cohort reported the earliest ages of onset for all milestones and the shortest lengths of time between milestones.
Participants in the Generations Study were asked about the ages when they first experienced five sexual identity milestones:
1. awareness of same-sex attraction;
2. self-identification as lesbian, gay, or bisexual;
3. same-sex sexual behavior;
4. disclosure as a sexual minority to a straight friend; and
5. Disclosure as a sexual minority to a family member.
Consistent with previous research, it found that members of the youngest age cohort (ages 18 to 26) reported the earliest ages of onset for all milestones and the shortest lengths of time between milestones.
Older adults with mild neurocognitive disorder due to Alzheimer’s disease would most likely obtain the highest score on the WAIS-IV __________ Index and lowest score on the __________ Index.
A. Verbal Comprehension; Working Memory
B. Working Memory; Perceptual Reasoning
C. Perceptual Reasoning; Processing Speed
D. Verbal Comprehension; Processing Speed
Correct answer
D. Verbal Comprehension; Processing Speed
Explanation
The WAIS-IV Technical and Interpretive Manual (Psychological Corporation, 2008) reports the following mean Index scores for individuals with mild Alzheimer’s disease: Verbal Comprehension 86.2, Perceptual Reasoning 85.8, Working Memory 84.3, and Processing Speed 76.6.
A division of labor and a hierarchy of authority are defining characteristics of which of the following organizational theories?
A. Weber’s bureaucracy
B. McGregor’s Theory Y
C. Fiedler’s contingency theory
D. Katz and Kahn’s open-system theory
Correct answer
A. Weber’s bureaucracy
Explanation
As described by Weber (1947), an ideal bureaucracy is characterized by a division of labor, a hierarchy of authority, clearly defined rules and procedures, impersonal relationships based on position, and selection and promotion decisions based on an applicant’s or employee’s technical competence.
Cataplexy, hypocretin deficiency, and short rapid eye movement sleep latency are symptoms of:
A. narcolepsy.
B. sleep apnea.
C. temporal lobe seizures.
D. Tourette’s syndrome.
Correct answer
A. narcolepsy.
Explanation
The DSM-5 diagnosis of narcolepsy requires irrepressible episodes of sleep (daytime naps) with at least one of the following: cataplexy, hypocretin deficiency, or REM sleep latency less than or equal to 15 minutes.
In a normal distribution, a percentile rank of ____ is one standard deviation above the mean of the distribution.
A. 50
B. 68
C. 84
D. 97
Answer C is correct. A percentile rank is a transformed score that indicates the percentage of scores that fall at or below a given score. In a normal distribution, when an examinee’s percentile rank is 84, this means that his/her score is one standard deviation above the mean.
Dr. Oz conducts a study to compare the effects of three treatments (drug, relaxation, and drug plus relaxation) on the systolic blood pressure of patients who have secondary hypertension as the result of three different conditions (tobacco use, chronic alcohol use, or obesity). To analyze the main and interaction effects of treatment and condition on systolic blood pressure, Dr. Oz will use which of the following statistical tests?
A. chi-square test for multiple samples
B. t-test for independent samples
C. two-way ANOVA
D. MANOVA
Answer C is correct. The first two things to do when choosing an inferential statistical test are to identify (1) the independent and dependent variables and (2) the scale of measurement of the dependent variable. This study has two independent variables – treatment and condition – and one dependent variable – systolic blood pressure. You may have had trouble with this question because you were uncertain about the scale of measurement of systolic blood pressure. However, a general rule is that most physical measurements (including systolic blood pressure) represent an interval or ratio scale, which means that the appropriate statistical test will be a t-test or analysis of variance. To choose between the t-test, the two-way ANOVA, and the MANOVA, you have to consider how many independent and dependent variables there are. There are two independent variables, which eliminates the t-test because it’s used when there’s only one independent variable; and there is one dependent variable, which eliminates the MANOVA because it’s used when there are two or more dependent variables. That leaves the two-way ANOVA, which is the correct answer. It’s used when a study includes two independent variables and one dependent variable that’s measured on an interval or ratio scale.
When test scores represent an interval or ratio scale and the distribution of scores is skewed, the best measure of central tendency for the distribution is usually which of the following?
A. mode
B. mean
C. median
D. minuend
Answer C is correct. The choice of the appropriate measure of central tendency not only depends on the scale of measurement of the data but also on several other factors including the shape of the data distribution. When the data represent an interval or ordinal scale, the mean is ordinarily the appropriate measure of central tendency. However, when the distribution is skewed, the mean may provide misleading information because its magnitude is affected by the extreme outliers. Consequently, for a skewed distribution, the median is a better measure of central tendency because it’s not affected by the extreme outliers and is more representative of the typical score in the distribution. (The minuend is the first term in a subtraction problem – e.g., 30 in the problem 30 – 10. It’s NOT something you need to be familiar with for the exam.)
The size of the standard error of the mean increases as:
A. the population standard deviation increases and the sample size decreases.
B. the population standard deviation decreases and the sample size increases.
C. the population standard deviation and the sample mean both decrease.
D. the population standard deviation and the sample mean both increase.
Answer A is correct. The standard error of the mean is the standard deviation of the sampling distribution of the mean and is calculated by dividing the population standard deviation by the square root of the sample size. Consequently, as the population standard deviation (the numerator) increases and the sample size (the denominator) decreases, the standard error of the mean increases.
A research study involves comparing the short- and long-term effects of three brief behavioral treatments for social anxiety disorder by randomly assigning clinic patients who have just received the diagnosis to one of the three treatments and assessing their symptoms before treatment and then one week, four weeks, 12 weeks, and 24 weeks following treatment. Which of the following research designs is being used in this study?
A. counterbalanced
B. mixed
C. between groups
D. within subjects
Answer B is correct. A mixed design is being used when a study has at least two independent variables and one variable is a between groups variable and the other is a within subjects variable. The study described in this question has two independent variables – treatment and time: Treatment is a between subjects variable because each subject will receive only one of the three treatments, while time is a within subjects variable because all subjects will be assessed at five different times.