Research and Program Evaluation Flashcards
A group counselor decides to make graphs of the psychological test data for all of her clients over the last year. When she plots the distribution of Beck Depression Inventory data, she notices it’s skewed to the left. Which of the following statements is true if a distribution is skewed to the left:
Select one:
A.
the mean is greater than the median
B.
the mean is less than the median
C.
the mean and the median are the same value
D.
the mode is the best measure of center
Feedback
B.
the mean is less than the median
When a distribution of data is skewed to the left, it means that although most of the scores are to the right of the distribution, there are a few extreme low scores pulling the distribution to the left. The mean is more impacted by extreme scores. Because the extreme scores in this distribution are low, the value of the mean will be made less than the value of the median.
A researcher is interested in the education level of people who seek counseling at a particular substance abuse treatment center. His data are reported as a ratio that compares the expected number of people in different categories of education level who go to the treatment center to the actual number of people in the different categories who go to the center. What statistical test would most likely be used to analyze this data:
Select one:
A.
chi-square
B.
t-test
C.
analysis of variance
D.
Pearson r correlation coefficient
A.
chi-square
A chi-square test is based on the ratio of actual outcomes to expected outcomes. An analysis of variance (ANOVA), t-test and Pearson r all require data on which you can compute means and standard deviations. If the researcher is considering just the number of people in different categories, as opposed to computing a mean (on a test score or some other variable), the analysis is more likely to require the use of a test like the chi-square.
A researcher is interested in comparing the effectiveness of two types of counseling to treat career-related depression. A cognitive approach is used at one career counseling office. A behavioral approach is used at another career counseling office. During the course of the study, one of the cognitive counselors quits her position and her clients have to be treated by a new counselor. At the conclusion of the study, results indicate that the behavioral approach works better. Which of the following provides the most likely alternative explanation for this result:
Select one:
A.
selection
B.
attrition
C.
history
D.
maturation
C.
history
History describes an event external to the research situation (i.e., an unplanned incident, a natural disaster, etc.). The fact that the counselor quit could have slowed the progress of the cognitive group, thus threatening the validity of the study.
A researcher is interested in comparing the effectiveness of two types of counseling to treat career-related depression. A cognitive approach is used at one career counseling office. A behavioral approach is used at another career counseling office. During the course of the study, one of the cognitive counselors quits her position and her clients have to be treated by a new counselor. At the conclusion of the study, results indicate that the behavioral approach works better. This will impact the study’s validity:
Select one:
A.
criterion-related
B.
content
C.
external
D.
internal
D.
internal
History is a threat to internal validity. You should have at least narrowed your answers down to “C” and “D” which both deal with the validity of research. The other two choices deal with the validity of tests and what was described in this question was a research situation.
To test the hypothesis that season of the year (winter, spring, summer, fall) affects the propensity of males and females to call a community hotline, a researcher records the gender of callers to the hotline for three weeks during each season for one year. The researcher would use which of the following tests to analyze the data she has collected:
Select one:
A.
t-test
B.
chi-square test
C.
ANOVA
D.
multiple regression
B.
chi-square test
A chi-square test is used when you want to compare the actual number of people (or percentage) of individuals belonging to different categories with the expected numbers belonging to the categories. Also, keep in mind that in order to use a t-test or ANOVA, you must have data on which you could calculate means and standard deviations on. The data in the question consists of simply counting the number of people in different categories and there is no calculation of means and standard deviations.
Which of the following statements regarding program evaluation is true:
Select one:
A.
program parameters should be specified prior to data collection techniques
B.
data collection techniques should be specified prior to program parameters
C.
program parameters and data collection techniques should be dealt with at the very end of the evaluation
D.
program parameters and data collection techniques should be specified at the same time
A.
program parameters should be specified prior to data collection techniques
The four steps of program evaluation (or evaluation research as it’s sometimes called) are as follows: 1. Specify program objectives; 2. Specify program parameters (how will program objectives be reached?); 3. Specify data collection procedures (how will the program objectives be measured or evaluated); and 4. Collect the data and perform the evaluation.
A career development researcher is interested in describing how people perform on a test that measures job satisfaction. He wants to use a measure of central tendency that is resistant to extreme scores. Which measure of central tendency should he not use:
Select one:
A.
mean
B.
median
C.
mode
D.
standard deviation
A.
mean
The researcher should not use the mean, which is a measure of central tendency that is not resistant to extreme scores (i.e., it is affected by extreme scores). The median and mode are not as affected by extreme scores. The standard deviation is not a measure of central tendency.
An experimenter wants to assess the effectiveness of a training course for improving SAT scores by comparing the pretest and posttest scores of a group of high school students. To analyze the data obtained in this study, the experimenter should use which statistical test:
Select one:
A.
factorial ANOVA
B.
correlation
C.
t-test
D.
chi-square
C.
t-test
Whenever you are comparing two means, a t-test is used as the statistical test. In this case, even though the two different sets of scores are calculated on the same set of people, a t-test can still be utilized.
Which of the following research designs is likely to offer the most internal validity:
Select one:
A.
one group pretest-posttest design
B.
one group time series design
C.
non-randomized control group design
D.
randomized control group design
D.
randomized control group design
Randomization is the most powerful method of control available to a researcher. Threats to internal validity are controlled by including a comparison (control) group and randomly assigning subjects to groups.
You are hired by a large insurance company to help improve sales. Two groups of employees are given a motivational pep talk that is either direct or indirect. Three groups are given either high doses, moderate doses, or low doses of a new drug designed to increase the performance of salespeople. Two groups attend a class in sales techniques that is taught either by an educator or an experienced salesperson. Afterwards, all subjects take a test of attitudes towards their jobs and all of their sales, in dollar amounts, are compared to previous levels. Which of the following is an independent variable in this study: Select one:
A.
sales in dollar amounts
B.
qualifications of the teacher of the class
C.
low dosage of the drug
D.
attitude towards sales performance
B.
qualifications of the teacher of the class
The independent variables are the pep talks (i.e., direct or indirect), drugs, and the qualifications of the teacher (i.e., educator or salesperson). The dependent variables are the attitudes toward the job and the sales in dollar amounts.
A counselor works in an agency that helps children who have been abused by their parents. She wants to conduct a study to help her determine if group therapy is more effective than individual therapy. She decides to conduct the study with forty children. The counselor should:
Select one:
A.
randomly assign the children to each type of therapy
B.
allow the children to volunteer for either group or individual therapy
C.
conduct group therapy with all of the children, and do individual therapy with half of them
D.
conduct group therapy and individual therapy simultaneously, and compare results with a control group
A.
randomly assign the children to each type of therapy
To determine which therapy is more effective, The counselor needs groups that are equivalent in every respect except for their status on the independent variable (in this case, the type of therapy). Random assignment is designed to ensure such equivalency. If subjects are assigned randomly to groups, extraneous variables (e.g., age, gender, intelligence) that may affect the dependent variable should be distributed about equally across all groups. Thus, groups should be equivalent except in regards to their status on the independent variable, and the independent variable, rather than an extraneous variable, will be the cause of any differences between the groups on the dependent variable.
A student of a graduate program in counseling is required to do research to complete her degree. She recruits undergraduates for her study designed to measure the effects of test-taking anxiety on test performance. She has two groups of students in her study. In the experimental group, an experimenter tells subjects that the test they will take is an assessment of their worth as human beings. In the control group, the experimenter does not say anything about the test. The student assigns the first 20 people who sign up to the control group and the next 20 people who sign up to the experimental group. The student obtains a significant result and describes her study to her advisor, professor Jon G., who tells her that her research is “devoid of scientific value.” Dr. G.’s criticism is probably based on the fact that:
Select one:
A.
the research she conducted is unethical
B.
the failure to randomly assign subjects threatened the internal validity of this research
C.
order effects will limit her ability to generalize her results to the population
D.
some professors have a psychological need to demean and humiliate their students
B.
the failure to randomly assign subjects threatened the internal validity of this research
In this case, the subjects were not assigned randomly to groups; rather, the first 20 students were assigned to one group and the next 20 were assigned to another. Thus, the research’s internal validity, or the conclusion that there is a causal relationship between the study’s variables, is threatened. For instance, students in the control group might have performed better because they are more motivated (as reflected by the fact that they signed up first), rather than because they were not given anxiety-evoking instructions. The student, therefore, cannot conclude with certainty that variability in the independent variable (the anxiety-evoking instructions) caused the differences between groups on the dependent variable (test performance). (c) Order effects can occur when a study involves exposing each subject to two or more levels of an independent variable. The effects of one level of the independent variable can be affected by previous exposure to another level.
You are interested in conducting a study to evaluate the effectiveness of a particular treatment. Which of the following methods would be most effective:
Select one:
A.
observation
B.
process recording
C.
case study
D.
experimental research
D.
experimental research
Of all the types of research, experimental research permits the most confidence in causal conclusions about the relationship between variables. (a) Observation Studies involve observing behavior in a systematic way and is used in field studies. Note that process recording (b) is a method of record keeping. (c) Case Studies involve an in-depth study of a single individual, community, or institution. Their results can not be generalized.
How is a z-score determined:
Select one:
A.
subtract the raw score from the mean and then divide the resulting number by the standard deviation
B.
subtract the mean from the raw score and then divide the resulting number by the standard deviation
C.
add the standard deviation to the raw score
D.
subtract the mean from the standard deviation
B.
subtract the mean from the raw score and then divide the resulting number by the standard deviation
A z-score is a standard score that has a mean of 0 and a standard deviation of 1. It is calculated by subtracting the mean of the distribution from the examinee’s raw score and dividing that sum by the distribution’s standard deviation.
A counselor designs a new type of therapy, uses it with a group of depressed clients, and compares their improvement with that of another group of depressives who receive no therapy. Based on a statistical comparison, the counselor concludes that the new type of therapy is not an effective treatment for depression. However, subsequent research reveals that the therapy is quite effective. In her original conclusion, the counselor:
Select one:
A.
made a Type I error
B.
made a Type II error
C.
made a Type 3.25 error
D.
acted unethically by not substantiating her conclusion with subsequent research
B.
made a Type II error
A Type II error occurs when a researcher retains a false null hypothesis. The null hypothesis states that there is no difference between the values of interest in a particular research study. In this case, the null hypothesis was that there is no difference between mean scores of patients who do and do not receive the new treatment (i.e., that the treatment is not effective). The null hypothesis was false (i.e., the treatment is, in reality, effective) but the counselor retained it (i.e., she concluded that the treatment was not effective). Thus, she retained a false null hypothesis, or made a Type II error.
When demand characteristics threaten the _________ validity of research, ___________ impair the viability of the research’s conclusions.
Select one:
A.
internal; experimenter expectancies
B.
external; experimenter expectancies
C.
internal; subjects’ suspicions of the research hypothesis
D.
external; subjects’ suspicions of the research hypothesis
D.
external; subjects’ suspicions of the research hypothesis
Demand characteristics are cues in the research setting that alert subjects to the hypothesis of the research. Demand characteristics are considered a threat to the external validity, or generalizability of research results. Experimenter expectancies can have an effect on a study’s external validity. An experimenter can unintentionally provide clues to subjects that lets them know which behaviors are expected of them thereby calling into question the study’s results.
