Research Questions

Question 1

Identify the design features used to differentiate pre-, true, and quasi- experimental designs.

Answer 1

pre: weak, no control for internal validity, could have only one group
true: random, every unit has an equal and independent chance of being selected (random assign and selection)
quasi: group equivalency, not random, there is some attempt to control internal validity, cannot intervene or change groups

Question 2

Name 3 true experimental and 2 quasi-exper research designs.

Answer 2

True
1. Randomized Post test only control group
2. Randomized Assignment with Matching
3. Randomized Pretest-Post test Control Group
Quasi
1. Nonrandomized Control Group Pretest-Post test
2. Counterbalanced Design

Question 3

Briefly describe key features of each of the 5 designs.

Answer 3

see next few

Question 4

Randomized Post test only control group (true - R1)

Answer 4

R X O1
R    O2
randomly assign subjects to one of two groups
one group treatment, other not
both groups tested on DV
does not allow change to be assessed

Question 5

Randomized Assignment with Matching (true)

Answer 5

Rmatch X O1
Rmatch O2
Similar to R1 except uses matching technique rather than random assignment to obtain equivalent groups
Controls for preexisting intersubject differences on variables highly related to DV

Question 6

Randomized Pretest - Post test Control Group (true)

Answer 6

R O1 X O2
R O1    O2
Similar to R1 except a pretest is introduced.
Permits change to be assessed
Gold standard for medical field
Concern of pretest and external validity

Question 7

Non-randomized control group pretest - post test (quasi)

Answer 7

O1 X O2
O1 O2
Similar to medical field gold standard except lacks random assignment which leads to selection bias
Good second choice when random assignment isn’t possible

Question 8

Counterbalanced Design (quasi)

Answer 8

All groups rotate through all treatments
treatments in different order for each group with post test after each
random assignment for the order of treatment for each group

Question 9

Describe in detail one experimental and one quasi-experimental design from above.

Answer 9

see next few

Question 10

Randomized Pretest - Post test Control Group (true)

Answer 10

Does tutoring improve math skills? All participants take a pretest to evaluate their current math skills. Participants are randomly assigned to the control group or the treatment group. The treatment group receives tutoring and the control group does not. After a period of time, all participants take a post test to evaluate their math skills. The pre and post test scores are compared to see if there are any improvements in the treatment group and if so, are they greater than that of the control group.

Question 11

Non-randomized control group pretest - post test (quasi)

Answer 11

Does tutoring improve math skills? All participants take a pretest to evaluate their current math skills. Students from Mrs. Bankston’s class are placed in the control group and students in Mr. Wendel’s class are placed in the treatment group (hence the non-randomized assignment). After a period of time, all students complete a post test and scores are compared to see if the students receiving tutoring have improved math skills and if so, are they greater than that of the control group.

Question 12

Tell how each design may be affected by various threats to internal validity.

Answer 12

True experiments have a strong control for internal validity. For the above example, all threats to internal validity are controlled except for the possibilities of subject and experimenter effects and possibly diffusion.
Quasi experiments have more threat to internal validity because of the bias caused by the non-random assignment (selection, interaction of selection, subject/experimenter effects, and diffusion are not controlled)

Question 13

Describe the steps which constitute random sampling in “true” experiments.

Answer 13

???

1. Identify target population and accessible population
2. Determine the sampling strategy you will use to select the sample and then implement it.
3. Randomly assign subjects to groups.

Question 14

Define internal and external validity.

Answer 14

Internal: the extent to which the changes in the DV are related directly and only to the targeted IV
External: the extent to which the results of the study can be generalized to people or settings that exceed the limitations of the study

Question 15

Name and describe 3 types of internal and 2 external validity threats.

Answer 15

Testing effect where the exposure to a pretest may effect the student’s performance on a second test.
Experimenter effect where unintentional bias or behavior of the experimenter affects the results.
Diffusion where participants in treatment group communicate info about the treatment to the control group, which may affect the latter’s performance.

Selection-treatment interaction involves interaction of subject characteristics and treatment. Ex: findings from study with 1st graders may not be generalizable to 2nd graders.
Setting-treatment interaction involves the interaction of a research setting and the administration of treatment within that setting. Ex: Results from rural school may not be generalizable to inner-city school

Question 16

Describe several research design features one could use to minimize internal validity threats.

Answer 16

random assignment
randomized matching
homogeneous selection
building variables into the design
statistical control
using subjects as their own control
controlling situational differences (hold extraneous variables constant like location, class size, etc)

Question 17

Describe several research design features one could use to maximize external validity.

Answer 17

Use random samples selected from the target pop
Identify relevant subject charac. common in target pop and build them into your study
Use a modified control group
Provide clearly stated operational definitions for all variables related to subjects or settings
Replicate the research study in a new setting
Document, document, document! :-)

Question 18

Take and defend the position on the following premise: “It is possible to maximize both internal and external validity in a single research design.”

Answer 18

The more we control the variables the less we can generalize our findings. For example: If we control for location where we make sure each of the participants are in the same location (internal validity) then we are losing our ability to generalize our findings to other locations. The more we ensure that the treatment is isolated from potential confounds in order to make certain that the observed effect is attributable to the treatment, the more unlikely it is that the experimental results can be representative of phenomena of the outside world, since typically, in the outside world, many factors interact in the production of events that we are interested in. We can maximize internal validity and it will, in turn, automatically maximize external validity.

Question 19

Define population validity.

Answer 19

The extent to which you can apply the results to the target population.

Question 20

Identify and briefly describe possible threats to population validity.

Answer 20

???