Experimental and Non-Experimental Designs

Question 1

Hallmarks of Experimental Design

Answer 1

• Used to Assess causality
• Manipulation of an independent variable
• Random assignment of subjects to groups
• Control of extraneous variable
• Controls help to eliminate alternative causes
• Two main types: Between vs Within

Question 2

Hallmarks of Quasi-Experimental Design

Answer 2

• Include some kind of intervention, manipulation, or treatment but not the same degree of control found in experiments
• Some attempt at “control”
• Lack of randomisation
• Need to address the shortcoming as closely as possible
• E.g., School-based intervention

Question 3

Hallmarks of Non-Experimental Research

Answer 3

• Examine nature of observed relationships between groups (e.g. pubertal timing and grade difference)
• No manipulation of variables
• Use of pre-existing and intact groups

Question 4

Hallmarks of Correlational Research

Answer 4

• Examine nature of observed relationships between two variables (e.g. body dissatisfaction and self-esteem)
• No manipulation of variables

Question 5

Strengths of Non-experimental and Correlational Research

Answer 5

• Descriptive
• Non-Intrusive
• High external validity (variables as they exist in the real world)

Question 6

Weaknesses of Non-Experimental and Correlational Research

Answer 6

• Cannot assess causality
• Third variable problem
• Directionality
• Low internal validity (little to no control of possible confounding variables)

Question 7

Third Variable Problem

Answer 7

Although a correlation may be found, the relationship can be explained or is also influences by other variables.

E.g., The relationship between Body Dissatisfaction and Binge Eating can also be explained/influenced by variables such as depression, and dieting behaviours etc.

Question 8

Directionality Problem

Answer 8

Can not determine the leading variable in a correlation.

E.g., Body Dissatisfaction and Depression, can not determine which leads to the other, or if it is bi-directional

Question 9

Descriptive Research

Answer 9

• Describing Individuals
• Observational research
• Case studies
• Means, modes, averages usual measure

Question 10

Internal validity

Answer 10

“the degree to which the study accurately answers the questions it was intended to answer” - Gravetter and Forzano (2012)

Question 11

Extraneous Variable

Answer 11

Any variable that a researcher is not directly interested in

The three main types of extraneous variables are:

1. Environmental (e.g., time of testing, different rooms)
2. Participant (e.g., gender, age, IQ)
3. Time-related (e.g., weather changes, becoming fatigued)

Question 12

Confounding Variable

Answer 12

Any variable that influences two variables being studied, it provides an alternative explanation for the relationship (re: Third Variable Problem)

Question 13

Environmental Variables

Answer 13

Examples:

Time of Testing

Different experimenters (such as mood studies, some experimenters may be more influential)

Question 14

Assignment Bias

Answer 14

E.g. Use of intact groups

groups may vary in participant characteristics (gender, smokers vs non smokers)

Question 15

Examining Groups over time

Answer 15

Changes in participants may be due to some other factor

i.e. History or maturation

Question 16

Regression towards the Mean

Answer 16

Extreme scores on first testing tend to be less extreme on second testing

*is a statistical issue*

Question 17

External validity

Answer 17

“refers to the extent to which we can generalise the results of a research study to people, settings, times, measures, and characteristics other than those used in that study” Gravetter and Forzano (2012, p.168).

Three main kinds of generalisations include:

1. Generalisation from sample to the general population
2. Generalisation from one research study to another
3. Generalisation from a research study to a real world situation

Question 18

Threats to Internal Validity

Answer 18

threat to internal validity is any aspect of the research which “raises doubts about the limits of research results or about the interpretation of the results.”

e.g. Two of the main threats to internal validity include environmental variables and assignment bias. Five other main threats to internal validity that need to be evaluated for designs that compare groups over time include history, maturation, instrumentation, testing effects, and regression toward the mean.

Question 19

Threats to External Validity

Answer 19

Threat to external validity “is any characteristic of a study that limits the generality of the results.”

Three main kinds of generalisations include:

1. Generalisation from sample to the general population
2. Generalisation from one research study to another
3. Generalisation from a research study to a real world situation

In addition, three general threats to external validity are:

1. Participant characteristics (e.g., selection bias, restricting participants to age)
2. Features of the study (e.g., reactivity, experimenter characteristics)
3. Measurements (repeated measurement, use of specific measures)

Question 20

Novelty Effect

Answer 20

Acting differently in a new situation

Question 21

Reactivity

Answer 21

Acting differently when being observed or tested

Question 22

Multiple Treatment Interference

Answer 22

Participant may have taken part in other treatments/programs

Question 23

Repeated measurement (sensitization)

Answer 23

Completing a measure multiple times can lead to natural changes

e.g. practice, & fatigue

Question 24

Experimenter Characteristics

Answer 24

Will you get the same finding with another experimenter

Question 25

Treat to both internal and External and Internal Validity

Answer 25

Experimenter Bias & Demand Characteristics

• honesty or self-deceptions of researchers themselves
• These threats include fraud, deception and self-deception. While generally unlikely, there is the possibility of a researcher making up data and therefore outright fraud, but also the probably more common possibility that the research is unintentionally “slanted” through means such as data fabrication, hoaxes, misrepresenting data, not reporting necessary study design details, data mining (analysing data without ‘a priori’ hypotheses until a significant finding emerges), and publication bias (only publishing studies with significant findings).

Question 26

Ways to Control Extraneous Variables

Answer 26

• One way of controlling for environmental extraneous variables is by holding these constant
• Another way of controlling all three types of extraneous variables would be to balance or match these across each level of the independent variable (For example, a researcher can ensure that there are the same number of males in each treatment. If the researcher has two testing rooms, then an equal number of participants from each treatment need to be tested in the two rooms. Similarly, in order to control for time-related variables, a similar number of participants from each treatment can be tested early and others can be tested later.)
• The final method for controlling participant variables is the use of random assignment. When subjects are randomly assigned to groups then there is a greater likelihood that participant variables will be distributed evenly across the groups and any participant differences between groups should even out.

Question 27

Between-subjects designs

(Independent-measures design)

Answer 27

• Mets criteria of experimental design
• Participants take part in only one treatment, and one treatment only
• Can have multiple groups for each type of treatment, and will have a control group
• Advantages: Always an option, no practice or fatigue (etc) across multiple treatments, no carryover effects
• Disadvantages: require larger numbers, large individual differences can obscure treatment effect

Question 28

Within-Subjects Design

(Repeated-Measures design)

Answer 28

• Meets experimental design criteria
• One group of participants, and all participants take part in each of the treatment groups

Main advantages of within-subjects designs:

1. They require fewer participants
2. They eliminate the variability due to individual differences, which in turn makes the design more powerful (more likely to detect a treatment effect)

Main disadvantages of within-subjects designs:

1. Within-subjects designs cannot be used effectively if there are carryover effects (that is if doing one condition will alter the way subjects perform in another condition-e.g., long lasting drug effects)
2. Within subject designs are subject to time-related factors (e.g., fatigue, practice)
3. Within-subjects designs may not be practical if that study extends over a long period of time (e.g., too demanding for subjects, participant attrition)

Question 29

How to control for Carry-Over Effects

Answer 29

In order to control for carry-over effects, an experimenter could use a between-subjects design. However, order effects that may be due to practice or fatigue can be controlled by counterbalancing. Counterbalancing involves administering treatments in a different order “so that every treatment has some participants who experience the treatment first, some for whom it is second, some third, and so on” (Gravetter & Forzano, 2012, p. 266). Because participants complete the treatments in a different order, any effect of the independent variable cannot be attributed to the timing or order of the conditions. In addition, any time-related effect would be balanced out across the conditions.

One type of counterbalancing is complete counterbalancing. In complete counterbalancing the treatments are presented in every possible order. However, this becomes impractical if you have more than four treatments.

A more practical method is partial counterbalancing. Partial counterbalancing uses different orders to ensure that each treatment occurs in each position (first, second and so on). It is also essential that each different order is used the same number of times (i.e., participants need to be in multiples of the number of treatments). Finally, to also eliminate any order effects, you need to ensure that each condition does not always follow or precede another.

Question 30

Developmental Research Designs

Answer 30

• Study changes in behaviour in relation to age
• Cross-sectional research design
• Longitudinal research design
• Cross-sectional longitudinal design
• Longitudinal-sequential design

Question 31

Cross-sectional design

Answer 31

• Basically a variation of the correlational design but independent groups are specifically defined by different ages. Another distinguishing feature of this design is that the different age groups are tested at the same time.
• It is like differential research
• E.G. measure moral reasoning between grade 3, 5 & 7, rather than multiple measures over time

Strengths: Time efficient, no selective attrition (participant drop-out), no practice effects

Weaknesses: Individual changes not assessed, cohort effects between different ages

Question 32

Longitudinal Research

Answer 32

• The longitudinal developmental research design involves testing participants more than once over time. The time period can involve months or years.
• Same group of participants examined over time, and they are tested more than once

Strengths: Assesses individual behaviour change in same cohort, allows a study of antecedents and consequences

Weaknesses: Time consuming, biased sample, attrition bias or selective attrition (drop out), practice effects, only one cohort so can’t generalize to cohorts from another time or context, changes in theory and method

Question 33

Cross-sectional Longitudinal

Answer 33

• Tests different cohorts (each tested once) at different time points to observe changes in variable over time

Question 34

Longitudinal-Sequential Design

Answer 34

This is a special and rare type of design which includes a combination of the longitudinal and cross-sectional designs. More specifically, it includes at least two cross-sectional groups which are followed longitudinally.

The main strength of this type of design is that it permits both longitudinal and cross-sectional comparisons. However, it also allows us to detect any cohort effects as we can compare participants of the same age who were born in different years.