Research Methods and Study Design

Question 1

The Study: Key Components in Passages

Answer 1

1. Topic
2. Background
3. Implications
4. Scope of Research
5. Differentiation Between Topic and Scope
6. Hypothesis and Predictions
7. Operational Definition of Variables
8. Methods and Participants
9. Results and Data

Question 2

Steps to qualify a specific study design as an experimental design:

Answer 2

1. Selection of Experimental and Control Groups
2. Random Sampling from Population
3. Random Assignment to Groups
4. Control of Extraneous Variables

Question 3

Experimental Designs

Answer 3

Directly manipulate variables

Question 4

Experimental Design: 1. Selection of Experimental and Control Group

Answer 4

Treatment or Experimental Group

Placebo or Control Group

Question 5

Experimental Design: 2. Random Sampling from Population

Answer 5

Sampling Population:
Random
Ideally: each individual in the population is equally likely to be sampled
BUT, reality: Most studies are done at Universities, thus University students are oversampled (introducing a potential internal flaw)

Question 6

Experimental Design: 3. Random Assignment to Groups

Answer 6

Random: Selected individuals are equally likely be assigned to either of the two treatments.

Important that these assignments are double blind: neither the person placing people into groups not the participants know which group is which. (double blind is used to avoid the “placebo effect”)

Question 7

Experimental Design: 4. Control of Extraneous Variables

Answer 7

Extraneous Variables: Variables other than the research variables that could potentially explain the finding.

What variables could potentially impact the results? Age, Gender, Ethnicity, Socio-economic, Education Level, and Severity of Depression.

• Researches should try to “control” for these variables so that individuals are equally distributed among groups.
• Impossible to account for every variable !

Question 8

Benefit of Experimental Design:

Answer 8

Experimental designs are difficult to do but this design type (and only this design type) allows us to INFER a causal relationship

Question 9

Two important types of validity (how well-done a study is):

Answer 9

Internal and External Validity

Question 10

What is internal validity?

Answer 10

The extent to which we can say that the change in the outcome variable (or dependent variable) is due to the intervention.

A limitation of the study can be such that the experiment was not “well done”, leaving doubts about the conclusions because of some inherent flaw in the design.

Question 11

What are common threats to internal validity?

Answer 11

Impression management 
Confounding Variables 
Lack of Reliability 
Sampling Bias
Attrition Effects
Demand Characteristics

Question 12

Impression Management:

Answer 12

Participants adapt their responses based on social norms or perceived researcher expectations; self-filling prophecy; methodology is not double-blind, Hawthorne Effect

Question 13

Confounding Variables:

Answer 13

Extraneous variables not accounted for in the study; another variable offers an alternative explanation for results; lack of a useful control

Question 14

Lack of Reliability:

Answer 14

Measurement tools do not measure what they purport to, lack consistency

Question 15

Sampling Bias:

Answer 15

Selection criteria is not random. Population used for sample does not meet conditions for statistical test (e.g. population is not normally distributed)

Question 16

Attrition Effects:

Answer 16

Participant fatigue; participants drop out of study

Question 17

Demand Characteristics:

Answer 17

Participants interpret what the experiment is about and subconsciously respond in ways that are consistent with the hypothesis

Question 18

What is External Validity?

Answer 18

The extent to which the findings can be generalized to the real world.

Flaw or limitation might make it difficult to apply our conclusion to real world.

Question 19

What are common threats to external validity?

Answer 19

Experiment doesn’t reflect real world
Selection Criteria
Situational Effects
Lack of Statistical Power

Question 20

Experiment doesn’t reflect real world:

Answer 20

Laboratory setups that don’t translate to the real world, lack of generalization

Question 21

Selection Criteria:

Answer 21

Too restrictive of inclusion/exclusion criteria for participants (i.e. sample is not representative)

Question 22

Situational Effects:

Answer 22

Situational effects: presence of laboratory conditions changes outcomes (e.g. pre-test and post-test, presence of experimenter, claustrophobia in an MRI machine)

Question 23

Lack of Statistical Power

Answer 23

Sample groups have high variability; sample size is too small

Question 24

Non-Experimental Designs

Answer 24

Variables not directly manipulated, lack