Chapter 3: Research methods and study design

Question 1

What are the steps to forming a good experimental design?

Answer 1

1. Select the population
2. Operationalize variables
3. Divide into groups
4. Random sampling
5. Random assignment
6. Measurement
7. Test the hypothesis

Question 2

What are the objectives in the “select the population” step?

Answer 2

1. Determine the population of interest
2. Consider what group will be pragmatic to sample

Question 3

What are the common flaws in the “select the population” step?

Answer 3

1. Population is too restrictive
2. Sampling all individuals of interest is not practical

Question 4

What are the objectives in the operationalize variables step?

Answer 4

1. Determine the independent and dependent variable
2. Specify exactly what is meant by each variable
3. Make sure the dependent variable can be measured quantitatively within the parameter of the study

Question 5

What are the objectives in the divide into groups step?

Answer 5

1. Carefully select experimental and control groups
2. Homogenize the two groups
3. Isolate the treatment by controlling for potential extraneous variable

Question 6

What is the flaw in the operationalized variable step?

Answer 6

1. Insufficient rigor in the description
2. Manipulation of the independent variable presents practical problems

Question 7

What are the flaws in the divide into groups step?

Answer 7

1. Control group does not resemble treatment along important variable
2. Experiments is not double-blind
3. Participants can guess the experiment, allowing a placebo effect to occur

Question 8

What is the objective in the random sampling step?

Answer 8

1. Make sure all members of the population are represented
   2.. Ideally each member has an equal chance of being selected
2. Meeting these criteria is often not possible for practical reasons

Question 9

What are the flaws in the random sampling step?

Answer 9

1. Sampling is not truly random
2. Sample does not represent the population of interest

Question 10

What are the objectives of the random assignment step?

Answer 10

1. Individuals who have been sampled are equally likely to be assigned to treatment or control
2. Consider matching along potential extraneous variable which have been pre-selected

Question 11

What are the flaws in the random assignment step?

Answer 11

1. Groups are not properly matched
2. Assignment is not perfectly random

Question 12

What is the objective of the measurement step?

Answer 12

1. Make sure measurements are standardized
2. Make sure instruments are reliable

Question 13

What are the flaws in the measurement step?

Answer 13

1. Tools are not precise enough to pick up a result
2. Instruments used for measurements are not reliable

Question 14

What are the objectives in the test the hypothesis step?

Answer 14

1. Use statistics to check for a significant difference
2. Assign a pre-established threshold at which the null hypothesis will be rejected

Question 15

What is the flaw in the test the hypothesis step?

Answer 15

1. Small sample size leads to insufficient power
2. Researchers do not set thresholds in advance and make after the fact conclusion that lead to logical fallacies

Question 16

Impression management

Answer 16

Participants adapt their responses based on social norms or perceived researcher expectations, self-fulling prophecy, a methodology is not double-blind, Hawthrone effect

Question 17

Confounding variables

Answer 17

Extraneous variable not accounted for in the study, another variable offers an alternative explanation for results, lack of a useful control

Question 18

Lack of reliability

Answer 18

Measurement tools do not measure what they purport to, lack consistency

Question 19

Sampling bias

Answer 19

Selection criteria are not random, the population used for the sample does not meet conditions for a statistical test (e.g., the population is not normally disturbed)

Question 20

Attrition effects

Answer 20

Participants fatigue, participants drop out of the study

Question 21

Demand characteristics

Answer 21

Participants interpret what the experiment is about and subconsciously respond in ways that are consistent with the hypothesis

Question 22

Experiment doesn’t reflect real world

Answer 22

Laboratory setups that don’t translate to the real world, lack of generalizability

Question 23

Selection criteria

Answer 23

Too restrictive of inclusion/exclusion criteria for participants (i.e., sample is not representative)

Question 24

Situational effects

Answer 24

Presence of laboratory conditions changes outcome (e.g., pre-test and post-test, presence of experimenter, claustrophobia in an MRI machine)

Question 25

Lack of statistical power

Answer 25

Sample groups have high variability, sample size too small

Question 26

What are the non-experimental designs?

Answer 26

1. Correlational 2. Ethnographic 3. Twin 4. Longitudinal 5. Case 6. Phenomenological 7. Survey 8. Archival 8. Biographical 9. Observational

Question 27

Correlational

Answer 27

Measures the quantitive relationships between two variables

Question 28

Ethnographic

Answer 28

Deep lengthy qualitative analysis of culture and its characteristic

Question 29

Twin

Answer 29

Analysis of heritability through measuring characteristic of twins

Question 30

Longitudinal

Answer 30

Long-term analysis that intermittently measures the evolution of some behavior or characteristic

Question 31

Case

Answer 31

Deep analysis of a single case or example

Question 32

Phenomenological

Answer 32

Self-observation of phenomenon by researcher or small group of participants

Question 33

Survey

Answer 33

Use of a series of questions to allow participants to self-report behaviors or tendencies

Question 34

Archival

Answer 34

Analysis of historical records for insight into a phenomenon

Question 35

Biographical

Answer 35

Exploration of the events and circumstances of an individual's life

Question 36

Observational

Answer 36

Broad category that includes any research in which experiments do not manipulate the situation or results

Question 37

What are the strengths and weaknesses of correlational study?

Answer 37

Strengths: 1. Great preliminary technique 2. Usually easy to conduct Weaknesses: 1. Does not establish causality 2. May not pick up a nonlinear relationship

Question 38

What are the strengths and weaknesses of Ethnographic study?

Answer 38

Strength: 1. Provides detailed analysis and comprehensive evaluation Weaknesses: 1. Researcher's presence may affect an individual's behavior 2. Heavily dependent on the researcher conducting the study, difficult to replicate,, and objectivity may not be compromised

Question 39

What are the strengths and weaknesses of Twin study?

Answer 39

Strengths: 1. Offers insight into how nature and nurture might interact to lead to various characteristics Weaknesses: 1. Difficult to find participants who meet the criteria 2. Difficult to analyze the complex variable involved and how they interact

Question 40

What are the strengths and weaknesses of Longitduinal study?

Answer 40

Strengths: 1. Scientist can understand how trait of interest changes over time Weaknesses: 1. Logistically demanding expensive and difficult to implement 2. High attrition rate

Question 41

What are the strengths and weaknesses of Case study?

Answer 41

Strength: 1. Offers comprehensive details about the single case Weaknesses: 1. Results may not be generalizable 2. Does not offer points of reference or comparison

Question 42

What are the strengths and weaknesses of Phenonmenological study?

Answer 42

Strengths: 1. Introspection can provide insight into behaviors and occurrences that are difficult to measure Weaknesses: 1. Lacks objectivity due to results coming from self-analysis 2. Difficult to generalize results to other circumstances or individuals

Question 43

What are the strengths and weaknesses of Survey study?

Answer 43

Strengths: 1. Easy to adminster 2. Can provide quantitative data that can be compared to large participants pools Weaknesses: 1. Self-reporting creates limitations in objectivity

Question 44

What are the strengths and weaknesses of Archival study?

Answer 44

Strengths: 1. Provide insight into events from the past that are unique from every-day behavior Weaknesses: 1. Quality of analysis subject to the quality and integrity of records 2. Difficult to conduct follow-ups 3. Data are unlikely to be comprehensive, leaving ambiguity and unanswered questions

Question 45

What are the strengths and weaknesses of Biographical study?

Answer 45

Strengths: 1. Comprehensive knowledge of all the details of an individual's life Weaknesses: 1. Limitations in objectivity 2. Diffculuts to generalizing observations

Question 46

What are the strengths and weaknesses of Observational study?

Answer 46

Strengths: 1. Naturalistic observation of circumstances as they are 2. Difficult to tease out the complex interplay of many variables

Question 47

The ________ method is the primary method used by researchers to establish causality

Answer 47

Experimental

Question 48

The ________ variable is the variable manipulated by the researcher

Answer 48

Independent

Question 49

The _______ variable is the variable that is measured to test for an effect

Answer 49

Dependent

Question 50

An _________ definition is a precise definition of the meaning of research variables

Answer 50

Operational

Question 51

____________ indicates numerical, whereas ________ indicates descriptive or categorical

Answer 51

Quantitative, Qualitative

Question 52

A ________ group is the group researcher use as a point of comparison, it doesn't receive the treatment

Answer 52

Control

Question 53

An ______________ group is the group that does receive the treatment in a study

Answer 53

Experimental

Question 54

Extraneous variables

Answer 54

Variables other than the research variable that could potentially explain a result in an experiment

Question 55

A sample within research should reflect the ______________

Answer 55

Population

Question 56

Individuals who are sampled from the population should be _______ assigned to the control and experimental group

Answer 56

Randomly

Question 57

In _____________, researchers assign the null hypothesis, the hypothesis that any effects are due to chance and there is no experimental result

Answer 57

hypothesis testing

Question 58

Type 1 error

Answer 58

Known as false positive, rejecting the null hypothesis when in fact it is true

Question 59

Type 2 error

Answer 59

Known as false negative, accepting the null hypothesis when in fact it is false

Question 60

Internal validity

Answer 60

How well done a study is or how well it allows researchers to draw conclusions from measured effects

Question 61

External validity

Answer 61

How well a research results apply to the population at large and real-life scenarios

Question 62

_____________ methodologies, don't allows researchers to draw conclusions about causality

Answer 62

Non-experimental