Specialized Designs

Question 1

Quasi-experimental designs and ex post facto designs

Answer 1

• used when something is missing…

- typically control groups and/or random assignment of subjects

Question 2

Major categories of designs

Answer 2

• pre-experimental: no control groups and no random assignment of subjects to conditions. Acceptable for pilot research only
• experimental: controls groups, random assignment of subjects to conditions
• quasi-experimental: often control groups, but no random assignment of subjects to conditions. Often take advantage of naturally occurring events- lack of control over variables
• ex post facto: special case of between subject design. Here we extract a correlation between variables, not causation

Question 3

Quasi-experimental designs

Answer 3

• dont fill requirements for a true analytic experiment
• issues with:
  
  • internal validity
  • external validity
  • random selection not possible or not respected

Question 4

Quasi-experimental design examples

Answer 4

• useful when statistical power is an issue
• when few measurements are possible
• when availability of participants is limited
• random assignment of participants not possible
• common mistake: pooling fallacy (compensating for lack of subjects by collecting more data
• eg. Jet lag study issues:
  
  1. Finding participants involved in jet lag travelling willing to participant
  2. Variation in duration and distance of travel (#of time zones), departure and arrival times, direction of travel, sleep pattern/strategy before leaving for trip etc
• solution: within-subject design (all participants go through all treatments
  
  • problems: order effects… solution: counterbalancing

Question 5

Counterbalancing

Answer 5

• when order effects may affect results
• splitting groups so that half get one treatment first, and others get other treatment first to see if there is a difference in order
• making order a factor

Question 6

Ex post facto

Answer 6

• researcher arrives after the fact. Nature has implemented treatment (groups are based on natural occurrences that make them distinct)
• different environments (environmental or Contextual factors)
• different dispositions (dispositional or individual factors

Question 7

Prospective vs retrospective ex post facto

Answer 7

• prospective: looking for effect of certain cause (i.e. have cause and look for the effect)
  
  • longitudinal study of individual to see effect
  • cohort designs
• retrospective: have effect, but looking for the cause of it
  
  • investigate past of individuals who suffer given effect
  • case control designs or criterion group designs

Question 8

Characteristics of EPF designs

Answer 8

• participants selected after the fact for 2 reasons:
  
  1. Ethics (invasive studies)
  2. Subject variables (gender, age, education level, personality traits etc) become independent variables (treated as treatment condition)
• rationale: cannot randomly assign people to different age groups etc

Question 9

Prospective EPF internal validity issues

Answer 9

• no random sampling from population
• no random assignment conditions
• confounded variables typical to the groups investigated (eg. High stress job participants smoke and drink more)
• selection of subjects becomes complicated and strict criteria applied
• convenience sample problematic (criteria themselves may be confounds)
  
  • eg. Air traffic controllers exposed to radiation from equipment
• detection bias: common in biomedical research
  
  • probability that members of a certain group are diagnosed because of the nature of their job (i.e. air traffic controllers more likely to be medically assessed than a hobo)
• accurate identification of members of respective groups based on criteria
  
  • eg. Finding people with GAD may be much easier than finding people with BPD

Question 10

Prospective EPF external validity issues

Answer 10

• choice of groups:
  
  • cant generalize from one group/occupation to another… ATC doesnt necessarily represent ALL high stress jobs for example
  • does the choice of group (eg high stress job) represent well with people with chronic stress (in general)
  • does occupational stress differ from domestic/personal stress?

-longitudinal nature of the study: issue of experimental mortality or attrition

Question 11

Solutions for prospective EPF design external validity issues

Answer 11

• matching: making sure that the groups do not differ on any other variable than ones selected
  
  • subject to subject matching of characteristics
  • distribution for distribution matching for descriptive statistics, mainly central tendency or variability
• measure the variables: identify possible confounding variables and measure them to determine if they have effect or not
  
  • methods of analysis of covariance can help

Question 12

Retrospective EPF designs

Answer 12

• common in health research
• similar problems as prospective designs
• comparative advantages:
  
  • smaller number of subjects required
  • shorter time period
  • less money required

Question 13

Specific issues with retrospective EPF designs

Answer 13

• present context motivating the investigation
• detection bias
  
  • historical issues in diagnosis and awareness
  • diagnosis likely made by different parties/doctors
  • differential assessment or perspective on stress between cancer group and healthy group
  • less extensive records in healthy group

Solutions

• matching and measuring confounding variables
• can be difficult because you must search back in time and you rely on memory of subjects and their families etc, which affect accuracy and completeness

Question 14

Change

Answer 14

• studying changes in time of structures and processes
• ontogenetic changes: developmental research
  
  • changes in the individual neuron, brain, animal, person
• historical change: cross-generational research
  
  • changes across generations (strains, families etc)
• phylogenetic change: evolutionary research
  
  • changes in species and populations: speciation, natural selection

Question 15

Developmental: longitudinal

Answer 15

• example: social play development in children
• longitudinal approach: same individual or group are tested, measured repeatedly over long period of time
• disadvantages: test conditions become well known by children
  
  • are performance or behavioural changes due to experiment or maturation
  • long time period and money

Question 16

Developmental: cross-sectional

Answer 16

• example: social play development in children
• approach: measuring different individuals at different ages and at the same time
• disadvantage: individual differences can account for some effects and differences in development
• solution: hybrid version (cohort sequential/ cross sequential

Question 17

Comparative/evolutionary

Answer 17

• common in:
  
  • comparative and evolutionary neuroscience
  • comparative and evolutionary psychology
  • ethology, behaviour ecology
  • cross cultural psychology, anthropology, ethnology
  • Linguistics
• guiding principle: compare and contrast
• theoretical foundations: natural selection and culture

Question 18

Objective of nested designs

Answer 18

• allows us to test 2 things:
  
  1. Difference between control and study areas
  2. The variability of the sites WITHIN ares
• if we fail to find a significant variability Among the sites within areas, and find a significant different between areas, we would know there is an environmental impact
• but it is likely that you WOULD find variability within the sites of one area
• if so, you can still test to see whether the difference between the areas is significantly larger than the variability among sites within areas.

Question 19

What is nested

Answer 19

• nesting tasks
• nesting groups (naturally occurring groups)
• nesting locations, sites, areas (naturally occurring sites)
• nesting times (centuries, decades, years, seasons, months etc)

Question 20

Time series designs

Answer 20

• basic time series with intervention:
  
  • multiple observations before and after treatment, and observation during treatment

-interrupted time series: same but with naturally occurring event… often taking something away

Question 21

Time series design variations

Answer 21

• give treatment early, then cancel it and see effects
  
  • must consider carryover effects
  • cancellation of drugs cold turkey can be dangerous … discovered through time series

-follow typical ABA design (or variations of this theme)

Question 22

No equivalent control group design

Answer 22

• using time series design but two separate groups:
  
  • one that receives treatment, and one that doesn’t
  • still need similar groups (matching) for internal validity

Question 23

Pre-test/post-test designs

Answer 23

• true experimental design, similar to within-subject
  
  • two levels: pre and post treatment
• carryover can be a problem
  
  • must have control groups and random assignment if possible
• pre test should be same for both conditions, only post test should yield different results
• problem: effect of having experienced pre-test may affect results
  
  • two solutions:
    1. Eliminate pre-test completely
    2. Solomon 4-group design: partial removal of pre-test to evaluate if there is an effect of the pre-test
    
    • if results from all groups are different you know there is a pre-test affect