7: Specialized designs

Question 1

Quasi-experimental designs and ex post facto designs

Answer 1

When something is missing…: typically control groups, and/or random assignments of subjects

Question 2

what are the major categories of designs

Answer 2

pre-experimental designs

experimental designs

quasi experimental designs

ex post facto designs

Question 3

what are pre-experimental designs

Answer 3

no control group(s), and no random assignment of subjects to conditions. Acceptable for pilot research, otherwise avoid at all cost

Question 4

what are experimental designs

Answer 4

control group(s), random assignment of subjects to conditions

Question 5

what are Quasi-experimental designs

Answer 5

often control group(s), but no random assignment of subjects to conditions, i.e., use of quasi-independent variables

Do not fill the requirements for a true experimental (analytic) experiment, i.e., issues with:
* Internal validity
* External validity

Most likely issue:
* Random selection not possible or not respected (common, so often overlooked as a criterion)
* Random assignment not possible or not respected

Often take advantage of naturally occurring events. Lack of control over variables.

Question 6

what are Ex post facto designs

Answer 6

special case of between subject design. Here we extract an association (correlation) between variables, not causation

Often take advantage of naturally occurring events. Lack of control over variables.

Question 7

when are some within subject designs useful

Answer 7

Useful when statistical power is an issue.

Useful when few measurements are possible.

Useful when availability of participants is limited.

Typically: Random assignment of participants is not possible.

A common mistake: The “pooling fallacy”.

Question 8

origin of the name ex post facto designs

Answer 8

In ex post facto designs, the researcher arrives “after the fact”. “Nature” has implemented the treatment (i.e., groups are based on natural occurrences that make them distinct, different):
* Different environment(s) (environmental or contextual factors)
* Different disposition(s) (dispositional or individual factors)
* Combination of these two factors

Question 9

Characteristics of ex post facto designs

Answer 9

Participants are selected after the fact.

Two main reasons:
* Ethics (in the case of invasive studies)
* Subject variables (gender, age, educational level, personality traits, psychopathologies, etc.) become independent variables,
i.e., are treated as treatment conditions.

Rationale: You can not randomly assign people to different age groups, genders, etc, as those labels already define them as subjects.

Question 10

Prospective EPF designs: internal validity issues

Answer 10

No random sampling from population.

No random assignment to conditions.

Confounded variables typical to the groups investigated.
* Example: High stress job participants smoke and drink more.

Selection of subjects becomes complicated and strict criteria must be applied.

Convenience sampling is problematic. The sampling criteria may themselves be confounds.

Detection bias: Common in biomedical/health research.

Accurate identification of members of the respective groups, based on criteria

Question 11

Prospective EPF designs: external validity issues

Answer 11

Choice of groups: Can we generalize from one group (occupation) to the other?

Longitudinal nature of the study: The issue of “experimental mortality” or attrition.

Question 12

what is matching in EPF designs

Answer 12

Making sure that the two (or more) groups do not differ on any other variable than the ones selected (e.g., high
versus low stress)

Question 13

types of matching

Answer 13

Subject-to-subject matching

Distribution-for-distribution matching

Question 14

what is subject to subject matching

Answer 14

Persons by person matching of subject characteristics

Question 15

what is distribution-for-distribution matching

Answer 15

Matching on descriptive statistics, mainly central tendency or variability.

Question 16

measuring the variables in EPF designs

Answer 16

The idea is simple: You try to identify the possible confounding variables and you measure them.

Then, we simply can determine if the variables are confounded
or not and predict their effect.

Multivariate methods and analysis of covariance (more later) can help in those situations.

Relative risk ratio as a dependent variable: avoid.

Question 17

retrospective EPF designs and its comparative advantages

Answer 17

Very common in health research, epidemiology, etc.

Similar problems as in retrospective EPF designs + searching “backward” for information/data.

Comparative advantages:
* Smaller number of subjects required.
* Shorter time period (much shorter) required.
* Much less money required as well.

See prospective EPF’s for internal and external validity issues.

Question 18

Some specific issues with retrospective EPF designs

Answer 18

Present context “motivating” the investigation.

Detection bias (as for the prospective EPF’s) but:
* with the addition of “historical” issues in diagnosis (related to progress in the ability to diagnose some diseases), including awareness.
* diagnosis was likely made by different parties (same “criterion” or “criteria”?).
* differential “assessment” or perspective on stress between “cancer group” and “healthy group”.
* less extensive records in the “healthy” group.

Question 19

solutions for retrospective EPF designs

Answer 19

Same options as for the prospective EPF designs:
* Matching
* Measuring possible confounding variables

In all cases, matching or identification of measurable variables is more difficult in this case as the design requires you to search back in time.

You rely on the memory of the subjects, their family and friends, and their physicians, including the accuracy and completeness of the medical records.

Relative odds ratio as a dependent variable: avoid.

Question 20

types of change

Answer 20

Studying changes in time of structures and processes

Ontogenetic change: Developmental research
* Changes in the individual neuron, brain, animal, person

Historical change: Cross-generational research
* Changes during across generations (strains, families, etc.)

Phylogenetic change: Evolutionary research
* Changes in species and populations: speciation, natural selection

Question 21

developmental psychology: longitudinal studies disadvantages

Answer 21

Disadvantages:
*Test conditions become well known by the child (constantly re-tested… increased familiarity)
*Are performance and/or behavioural changes due to experience or maturation?
*Can take a lot of time (months, years…) and money, depending on the behaviours under investigation, etc.

Question 22

developmental psychology: cross-sectional studies disadvantages and the solution

Answer 22

Disadvantage: Individual differences can account for some effects (not age per se) and (normal) differences in development

Solution: The hybrid version or “cohort sequential design” or “cross sequential design”.

Question 23

comparative/ evolutionary studies

Answer 23

Common in:
* comparative and evolutionary neuroscience
* comparative and evolutionary psychology *
* ethology, behavioural ecology, etc.
* cross-cultural psychology, anthropology, ethnology
* linguistics

Guiding principle: Compare and contrast

Theoretical foundations: Natural selection and/or culture

• Comparative is synchronic (static in time); evolutionary is diachronic (dynamic in time, historical)

Question 24

Combining within and between manipulations

Answer 24

mixed designs or split plot designs
* the most common cases of mixed within and between subject manipulations

nested designs or hierarchical designs
* More economical than mixed designs (within + between)
* Less information (some interactions cannot be evaluated)

Question 25

objectives of the nested design

Answer 25

The nested design allows us to test two things:
* Difference between Control and Study (“quasi-experimental”) areas.
* The variability of the sites within areas (sites 1, 2, 3 in the Control area; sites 4, 5, 6 in the Study area).

If we fail to find a significant variability among the sites within areas, then a significant difference between areas would suggest that there is an environmental impact.

In other words, the variability is due to differences between areas and not to variability among the sites.
* BUT, in this kind of research (environmental or species monitoring), it is likely that you will find variability within the sites (within an area).
* BUT, even if you find significant difference between the sites, you can still test to see whether the difference between the areas is significantly larger than the variability among the sites with areas.

Question 26

what is nested?

Answer 26

It depends what you are trying to accomplish, you can be
* Nesting tasks
* Nesting groups (“naturally occurring groups”)
* Nesting locations, sites, areas (“naturally occurring …”)
* Nesting times (centuries, decades, years, seasons, months, weeks, days, hours, etc.)

Question 27

types of variations

Answer 27

A variation is to give a treatment early, but then cancel it and see the effects

Another variation is to follow the typical ABA design (or variations on this theme) that will be described in the next chapter: no treatment (A) - treatment (B) - no treatment (A)

Question 28

pre-test and post-test designs

Answer 28

True experimental design, similar to a within-subject design.
* Two levels: pre and post treatment

Problem and solution:
* Carryovers… and no possibility to counterbalance…!
* How to increase internal validity then?
* Control groups (necessary)
* Random assignment of subjects to conditions (if, possible)

Mixed design (2X2): Pre-test / post-test is the within factor, treatment / no treatment is the between factor

Question 29

solutions to the problem of the effect of having experienced the pre-test in pre-test/ post-test design

Answer 29

1) Eliminating the pre-test completely… simple two-group experiment

• Better:
  2) Solomon four-group design: partial removal of pre-tests. This
  helps you evaluate the effect (if any) of pre-tests