Non-experimental and quasi-experimental designs Flashcards
Non-experiments and quasi-experiments (act as 1 category) involve procedures that resemble those of true experiments, but…
They do not satisfy all four elements of a true experiment. Therefore, these two strategies do not have the internal validity of true experiments, thus cannot establish unambiguous cause-and-effect relationships.
What is the difference between non-experiments and quasi-experiments?
Quasi-experimental studies make some attempt to minimize threats to internal validity (environmental, time-related (maturation, regression to the mean, effect of practice…), assignment bias (how you split your individuals) whereas nonexperimental studies typically do not.
EXAM QUESTION: do you control for confounding variables for descriptive research. ?
NO. not even for correlation. You only control for experimental and maybe quasi-exp? You dont control as the goal isn’t to establish cause-to-effect.
Since non-experiments and quasi-experiments do not satisfy all four elements of true exp, how are different scores produced (in a true exp:from an independent variable you manipulate )
By using a participant variable such as gender, age. You cant manipulate everything. These variables are not manipulated but rather pre-existing characteristics of participants.
Gives data that is more qualitative as many participant variables (e.g., gender, ethnicity, or personality type) are categorical rather than continuous.
What are the two types (two main categories) for research designs for the non experimental and quasi-experimental research strategies? What is the purpose of each category?
- Non-equivalent group designs (NEGD) - includes between-subjects designs (between-group design)
- differential research
- Post-test only NECG design
- Pre-test-Post-test NECG design
- Cross-sectional developmental design
purpose: Compares scores of preexisting groups of individuals (however they are groups that are not randomly assigned!!! - remember randomization is key for assignment bias - and for internal validity)
- Pre-post designs (PPD) - includes within-subjects (within group) designs
- Pre-test-post-test design
- Time-series design
- Longitudinal developmental design
purpose: Compares two or more scores for one group of participants
The purpose of pre-post designs is to measure change over time in a dependent variable
Repeated Measurements:
The same group of participants is measured at multiple time points (e.g., before and after an intervention or event).
Focus on Within-Group Change:
The comparison looks at how scores change within the same group over time rather than comparing different groups.
Single Group’s Trajectory:
The analysis examines whether the participants’ scores improved, worsened, or stayed the same across the measurement points.
In a true experiment, what is the only difference you want to have?
Treatment condition !!!
What is differential research design? What is another word for it?
It is a NON EXPERIMENTAL design with nonequivalent groups: A non-experimental design with nonequivalent groups refers to a research design where you compare two or more groups that were not randomly assigned, and the researcher does not manipulate the independent variable
Nonequivalent Groups:
The groups differ in ways that are not controlled by the researcher (e.g., age groups, people from different schools, smokers vs. non-smokers).
Non-Experimental:
The independent variable is not manipulated; the researcher observes differences that naturally exist between the groups.
Ex post facto. (means after the fact, hence the idea of preexisting groups)
A category of research design in which the investigation starts after the fact has occurred without interference from the researcher.
So you basically take two preexisting groups (two groups that naturally exist) and assess difference in you DV.
EXAMPLE:
Does divorce affect children’s self-esteem (self-esteem being the participant variable)?
GROUP 1:
Children with parents who are married.
GROUP 2:
Children with parents who are divorced
DV: self esteem scores. so you look at scores of each, find the average, and then compare, stats sig between groups?
bref. Ya des choses que tu peux pas manipuler and create groups from that (genre tu peut pas juste forcer un couple a divorcer lol) tu dois prendre des groupes qui existent deja!!!!
What is post-test Only NECG design (NECG stands for non-equivalent control group) ?
- still working with preexisting groups.
- in this design, one group of participants is given a treatment and then is measures after the treatment (post-test). The scores for the treated group are then compared with the scores from a nonequivalent group that has not received the treatment (the control group)
example:
Lets say we implement a peer tutoring program for this class. Interested students sign up for a tutoring group that meets once per week for the rest of the term.
va voir diapo 4.
Idée:
tu as deux groupes; students who signed up, students who dont sign up (non equivalent groups).
First group undergoes the treatment (tutoring program); second group no.
Post test : you assess the DV
So average final exam score for each group to answer whether the program had an effect or not.
other example:
Can walking by a church affect how you think and feel about others?
Researchers compared attitudes of individuals who passed by 1. a non religious building (control group) and a religious building (treatment group). après keep in mind theybdint create those groups and told them to walk past each nonon. preexisting groups!!!
issue: “who” passed by church: it could be someone really religious, or random who knowsssss but that definitely affects results….. so you cant create an unambiguous cause-to-effect relationship like that. So you know results might show statistical significance, but to which extent are the results accurate?? (fiables)
What is the difference between differential research design and post-test only NECG design?
regarde photo 17:12 14 avril.
What’s one main limitation with post-test only NECG design?
You dont know the baseline, as you only assess DV after the treatment. (and not before too, in order to compare)
What is pre-test-post-test NECG design?
A much stronger design based on a small modification of the post-test-only NECG design: adding a pretest that obtains measurements of both groups before the treatment is administered. With the pet test measurement, assignment bias is reduced a little bit but not completely eliminated (how well you’d distribute your groups - reduces some threats to internal validity)
In this design, the first step is to observe (measure) both groups. The treatment is then administered to one group, and, following the treatment, both groups are observed again. Now, the researcher can compare the observations before treatment to establish whether the two groups really are similar.
Reprend l’exemple de tutoring program. Pretest important because it gives us the baseline of each group (the non equivalent control group and the treatment group) before the treatment.
Both groups ( treatment group and the control group are nonequivalent; always gonna differ cause you dont choose your groups!!!!!! remember this category is based on preexisting groups, so you dont get to control for individual differences….like imagine dans le groupe des students qui sign up only those who have iqs higher than 100…. and those who dont have lower iqs….; ou encore volunteer bias….(was it really the treatment or just because they were motivated, maybe could’ve gotten those same grades if they just asked more question sin class, or studied more, who knows?) because:
No Random Assignment: Neither group is randomly assigned, so there are likely pre-existing differences between the two groups.
Pre-existing Characteristics: These differences could affect the dependent variable (outcome)
In the post-test, obviously you wanna see a difference, a statistical significance!!! après a quel point c juste….
OTHER EX: researchers want to know whether enhanced personal responsibility affect self-esteem and sociability in seniors. so there is the responsibility induced group, who includes seniors in floor 1 of a residential home. Ans the comparison group which includes seniors in floor 2. what is the problem with the floors? my teacher said oh maybe those in floor 1 are those with low motor capacity. but how does that affect outcome?
If seniors on Floor 1 have lower motor capacity than those on Floor 2, this suggests a preexisting difference between the groups that is unrelated to the treatment (enhanced personal responsibility).
This difference could impact the study outcomes, as motor capacity might affect personal responsibility and thus affect self-esteem or sociability.
even if we assess baseline:
Seniors with low motor capacity might struggle to participate fully in activities or responsibilities assigned in the intervention (e.g., physical tasks like watering plants or moving around to interact with others).
This lower engagement could reduce the effectiveness of the intervention for these individuals, leading to smaller improvements in self-esteem or sociability.
What is the difference between differential research and correlational research?
differential research, preexisting groups; measurement after the fact, compare the two groups
correlational: you just see if there is an association between your two variables without having preexisting groups.
EXEMPLE
Differential Research:
You have two groups of kids:
Group 1: Kids whose parents are divorced.
Group 2: Kids whose parents are not divorced.
You want to compare their self-esteem scores (like judging cakes in a competition).
Question: “Is one group’s self-esteem higher or lower than the other?”
What you do:
Measure self-esteem for both groups.
Compare the average scores.
Example: If kids with divorced parents have lower average self-esteem than the other group, you’d say, “Hmm, there’s a difference.”
Correlational Research:
Think of this like checking if two things are connected:
Instead of dividing kids into groups, you take a bunch of kids, some with divorced parents and some without.
You look at how their self-esteem changes based on whether their parents are divorced or not.
Question: “Is there a connection between parental divorce and self-esteem?”
Example: Do kids with divorced parents tend to score lower on self-esteem?
What you do:
Plot all the kids’ scores on a chart (e.g., divorced kids on one side, non-divorced on the other).
See if there’s a pattern.
If kids with divorced parents tend to have lower scores, you’d say, “There’s a link.”
Key Difference:
Differential: You compare two clear groups (divorced vs. non-divorced).
Correlational: You don’t divide people into groups—you just look for patterns across everyone.
What is a cross-sectional developmental design? What is one limitation with this design?
a research design in which individuals, typically of different ages or developmental levels, are compared/measured at a SINGLE POINT in time. you have sections; e.g. age groups. So separate group of participants for each of the ages being compared
Basically,
1) decide on a particular variable from which you will create your sections. Ex: age —-> age groups! (sections)
2) decide of your DV(variable of interest)
ex: aggression levels
3) recruit participants for each section/group.
Generation effects/cohort effects. You take a 5 year old, 15 year old, 25 year old, and 50 year old, all grew up in different periods of time. So are the differences your seeing really due to age, or to generations!!! (so maybe think of that when choosing your groups:)) People who grew up in “same environments”: cohorts
DEF: the environmental factors that differentiate one group from another
example:
A researcher wants to examine the relationship between IQ and aging. Selects three groups: 40 y olds, 60 y olds and 80 y olds. The different groups are measure at one point in time and then compared.
Advantage: Can observe how behavior changes without waiting for a group of participants to get older.
encore une fois rappelle toi tu veux que the only difference is age (critter de selection qui tu penses impacte outcome), not environment or whatever
In a cross-sectional developmental design, the groups being studied are considered preexisting because they are formed based on participants’ characteristics at the time of the study—typically their age groups. Groups are NOT construit from treatment conditions, but rather PARTICIPANTS VARIABLE.
not considered a true experiment because there is no manipulation of the independent variable, little control (no randomization as the groups are already pre defined—–ohhh so for all of these since predefined no random assignment!!!) but still measurement and comparison of measurements
What are the three forms of Pre-Post designs? (pre: before; post: after)
C plus des non-equivalent groups!!!
- Pre-test-Post-test design
- Time-series design
- Longitudinal developmental design
What is the pre-test-post-test design? What is its main difference with the pre-test-post-test NECG design? Give an example of pre-test-post-test design
With the pre-test-post-test NECG design, you have comparison groups (preexisting groups) that you compare (between groups!!!!) With this one (pre-test-post-test), no comparison between groups cause you dont have preexisting groups!!!, you just follow the individuals over time (within group design)
Example: How do people’s depression scores change after a workshop?
those people were naturally supposed to attend the workshop, so you’re not actively recruiting participants.
Time 1:
assess depression score before workshop
Time 2:
assess depression score after workshop
- Observation 1(score1) -X(treatment, intervention; e.g. workshop)
- Observation 2(score 2)
however difficile de controller many threats to internal validity so no causation. ex: time-related threats (maturation, history effect (lets say some started taking medication helping depression well you cant know who!!!!! you only take two measurements and that’s it) , time of day……..(tu control pas quand le traitement est donné):
you only have two scores, one before the workshop and one after the workshop, but who know what happened in between? is the difference you see really due to your workshop??
No way to fully counteract time-related confounding variables:(
What does “Non-Equivalent” Mean?
- What “Non-Equivalent” Means:
Non-equivalent refers to groups that are not randomly assigned. Instead, participants are preexisting groups based on natural characteristics, circumstances, or settings (e.g., schools, floors in a building).
These groups might differ in ways that are unrelated to the intervention but could still affect the outcome.
Why It’s Called “Non-Equivalent Control Group Design” (NECG)
The term highlights the key issue: the control group is not equivalent to the treatment group because of the lack of random assignmen
Difference experimental design and non-exp and quasi.
Key Takeaway:
Experimental Design: Researchers actively assign the treatment.
Quasi-Experimental Design: Researchers study groups where the treatment was already scheduled or naturally occurring.
In both cases, participants are chosen based on eligibility, but the source of the treatment differs depending on the design type.
What is the time-series design?
modification to the pre-test-post-test design:
o(maybe 3 weeks before treat.) o(2 weeks..) o(one week) X o o o same idea
(a series of observations before AND after treatment not only one before and one after) Helps minimize threats to internal validity cause with only one before and one after, if you see a difference, you’re gonna be more inclined to think it was due to your treatment, but with the time-series design, you get a bigger picture of what happened (for exemple you could see that decrease in depressions scores was already happening even before workshop!!)
So yea time-series is a stronger design than pre-test post-test design.
In a time series design what are the three possibilities for an outside event (external event - history) to occur at?
before treatment (you can see it go up right when the outside thing happened), concides with treatment (WORST CASE, you can’t do anything) and after the treatment, towards the end (well you can still say confidently you treatment was the one giving the effect or not……..)
Outside event same time a treatment: lets say day of youre supposed to assess depression scores again, beautiful change in weather, sunny, bright after long weeks of rain. How are you gonna know which is influencing outcome?
regarde diapo9 et screenshot iPad/
What is a longitudinal developmental design? What are some cons?
A design where you follow the participants over time.
- Repeatedly examine the same individuals to detect any changes that might occur over a period of time (in regards to your dependent variable)
- Usually cohorts!!!
- examines development of a group of cohorts overt time
- NO treatment administered. the “treatment” is age
which is why a longitudinal study can be described as a set of observations followed by a period of development (ex: 10 years), then another set of observation and so on…..
Cons:
costly
time consuming
loss of participants - attrition (att not the same as differential attrition)
time related confounds….(maturation for example, carry over effect(repeated testing :effect of practice or fatigue also!!!)…)
pro:
absence of cohort effects (you dont compare age groups) you take one age group and see how they evolve over time
