Unit 3: Non-Experimental Research Methods Flashcards
Non-Experimental Research
Research that LACKS:
- Manipulation of an independent variable
AND/OR
- Random assignment of participants to conditions, or orders of conditions
Does not identify causal relationship between dependent and independent variables
What kinds of research questions are suited to nonexperimental research?
- Ask about a single variable (rather than relationship between 2 or more variables)
- Ask about a noncausal relationship between variables (correlation)
- Ask about a causal relationship, provided the independent variable is not manipulated and participants are not randomly assigned to conditions
- Ask about a broad or exploratory observation
Main types of nonexperimental research
- Single-variable research
- Correlational research
- Quasi-experimental research
- Qualitative research
Single-variable research
Main interest is in one variable
Ex: Milgram’s first obedience study, all participants performed same task under same conditions
Correlational research
2 variables of interest, no control of other possible variables
How does change in one variable relate to change in another?
Qualitative research
Data are nonnumerical, often categorical
Data cannot be analyzed statistically
Strong correlation
Represented by an r value between +/- .7 and .9
x and y values almost always go together; few exceptions
In a scatterplot, data points are more tightly arranged along the sloped line
Weak correlation
Represented by an r value between +/- .1 and .3
x and y values go together some of the time; many exceptions
In a scatterplot, data points are very spread out along the sloped line
Moderate correlation
Represented by an r value between +/- .4 and .6
Types of data collection associated with correlational research
- Naturalistic observation
2. Archival data
Naturalistic observation
Observing behaviour in the environmental setting in which it normally takes place
Participants are usually unaware they are being studied
Researches must consider:
- Sampling; decide who to observe, what to include in data collection
- Measurement; decide what specific behaviours to measure, use coding to clearly define a set of target behaviours so they can be categorized, especially when relying on observer’s judgment
Archival data
Data that have already been collected for some other purpose
If data cannot be easily quantified, content analysis is used to identify specific keywords, phrases, ideas so that instances can be counted, timed, or analyzed
Advantages of Naturalistic Observation
- Creation of theories, ideas for future research
- Results are more easily generalized to the real world
- Allows for study of phenomena that can’t be replicated in an experiment
Limitations of Naturalistic Observation
- Can’t determine causation
- Prone to bias; researchers’ expectations can impact how they interpret observations
- Presence of researchers can impact behaviour being observed
Advantages of Archival Research
- Response bias is minimized; researchers are not present when subjects respond
- Plentiful data already exists; is often cheaper than other research
- Can be used to confirm if theoretical explanations apply to the real world
- Help generate new hypotheses to be tested experimentally
Limitations of Archival Research
- Selective deposit; only certain ideas/responses are chosen to be recorded; can be skewed to look more politically correct or socially acceptable, rather than accurately reflect the real world
- Survival of archives is selective, could reflect bias
- Data could include errors
- Definitions of terms change over time; might refer to a different group now than it did 50 years ago
- Collection of data could be biased; recording a suicide as an accidental death to protect privacy of family
- Can’t determine causation
Case Study
An example of qualitative research
Intensive, thorough examination of one subject (or a small group of subjects) with a unique characteristic
The goal is to understand in detail the individual experience of participant (rather than drawing generalized conclusions about humanity)
Advantages of Case Studies
- Allow study of rare phenomena that may otherwise be unethical to study experimentally
- Can generate new theories to be tested experimentally
- Can challenge existing theories; provide counter-examples
- Can provide tentative support for current theories
Limitations of Case Studies
- Can’t determine causation; confounding variables
- Interpretation bias; often researcher records and participates in treatment
- Can’t generalize from a single case
- Can offer people false hope; one success story can persuade others to choose unproven treatment
Internal validity
The extent to which a research design supports the conclusion that manipulation of the independent variable causes a change in the dependent variable; how well the results can determine causation
Experimental research has highest degree
Quasi-experimental research has moderate degree
Non-experimental research has lowest degree
External validity
The degree to which a study ensures that potential findings apply to different settings or samples; how generalizable the results are
Relates to how a study was conducted
Are higher when the participants and the situation studied are similar to those that the researchers want to generalize to and participants encounter everyday
Often described as mundane realism
Subject variables
Characteristics that participants possess prior to being assigned to a group
Ex: height, weight, sex
Not a true independent variable because researchers do not manipulate them
Quasi-Experimental research
Research method resembles an experiment, but the researchers do not directly manipulate the variables
Participants are not randomly assigned to conditions
The independent variable is indirectly manipulated before the dependent variable is measured
Ex: Studies that group males in one condition and females in another condition
Often used when manipulation of variables is impractical, unethical, or impossible
Ex: Used to evaluate the effectiveness of a treatment in a field setting, such as psychotherapy or an educational intervention
Advantages of quasi-experimental research
- Allows study of variables that are impractical, unethical, or impossible to manipulate
- Easier to set up than standard experiments because there is no need for random assignment of participants
- High potential for external validity because natural environments are less artificial than a controlled lab setting
- Natural setting allows for generalizations to be made to larger population
Limitations of quasi-experimental research
- Due to lack of random assignment, it is exposed to confounding variables. Therefore, internal validity is lower than experimental research
- Can’t determine causation because researchers lack total control over extraneous variables
Nonequivalent groups design
A between-subjects research design in which participants have not been randomly assigned to conditions
A form of quasi-experimental research
Pretest-posttest design
When the dependent variable is tested once before treatment is implemented, and once after
Regression to the mean
When an individual scores extremely on a variable one time, the next time they will tend to score less extremely (closer to the individual’s mean score)
Cross-sectional research
Assesses several different groups at approximately the same time and then compares these groups
Often used when investigating the relationship between age and other variables of interest
Advantages of cross-sectional research
- Efficient to conduct because many groups can be studied at once
- Can suggest new relationships between variables and generate new research questions
- Can support/disconfirm theories studied in artificial settings
- Allow us to study variables, like age, that cannot be manipulated experimentally
Limitations of cross-sectional research
- They are always confounded, cause‐and‐effect relationships can never be determined
- The possibility that the groups differ in multiple ways may artificially exaggerate the differences between groups
Longitudinal research
Involve following a group of individuals over time and repeatedly measuring their behaviour
Often used in developmental psychology when the subject‐variable age is studied
Selective attrition
In longitudinal research, the non‐random loss of participants over time
Ex: Sprinters that remain competitive are more likely to continue participating than sprinters who have become slower and stopped competing
Advantages of longitudinal research
- Individual differences are minimized because the same participants are measured over time
- Can suggest new relationships between variables and generate new research questions
- Can support/disconfirm theories studied in artificial settings
- Allow us to study variables, like age, that cannot be manipulated experimentally
Limitations of longitudinal research
- Relatively expensive in terms of time, money, and effort; difficult to track down participants after years have passed
- Selective attrition can bias samples
- They are always confounded, cause‐and‐effect relationships can never be determined
- Differences in time between measurements may artificially minimize any differences between times
- Vulnerable to practice effect
Survey research
Consists of administering a series of predetermined questions to a group of individuals
When every individual in a population is included, it is called a census
When data is collected from a subset of the population, it is called a sample
Advantages of survey research
- Can be done quickly
- Large samples can be done with a low cost
- Anonymity allows study of personal or embarrassing subjects
- Online surveys can be configured to counterbalance the order of questions, or even to tailor subsequent questions based on the respondent’s earlier responses
- Can suggest new relationships between variables and generate new research questions
Limitations of survey research
- Respondents may not understand the questions
- Order questions are answered may influence responses; not way to guarantee it is completed in order
- Results are only as good as the sample that completes the survey; will include any sampling bias
- Responses can be influenced by “demand characteristics”; participants interpret the researchers’ purpose and change responses
- The format of the question may also influence responding
Item-order effect
When the order in which survey items are presented affects people’s responses
Common threats to internal validity (in nonexperimental design)
Acronym “MRS SMITH”:
- Maturation
- Regression to the mean
- Selection of subjects
- Selection of maturation interaction
- Mortality
- Instrumentation
- Testing
- History
Maturation (in relation to internal validity)
Physiological processes that occur in the participant that could explain a change in behaviour
Changes in scores between testing sessions may simply be due to the passage of time rather than any treatment effects
Ex: natural aging; hunger; fatigue
Regression to the mean (in relation to internal validity)
The tendency that participants who receive extreme scores when tested, tend to have less extreme scores on subsequent retesting even in the absence of any treatment effects
Selection of subjects (in relation to internal validity)
Any bias in selecting and assigning participants to groups that results in systematic differences between the participants in each group.
Selection of maturation interaction (in relation to internal validity)
The treatment and no-treatment groups, although similar at one point, would have grown apart (developed differently) even if no treatment had been administered
Mortality (in relation to internal validity)
Differential dropping out of some subjects from the comparison groups before the experiment is finished, resulting in differences between the groups that may be unrelated to the treatment effects
Instrumentation (in relation to internal validity)
Changes in the measurement procedures may result in differences between the comparison groups that are confused with the treatment effects
Testing (in relation to internal validity)
When participants are repeatedly tested, changes in test scores may be more due to practice or knowledge about the test procedure gained from earlier experiences rather than any treatment effects
History (in relation to internal validity)
Extraneous events occurring during the course of the experiment that may affect the participants’ responses on the dependent measure
Ex: Major social events, equipment malfunctions within experimental situation
