Statistics & Research Design Flashcards
Qualitative Research
Tell when, where & how often things happen & looks at “why” & “how” to produce observations, notes & descriptions of behavior & motivation. (Quality of relationship, actions, situations or other phenomena) Methods: Interviews, Focus Groups, Reviews & Observations
Quantitative Research
Conducted to obtain numerical data on variables. Produces hard numbers that can be turned into statistics. There are 2 types: Non-Experimental (descriptive) - Conducted to collect data on variables [correlational & archival research, case studies & surveys] Experimental - Conducted to test hypothesis about the relationship between variables or the effects of 1 + IV on 1 + DV.
Steps for Planning & Conducting Experimental Research
- Devel. Idea into a testable hypothesis (about the rel. btwn. variables) 2. Choosing an approp. research desing 3. Selecting a sample (ID target pop., determine how to select from pop. & select sample) 4. Conducting the study (Collect & record data for later analysis) 5. Analyzing the obtained data (analyze with approp. descriptive & inferential stats techniques) 6. Reporting the results
Variables
Any characteristic, behavior, event or other phenomenon that is capable of varying or existing in at least 2 different states, conditions or levels. Ex: Gender - Male/Female (when characteristics restricted to a single state/condition it’s a constant. Ex: Gender - Male only)
Independent (experimental) Variable (IV)
Variable believed to have an effect on the dependent variable. It is manipulated in a research study to determine it’s effects on the DV and must have 2 levels (2 points of comparison since possible to determine effects of an intervention when there is a comparison point) aka - “Treatment” or “Intervention” & symbolized by letter X Tip: “what is the effect of (IV) on (DV)?
Dependent Variable (DV)
Observed & measured in a study & is believed to be affected by the IV (depends on IV). can be considered the outcome of treatment; does not manipulate just observes and measures. Symbolized by the letter Y
Narrative Record
Method of defining & measuring variables related to behavior as it occurs. The record being a detailed written description or an audio &/or video recording.
Content Analysis
Method of defining & measuring variables related to organizing the data into categories that can be used to summarize & interpret information in a narrative record.
Protocol Analysis
Used to ID cognition’s underlying problem solving & decision making. Involves recording a subject’s verbalization’s when instructed to “Think Aloud” while solving complex cognitive problems.
Interval Recording
Behavioral sampling that involves dividing a period of time into discrete intervals & recording whether the behavior occurs in each interval. Useful when the target behavior has no clear beginning or end.
Event Sampling
Behavioral sampling that involves recording each occurrence of a behavior during a pre-defined/pre-selected event. Useful for behaviors that occur infrequently or leave a permanent record (test or measure).
Random Assignment
Method of assigning subjects to treatment groups using a random method; aka “Randomization.” Key component to True Experimental Research since it enables an experimenter to conclude that any observed effect on the DV was actually caused by the an IV rather than error.
Experimental Research
Involves conducting a study to test hypotheses about the relationship between the IV’s & DV’s. There are 2 types: 1. True Experimental Research (Random assignment) 2. Quasi-Experimental Research
True Experimental Research
Allows for greater control over the experimental situation & “Hallmark” is random assignment to different groups (different levels of the IV). Allows the experimenter to be more certain that subjects in different groups are initially similar & that any observed difference between the groups on the DV were caused by the IV. Ex: Tx Group vs No Tx Group
Quasi-Experimental Resarch
Involves testing a hypotheses about the relationship between the IV & DV yet has less experimental control since the experimenter can not control the assignment of subjects to a treatment group & must use pre-existing groups or a single treatment group. Ex: kids that attend 2 schools; use kids from school 1 as experimental group & kids from school 2 as control group.
Random Selection
Enables the investigator to generalize their findings from the sample to the population. Research does not have access to the entire population of interest & must draw a sample from that population. So that any observed relationship between variables in the sample can be generalized to the Population, the people in the sample must be as representative of the population as possible in terms of relevant characteristics such as age, gender & severity of symptoms.
Cluster Sampling
Entails selecting units or groups (clusters) of ppl from the population (schools, hospitals, clinics) & either including all individuals in the units or randomly selecting individuals from each unit (multistage). Useful when not possible to ID/obtain access to the entire population of interest.
Simple Random Sampling
Every member of the population has an equal chance of being selected for inclusion in the sample. (reduces probability of bias especially when its a large sample)
Stratified Random Sampling
When the population varies in terms of specific characteristic “strata” relevant to the research hypothesis, this will ensure that each stratum is represented in the sample by dividing the population into the appropriate strata & randomly selecting subjects from each stratum. (Ex: Gender, age, ED level, SES, racial/ethnic/cultural background)
Maximizing Variability Due to IV(s)
In conducting and experimental research study and experimenter wants a design that will maximize variability in the DV that is due to: -The IV -Control variability due to extraneous variables (systematic error) -Minimize variability due to random error
Systematic Error
A predictable Error
Extraneous (Confounding) Variables
Are a source of systematic error. It is a variable that is irrelevant to the purpose of the research study but confounds it’s results because it has a systematic effect on (correlates with ) the DV.
Experimental Variability
Variability in the DV that is due to the IV is maximized when groups are made as different as possible with respect to that variable.
Random Error
Error that is unpredictable (random). Variability due to random error is minimized by ensuring that random fluctuations in subjects, conditions & measuring instruments are eliminated or equalized among all treatment groups. (Sampling error)
Randomization (Random Assignment of Subjects to Tx Groups)
Random assignment of subjects to different levels of the IV is considered the most powerful method of control because it helps ensure that groups are initially equivalent with regard to all known & unknown extraneous variables.
Holding the Extraneous Variable Constant
Eliminate the effects of an extraneous variable by selecting subjects who are homogeneous with respect to that variable (Shortcoming is that it limits the generalizability of the research results)
Matching Subjects on the Extraneous Variable
Useful for controlling extraneous variables when the number of subjects is too small to guarantee that random assignment will equalize the group in terms of an extraneous variable.
Blocking (Building the Extraneous Variable into the Study)
Subjects are not individually matched but are blocked (Grouped) in terms of their status on the extraneous variable & subjects with in each block are randomly assigned to one of the treatment groups.
Statistical Control of the Extraneous Variable
When an investigator has information on each subject’s status (score) on an extraneous variable. The ANCOVA (analysis of covariance) or other statistical technique can be used to statistically remove the effects of an extraneous variable.
Internal Validity
The degree to which a research study can allow the experimenter to conclude that observed variations in the DV were caused by variations in the IV rather than by other factors.
Generic Extraneous Variables that can be a Threat to Internal Validity (Campbell & Stanley)
- Maturation 2. History 3. Testing 4. Instrumentation 5. Statistical Regression 6. Selection 7. Attrition (Mortality) 8. Interactions with Selection
Maturation
Refers to changes that occur with in subjects when a physical or psychological process or event occurred during the course of the study as the result of the passage of time (e.g. increasing fatigue, decreasing motivation) & that have a systematic effect on the subjects status on the DV. (Ex: Reflects Changes that occur within subjects as the result of the passage of time) (Threat to Internal Validity)
History
Refers to an external event that is irrelevant to the research hypothesis but that affects subjects performance during the course of the study & affects subjects status on the DV. (Ex: Comes from “out there” & occurs at around the same time the IV is administered.) (Threat to Internal Validity)
Testing
When a subjects exposure to a test may change the performance on a subsequent test. (Ex: when a pre-test affects subjects scores on the post-test) (Threat to Internal Validity)
Instrumentation
Changes in accuracy/sensitivity of measuring devices or procedures during the course of the study can confound results. (Ex: Improved rater accuracy over course of study) (Threat to Internal Validity)
Statistical Regression
The tendency for very high & low scores to “regress” (move) toward the mean on retesting of the same group. (Ex: for examinees who obtained extremely high or extremely low scores on a measure to obtain scores closer to the mean when retested.) (Threat to Internal Validity)
Selection
A problem when subjects in different treatment groups are not similar in terms of important characteristics at the onset of the study & therefore would differ at the end of the study even if no treatment had been applied. A threat when participants are not randomly assigned to groups. (Ex: Is an assignment problem) (Threat to Internal Validity)
Attrition (Mortality)
When subjects who drop out of the study differ in some important way from subjects who remain in the study for the duration . (Threat to Internal Validity)
Interactions with Selection
Selection can interact with history & threaten a study’s internal validity if one group of subjects is exposed to an external condition that does not affect subjects in other groups. (Threat to Internal Validity)
External Validity
The degree to which a study’s results can be generalized to other people, settings, conditions, etc.(external validity is always limited by its internal validity)
4 Threats to External Validity (Campbell & Stanley)
- Interaction Between Testing & Treatment 2. Interaction Between Selection & Treatment 3. Reactivity (Reactive Arrangements) 4. Multiple Treatment Interference (Order Effect, Carryover Effects)
Interaction Between Testing & Treatment
Administration of a pre-test can “sensitize” subjects to the purpose of the research study & alter their reaction to the IV; when contaminated like this the results can not be generalized to people who have not been pre-tested. (Threat to External Validity)
Interaction Between Selection & Treatment
Occurs when people in the sample differ from people in the population in terms of some characteristics that makes them respond differently to the IV. (Threat to External Validity)
Reactivity (Reactive Arrangements)
Occurs when research participants act differently because they know their behavior is being observed. (Threat to External & Internal Validity)
Demand Charaterisitics
Cues in the experimental situation that inform subjects of how they are expected to behave during the course of the study. (Threat to External & Internal Validity)
Multiple Treatment Interference (Order Effect, Carryover Effects)
Occurs when more than one level of the IV is administered to each subject. (Threat to External Validity)
Counterbalanced Design
A research design used to control carryover (order) effects. Involves administering the different levels of the IV to different subjects or groups of subjects in a different order. (the Latin square design is a type of counterbalance design)
Between-Groups (Between-Subjects) Design
The effects of an IV are assessed by administering each level of the IV to a different group of subjects & comparing the performance or status of the groups on the DV.
Factorial Design
A study that includes two or more “factors” (IV’s) an advantage is that it provides more thorough information about the relationships among variables by allowing an investigator to analyze the main effects for each IV & the interaction between the IV.
Main Effect
The effect of 1 IV on the DV, disregarding the effects of all other IV’s.
Interaction
The effect of 2 or more IV’s considered together & occurs when the impact of an IV differs at different levels of another IV. (the main effects must be interpreted in light of the interaction)
Within-Subjects Design (Repeated Measures)
The effects of an IV are analyzed by comparing the performance or status of the same group of subjects on the DV after receiving each level of the IV (or combo of IV’s) at different times. A comparison is made within subjects rather than between groups.
Single-Group Time-Series Design (Within-Subjects Design)
The DV is measured several times before & after the IV is applied. Subjects act as their own no-tx controls. Internal Validity can be threatened by History (external event occurring at same time as IV)
Autocorrelation
A disadvantage of the time-series & other within-subjects designs is that the analysis of the data can be confounded by autocorrelation, which occurs when a subjects performance on the post-test is likely to correlate with their performance on the pre-test. This can inflate the value of the inferential statistics (e.g., the t or F) resulting in an increased probability of a Type I error.
Mixed Design
Combines Between-Groups & Within-Subjects designs so that comparisons can be made.
This type of design is common & involves measuring the DV over time or across trials.
Single-Subject Designs
They all include at least 2 phases: 1. At least 1 Baseline Phase(A) - No Tx 2. 1 Treatment (B) Phase They involve measuring the DV at regular intervals during each phase of study. (This controls for Maturational Effects; threats to Internal Validity)
AB Design
Includes: 1. A single Baseline (A) Phase (No Tx) 2. A single Treatment (B) Phase.
Reversal (Withdrawal) Designs (ABA, ABAB, etc.)
Extended the AB design by including a minimum of: 1. 2 Baseline (A) Phases (No Tx) 2. one Treatment (B) Phase. The Tx is withdrawn (“reversed”) during the 2nd & subsequent baseline phases. If the subject’s performance on the DV follows the predicted pattern (i.e., if it changes in the expected direction after the Tx is applied & withdrawn) a researcher can conclude that changes in the DV are due to the IV rather than to Hx.
Multiple Baseline Design
When it is not ethical to withdraw and effective Tx, and investigator can use this design , which involves sequentially applying the IV (Tx) to 2 or more “baselines” (i.e., either to 2 or more behaviors, settings, or subjects). An Advantage is that, once the IV is applied to a baseline it is not withdrawn during the study.
Descriptive Statistics
Used to describe & summarize the data collected on a variable or the relationship between variables.