bunsen burner: on Flashcards
research design and statistics
null vs alternative hypothesis
null: iv has no impact on dv
alternative: the iv does have an impact on dv
What are internal and external validity?
Standards to judge the usefulness of experimental research results
Define internal validity.
Extent to which accurate conclusions about cause-effect relationships can be drawn
Define external validity.
Extent to which conclusions can be generalized beyond the study’s conditions
What are the five subcategories of external validity?
- Population validity
- Ecological validity
- Temporal validity
- Treatment variation validity
- Outcome validity
What is population validity?
Generalizability of study results to other people in the population
What is ecological validity?
Generalizability of results to other settings or environments
What is temporal validity?
Generalizability of results across time
What is treatment variation validity?
Generalizability of results to variations of the treatment
What is outcome validity?
Generalizability of results to different but related dependent variables
What is a major threat to internal validity?
History
What does history refer to in the context of internal validity?
Events occurring during a study that affect its results
How can history threats be controlled?
Include more than one group and randomly assign participants
What is maturation?
Physical, cognitive, and emotional changes occurring within subjects over time
How can maturation threats be controlled?
Include more than one group and randomly assign participants
What is differential selection?
Differential assignment of subjects to treatment groups affecting results
How can differential selection be controlled?
Randomly assign participants to groups
What is statistical regression?
Threat to internal validity when extreme pretest scores affect results
How can statistical regression be controlled?
Avoid including only extreme scorers or ensure group equivalence
What does testing refer to in the context of internal validity?
Effects of a pretest on responses to a posttest
How can testing threats be controlled?
Not administering a pretest or using the Solomon four-group design
What is instrumentation?
Threat to internal validity when measurement instruments change over time
How can instrumentation threats be controlled?
Ensure instruments don’t change over time
What is differential attrition?
When participants drop out for different reasons affecting study results
Why is differential attrition difficult to control?
Lack of information on how dropouts differ from remaining participants
What is reactivity in the context of external validity?
Reactivity threatens a study’s external validity when participants respond differently to the independent variable than they would normally respond.
Contributing factors include demand characteristics and experimenter expectancy.
What are demand characteristics?
Cues that inform participants of what behavior is expected of them.
These cues can influence how participants respond in a study.
What is experimenter expectancy?
When the experimenter acts in ways that bias the results of the study.
This can involve direct actions affecting participants or indirect actions like inaccurate recording of responses.
What are the best ways to control reactivity?
- Unobtrusive measures
- Deception
- Single-blind technique
- Double-blind technique
The single-blind technique keeps participants unaware of their group assignment, while the double-blind technique keeps both participants and researchers unaware.
What is multiple treatment interference?
Also known as carryover effects and order effects, it occurs when a within-subjects research design is used, affecting how participants respond to different levels of the independent variable.
Example: Administering low, moderate, and high doses of a drug sequentially.
How can multiple treatment interference be controlled?
Using counterbalancing, which involves having different groups of participants receive the different levels of the independent variable in a different order.
The Latin square design is a specific type of counterbalanced design.
What is a selection-treatment interaction?
A threat to external validity when research participants differ from individuals in the population, affecting their response to the independent variable.
Example: Volunteers may be more motivated than non-volunteers.
How can selection-treatment interaction be controlled?
By randomly selecting subjects from the population.
This ensures a more representative sample.
What is a pretest-treatment interaction?
Also known as pretest sensitization, it threatens external validity when taking a pretest affects participants’ responses to the independent variable.
Example: Pretesting on a controversial issue may increase attention to related information during the study.
What is the Solomon four-group design?
A research design used to identify the effects of pretesting on a study’s internal and external validity.
It includes four groups to compare the effects of pretesting on both the independent and dependent variables.
Fill in the blank: The best way to control reactivity is to use _______.
[unobtrusive measures, deception, single-blind technique, double-blind technique]
True or False: Experimenter expectancy can only affect participants directly.
False
It can also involve indirect actions that bias the study’s results.
What are the two main categories of research approaches?
Qualitative and quantitative
What is the primary goal of grounded theory in qualitative research?
To derive a general, abstract theory of a process, action, or interaction grounded in the views of the participants
What are the primary data collection methods used in grounded theory?
Interviews and observations
What is the purpose of phenomenological research?
To gain an in-depth understanding of the ‘lived experience’ of participants
What is the primary source of information in phenomenological research?
In-depth interviews
What does ethnography involve?
Studying participants in their natural culture or setting while they engage in their naturally occurring activities
What is the primary data collection method in ethnography?
Participant observation
What is thematic analysis?
A method for identifying, analysing, and reporting patterns (themes) within the data
Is thematic analysis a stand-alone method?
Yes, but it also sometimes serves as the starting point for other methods
What are the primary sources of information for thematic analysis?
In-depth interviews and focus groups
What is triangulation in qualitative research?
The practice of comparing and combining different sources of evidence to reach a better understanding of the research topic
Why is triangulation used in qualitative research?
To increase the credibility of a study’s data and results
What are the four types of triangulation distinguished by Denzin?
- Methodological triangulation
- Data triangulation
- Investigator triangulation
- Theory triangulation
What does methodological triangulation involve?
Using multiple methods to obtain data, such as interviews, focus groups, observations, questionnaires, and documents
What is data triangulation?
Using the same method to obtain data at different times, in different settings, or from different people
What is investigator triangulation?
Using two or more investigators to collect and analyze data
What is theory triangulation?
Interpreting data using multiple theories, hypotheses, or perspectives
What is quantitative research?
Quantitative research is used to identify and study differences in the amount of behavior and produces data that’s expressed numerically and can be analyzed in a variety of ways.
Reference: Drummond & Murphy-Reyes, 2016, p. 10
What are the three types of quantitative research?
The three types of quantitative research are:
* Descriptive
* Correlational
* Experimental
What is the purpose of descriptive research?
Descriptive research is conducted to measure and describe a variable or set of variables as they exist naturally.
Reference: Gravetter & Forzano, 2016, p. 371
Define correlational research.
Correlational research involves correlating the scores or status of a sample of individuals on two or more variables to determine the magnitude and direction of the relationship between the variables.
In correlational research, what do the terms predictor and criterion refer to?
In correlational research, the independent variable is referred to as the predictor or X variable, and the dependent variable is referred to as the criterion or Y variable.
What is the goal of experimental research?
Experimental research is conducted to determine if there’s a causal relationship between independent and dependent variables.
What distinguishes true experimental research from quasi-experimental research?
True experimental research has more control over study conditions, allowing for greater confidence in causal relationships.
What is the significance of random assignment in true experimental research?
Random assignment helps ensure that groups are equivalent at the beginning of the study.
What are the two phases included in single-subject designs?
Single-subject designs include:
* A baseline (no treatment) phase (A)
* A treatment phase (B)
What is the purpose of measuring the dependent variable multiple times in single-subject designs?
It helps determine if a change in the dependent variable is due to the independent variable or other factors such as maturation.
What does the AB design consist of?
The AB design consists of a single baseline (A) phase and a single treatment (B) phase.
What is a limitation of the AB design?
The AB design does not control for history, as changes in the dependent variable could be due to an unintended event occurring simultaneously.
What characterizes a reversal design?
A reversal design includes at least one additional baseline phase and may be referred to as an ABA or ABAB design.
What is the sequence of phases in an ABAB design?
The sequence is:
* Baseline phase
* Treatment phase
* Withdrawal of treatment (second baseline phase)
* Application of treatment (second treatment phase)
What is the advantage of adding phases in a reversal design?
It helps determine if changes in the dependent variable are due to history rather than the independent variable.
What does the multiple baseline design involve?
The multiple baseline design involves sequentially applying the independent variable across different baselines, which can be different behaviors, tasks, settings, or subjects.
Provide an example of how multiple baseline design can be applied.
A psychologist might evaluate response cost for reducing a child’s undesirable interactions by applying it sequentially to different behaviors such as name calling, hitting, and making obscene gestures.
What is an advantage of the multiple baseline design over reversal designs?
Once the independent variable is applied to a behavior, it does not have to be withdrawn during the course of the study.
What are the three types of group designs in experimental research?
Between-subjects, within-subjects, mixed designs
Define between-subjects design.
A study with two or more groups of subjects, each exposed to a different level of the independent variable
In a study comparing low, moderate, and high doses of an antidepressant, how are subjects assigned in a between-subjects design?
Each group receives a different dose: low, moderate, or high
Define within-subjects design.
Each participant is exposed to some or all levels of the independent variable at different times
What is a single-group within-subjects design?
Evaluates effects by sequentially administering different doses to all subjects
What is a time-series design?
A group version of the single-subject AB design, measuring dependent variables at regular intervals
Define mixed design.
Includes at least two independent variables, one being between-subjects and another being within-subjects
In a mixed design, how is drug dose treated?
As a between-subjects variable
What is a factorial design?
A research design with two or more independent variables
What is the advantage of factorial designs?
Allows obtaining information on main effects and interaction between variables
Define main effect.
The effect of one independent variable on the dependent variable
Define interaction effect.
The combined effect of two or more independent variables on the dependent variable
True or False: A significant interaction effect means main effects should be interpreted with caution.
True
What are the two independent variables in a study evaluating therapy types and drug doses?
Type of therapy and drug dose
How many groups are subjects assigned to in a factorial design with three therapy types and three drug doses?
Nine groups
What might results indicate if cognitive therapy is significantly more effective than the other therapies?
There are main effects for type of therapy
Fill in the blank: The effects of drug dose differ for different types of therapy, indicating an _______.
Interaction between type of therapy and drug dose
What is the main goal of sampling methods in research studies?
To collect data from a sample of individuals selected from the target population
What are the two main categories of sampling methods?
- Probability sampling
- Non-probability sampling
What is probability sampling?
A method that requires the random selection of the sample from the population
What is sampling error?
A problem when a sample is not completely representative of the population due to chance factors
When is sampling error most likely to occur?
When the sample size is small
What is simple random sampling?
A method where all members of the population have an equal chance of being selected
What is systematic random sampling?
Selecting every nth individual from a random list of the population
What is stratified random sampling?
Dividing the population into subgroups and selecting a random sample from each subgroup
What is cluster random sampling?
Randomly selecting a sample of clusters and including all individuals or a random sample from each selected cluster
What is non-probability sampling?
A method where individuals are selected based on non-random criteria
What is sampling bias?
Occurs when participants over- or underrepresent one or more population characteristics
When is non-probability sampling most useful?
For qualitative and exploratory studies
What is convenience sampling?
Including individuals who are easily accessible to the researcher
What is voluntary response sampling?
A sample that consists of individuals who volunteered to participate
What is purposive sampling?
Also known as judgmental sampling, where researchers use their judgment to select appropriate individuals
What is snowball sampling?
A method where initial participants recommend others for inclusion in the study
What does Community-Based Participatory Research (CBPR) aim to achieve?
Combining knowledge with action to achieve social change and improve health outcomes
What is action research?
Research aimed at improving a social problem
What are the roles of community members in CBPR?
Equal partners in all phases of the research
What are the nine core principles of CBPR identified by Israel and colleagues?
- Recognize the community as a unit of identity
- Build on the community’s strengths and resources
- Facilitate equitable, collaborative partnerships
- Foster co-learning and capacity building
- Integrate knowledge generation and intervention
- Focus on relevant public health problems
- View system development as cyclical
- Disseminate research results to all partners
- Understand CBPR is a long-term process
True or False: CBPR only involves adult community members.
False
Fill in the blank: CBPR recognizes the unique _______ that each partner brings to the research process.
[strengths]
What is correlation used for?
To determine the degree of association between two or more variables
What are independent and dependent variables in correlation?
Independent variable is the predictor (X variable); dependent variable is the criterion (Y variable)
What is the range of most correlation coefficients?
-1.0 to +1.0
What does a correlation coefficient of -1.0 indicate?
A perfect negative correlation
What does a correlation coefficient of +1.0 indicate?
A perfect positive correlation
What does the letter ‘r’ symbolize in correlation?
Correlation coefficients
What does a subscript with two different letters (e.g., ‘xy’) mean?
It measures the relationship between two different variables
What does a subscript with two same letters (e.g., ‘xx’) indicate?
It’s a reliability coefficient
What are the three assumptions of most correlation coefficients?
- Linear relationship; 2. Unrestricted range of scores; 3. Homoscedasticity
What happens if the relationship between variables is nonlinear?
The correlation coefficient may underestimate the actual relationship
What is the consequence of having a restricted range of scores?
The correlation coefficient may underestimate the actual relationship
What is homoscedasticity?
The variability of criterion scores is similar for all predictor scores
What is the Pearson r used for?
When both variables are measured on a continuous scale and the relationship is linear
What is an alternative to the Pearson r?
Eta
What is the Spearman rho used for?
When data on both variables are reported as ranks
What does the point biserial correlation coefficient measure?
When one variable is continuous and the other is a true dichotomy
What is an example of a true dichotomy?
Being pregnant or not being pregnant
What does the biserial correlation coefficient measure?
When one variable is continuous and the other is an artificial dichotomy
What is an example of an artificial dichotomy?
Final exam scores categorized as pass and fail
What is the contingency correlation coefficient used for?
When both variables are measured on a nominal scale
What is the Coefficient of Determination?
A measure of shared variability that indicates the amount of variability in one variable explained by another variable.
It is derived by squaring the bivariate correlation coefficient.
If the correlation coefficient between job knowledge and job performance is .70, what percentage of variability in job performance is explained by job knowledge?
49%
This is calculated by squaring the correlation coefficient (.70^2 = .49).
What is regression analysis used for?
To estimate criterion scores using obtained predictor scores.
It produces a regression equation for prediction.
What increases the accuracy of prediction in regression analysis?
The correlation between the predictor and criterion increases.
Higher correlation leads to better predictions.
What are multivariate correlational techniques?
Extensions of bivariate correlation and regression analysis that use two or more predictors to estimate status on one or more criteria.
What is multiple regression?
A technique used to estimate status on a single criterion using two or more predictors measured on a continuous scale.
What are the two forms of multiple regression?
- Simultaneous (standard) multiple regression
- Stepwise multiple regression
What is multicollinearity?
When predictors are highly correlated with one another, leading to redundancy in information.
What is canonical correlation used for?
To estimate status on two or more continuous criteria using two or more continuous predictors.
What is discriminant function analysis?
A technique used to estimate status on a single criterion that’s measured on a nominal scale.
When is logistic regression used?
As an alternative to discriminant function analysis when its assumptions are not met.
What is Structural Equation Modeling (SEM)?
A combination of factor analysis and multiple regression used to test hypotheses about relationships among observed and latent variables.
What are observed variables in SEM?
Variables that are directly observed and measured.
Also known as manifest variables and indicators.
What are latent variables in SEM?
Variables that cannot be directly observed or measured but are inferred from observed variables.
Also known as factors and constructs.
What is the difference between exogenous and endogenous variables in SEM?
- Exogenous variables: Not explained or predicted by other variables in the model.
- Endogenous variables: Explained or predicted by other variables in the model.
What is the first step in the SEM process?
Model specification, which involves specifying a model based on theory and/or previous findings.
What does the path diagram in SEM depict?
The hypothesized relationships among observed and latent variables.
Observed variables are indicated with squares or rectangles, latent variables with circles or ellipses.
What is the fourth step of the SEM process?
Model evaluation, which uses goodness-of-fit index (GFI) or other methods to determine the model’s fit with the data.
What is the fifth step of the SEM process?
Model modification, which involves revising the model to better fit the data if necessary.
What are inferential statistics used for?
To determine if the results of a research study are due to the effects of an independent variable on a dependent variable or to sampling error.
What is a sampling distribution of means?
The distribution of mean scores obtained from a large number of same-sized samples drawn from a population.
What is sampling error?
A type of random error that causes sample means to vary due to chance rather than exposure to the independent variable.
What does the central limit theorem predict about the sampling distribution of means?
- The sampling distribution will increasingly approach a normal shape as the sample size increases.
- The mean of the sampling distribution of means will equal the population mean.
- The standard deviation of the sampling distribution (standard error of means) will equal the population standard deviation divided by the square root of the sample size.
What is the null hypothesis?
A hypothesis indicating that the independent variable does not have an effect on the dependent variable.
What is the alternative hypothesis?
A hypothesis indicating that the independent variable does have an effect on the dependent variable.
What are the two possible correct decisions in hypothesis testing?
- Retain a true null hypothesis.
- Reject a false null hypothesis.
What is a Type I error?
Rejecting a true null hypothesis, concluding that the independent variable has an effect when it does not.
What is the probability of making a Type I error called?
Alpha.
What are common values for alpha?
- .05
- .01
What is a Type II error?
Retaining a false null hypothesis, concluding that the independent variable has no effect when it actually does.
What is the probability of making a Type II error called?
Beta.
How can the probability of a Type II error be reduced?
By increasing statistical power.
What is statistical power?
The ability to reject a false null hypothesis.
How does the size of alpha affect statistical power?
The larger the size of alpha, the greater the power.
Why is alpha usually set at .01 or .05?
To reduce the probability of making a Type I error.
What is the effect of the independent variable on statistical power?
An independent variable is more likely to have a significant effect when of sufficient magnitude and administered for a sufficient length of time.
How does sample size influence statistical power?
The larger the sample, the greater the power.
What type of statistical tests are more powerful?
Parametric tests are more powerful than nonparametric tests.
What types of data can parametric tests be used on?
Interval or ratio data.
What are examples of parametric tests?
- t-test
- Analysis of variance (ANOVA)
What is a nonparametric test used for?
To analyze nominal data.
What is the chi-square test?
A nonparametric test used to analyze nominal data.
What does population homogeneity refer to in statistical power?
The more homogeneous the population regarding status or scores, the easier it will be to detect group differences.
What is frequentist statistics?
An approach to data analysis that involves drawing conclusions about a parameter from data collected in a current study.
What is Bayesian statistics?
An alternative method of analysis that combines information from current data with previous knowledge about a parameter.
How do frequentist and Bayesian statistics differ in their conceptualization of probability?
Frequentist defines probability as frequency of occurrence; Bayesian defines it as degree of belief about occurrence.
What is the correct interpretation of a 95% frequentist confidence interval?
95% of the confidence intervals would contain the true population mean if the study were replicated.
What does a 95% Bayesian credibility interval indicate?
There is a 95% chance that the true population mean is within the interval limits.
What is Bayes’ theorem used for in Bayesian analysis?
To combine previous knowledge (prior) with current data (likelihood function) to derive updated knowledge (posterior).
What is a prior in Bayesian analysis?
The probability distribution for a parameter before collecting new data.
What is a likelihood function?
The probability distribution derived from data collected in the current study.
What is the posterior in Bayesian analysis?
The updated probability distribution for the parameter obtained by synthesizing the prior distribution and likelihood function.
What are the advantages of Bayesian statistics?
- Combines previous knowledge with current data
- Allows direct testing of research hypotheses
What is a major criticism of Bayesian statistics?
The subjectivity of the prior.
What software is mentioned for conducting Bayesian hypothesis testing?
JASP.
True or False: Bayesian statistics provides a single, well-defined method for choosing a prior.
False.
What are the two types of inferential statistical tests?
Nonparametric and parametric tests
Nonparametric tests analyze nominal and ordinal data, while parametric tests analyze interval and ratio data.
What type of data do nonparametric tests analyze?
Nominal and ordinal data
Examples include the chi-square test.
What type of data do parametric tests analyze?
Interval and ratio data
Examples include the t-test and analysis of variance.
What assumptions must be met to use parametric tests?
Data must be normally distributed and variances must be homogeneous
Homogeneity of variances means that the variances for different groups are similar.
When are nonparametric tests typically used?
When parametric assumptions are violated and group sizes are small and unequal
This allows for analysis when the data does not meet the requirements for parametric tests.
What is the first factor to consider when choosing an inferential statistical test?
Scale of measurement of the data
This narrows the choices for appropriate statistical tests.
What are the two types of chi-square tests?
Single-sample chi-square test and multiple-sample chi-square test
The single-sample test is used for one variable, while the multiple-sample test is for two or more variables.
What does the single-sample chi-square test analyze?
Data from a descriptive study that includes only one variable
This test is also known as the chi-square goodness-of-fit test.
What does the multiple-sample chi-square test analyze?
Data from a descriptive study with two or more variables or an experimental study with independent and dependent variables
It is used when variables cannot be clearly identified as independent or dependent.
What is the purpose of the Student’s t-test?
To compare two means when one independent variable has two levels and one dependent variable is measured on an interval or ratio scale
Example: Comparing mean exam scores of two different workshops.
How many types of t-tests are there?
Three types
They are the t-test for a single sample, t-test for unrelated samples, and t-test for related samples.
What is the one-way analysis of variance (ANOVA) used for?
To compare one independent variable with more than two levels against one dependent variable measured on an interval or ratio scale
Example: Comparing effects of different therapies on depressive symptoms.
What is the F-ratio in one-way ANOVA?
A ratio that compares variability due to treatment effects plus error to variability due to error only
It helps determine if the independent variable has an effect on the dependent variable.
What is the difference between experimentwise error rate and familywise error rate?
Experimentwise error rate refers to the Type I error rate for all analyses, while familywise error rate refers to the rate for a subgroup of analyses
Some authors distinguish between the two in terms of their application.
What are the main forms of analysis of variance to be familiar with?
Factorial ANOVA, mixed ANOVA, randomized block ANOVA, ANCOVA, MANOVA, and trend analysis
Each has specific applications based on study design.
What does the factorial ANOVA analyze?
More than one independent variable and their interaction effects
It produces separate F-ratios for each independent variable’s main effects.
What is the purpose of ANCOVA?
To control the effects of an extraneous variable on a dependent variable by statistically removing its effects
The extraneous variable is referred to as the covariate.
What is the multivariate analysis of variance (MANOVA) used for?
When a study includes one or more independent variables and two or more dependent variables measured on an interval or ratio scale
It allows for analyzing multiple dependent variables simultaneously.
Fill in the blank: The one-way ANOVA produces an _______.
F-ratio
True or False: The t-test can be used when comparing means from three or more groups.
False
The one-way ANOVA is more appropriate for three or more groups.
What does a statistically significant F-ratio indicate in an analysis of variance?
At least one group is significantly different from another group.
What are planned comparisons also known as?
Planned contrasts and a priori tests.
When are planned comparisons designated?
Before the data is collected.
What is an example of a study hypothesis mentioned in the text?
Adding instructor-led study sessions to lectures will improve final exam scores.
List the four teaching methods evaluated in the psychology professor’s study.
- Lectures only (L)
- Lectures with peer-led study sessions (LP)
- Lectures with instructor-led in-person study sessions (LIP)
- Lectures with instructor-led Zoom study sessions (LIZ)
What method does the psychology professor use to compare the means of groups L and LIP?
Two t-tests.
What are post hoc tests also known as?
A posteriori tests.
What does a significant F-ratio from an ANOVA indicate?
At least one teaching method differs significantly from another.
What does the Bonferroni procedure aim to control?
The experimentwise error rate.
How is the adjusted alpha level calculated using the Bonferroni procedure?
Divide alpha by the total number of statistical tests.
What are frequently used post hoc tests mentioned in the text?
- Tukey’s honestly significant difference (HSD) test
- Scheffe test
- Newman-Keuls test
What does practical significance refer to?
The magnitude of the effects of an intervention.
What is Cohen’s d used to measure?
Effect size.
How is Cohen’s d calculated?
Divide the mean difference between the groups by the pooled standard deviation.
What does a Cohen’s d of .50 indicate?
The treatment group’s mean is one-half standard deviation above the control group’s mean.
What are the guidelines for interpreting Cohen’s d?
- d < .2: small effect
- .2 ≤ d < .8: medium effect
- d ≥ .8: large effect
What is clinical significance?
The importance or meaningfulness of the effects of an intervention.
What is the Jacobson-Truax method used for?
Evaluating the clinical significance of an intervention.
What is the first step in the Jacobson-Truax method?
Calculate a reliable change index (RCI).
How is the reliable change index (RCI) calculated?
Subtract the pretest score from the posttest score and divide by the standard error of the difference.
What RCI value indicates a reliable change?
Greater than +1.96 or less than -1.96.
What does the second step of the Jacobson-Truax method involve?
Identifying the test cutoff score that distinguishes between dysfunctional and functional behavior.
How can the cutoff score be determined?
Calculate the score midway between the mean score for dysfunctional and functional populations.
What classifications can an individual be assigned based on the Jacobson-Truax method?
- Recovered
- Improved
- Unchanged/indeterminate
- Deteriorated
What is the randomized block ANOVA used for?
The randomized block ANOVA is used to control the effects of an extraneous variable on a dependent variable by including it as independent variable so that its main and interaction effects on the dependent variable can be determined.
