Research Flashcards
quasi-experiment
uses preexisting groups
the IV cannot be altered (ex: gender or ethnicity)
you cannot state that the IV caused the DV
an example of a quasi-experimental study: ex post facto
regression
or statistical regression
extremely high and low scores will regress toward the mean if the measure is given again
internal validity
whether the DVs were truly influenced by the experimental DVs or whether other factors had an impact (confounding factors/contaminating variables/extraneous variables)
external validity
generalizability
parsimony
the best explanation is the easiest and least complex
Occam’s Razor
synonymous with parsimony
interpret the results in the simplest manner
no matter the study, there will be flaws. minimize the worst ones
(car windshield sticker, bubbles)
all correlational research is said to be ____
confounded
1 periodical for research
APA’s Journal of Counseling Psychology
basic research
conducted to advance our understanding of theory
applied research
conducted to advance out understanding of how theories, skills, and techniques can be used in terms of practical application
AKA action research or experience-near research
IV vs DV
I manipulate the IV
DV is the Data our outcome
casual-comparative design
a true experiment except for the fact that groups were not randomly assigned; so you didn’t truly control the IV
data gleaned from the casual comparative can be analyzed with a test of significance (e.g. t-test or ANOVA) just like any true experiment
DV must be ____ measured
that which is directly measured
ex: you hypothesize that biofeedback will reduce anxiety and increase test scores. the DV is test scores because you’re not measuring anxiety in the experiment.
You need ___ participants for a true experiment
30
15 in control group, 15 in experimental group
Surveys need ___ people with a response rate of at least ____.
100 people
50-75%
organismic variable
a variable the researcher cannot control/manipulate, yet exists such as height, weight, gender
AKA status variable
___ pioneered hypothesis testing
R. A. Fisher
The null hypothesis states that
the IV does not affect the DV
experimental hypothesis
your hunch, that the IV does affect the DV
AKA affirmative hypothesis
t-test
used to determine if a significant difference between 2 means exists (on one variable)
find the “critical t” in a table
if your t value is above it, then you reject the null
if your t is below it, then you accept the null
between-subjects design vs. within-subjects design
between-subjects - uses different subjects for each condition
within-subjects - same subjects are studied (e.g., get a pre- and post-test after administering the IV)
parameter
a value obtained from a population
vs. statistic - a value drawn from a sample
P represents
probability or the level of significance
AKA alpha level
in research, P should be set at ___
P = .05 or lower
(e.g. .01, .001)
P = significance level
.05 might be referred to as “95% confidence interval”
A significance level of .05 means
the differences observed would occur via chance only 5 times out of 100
the probability of committing a Type I or alpha error is .05
the 95% confidence interval means
P = .05
the differences observed would occur via chance only 5 times out of 100
alpha error
AKA Type I error
When the researcher rejects the null when it is true
The probability of committing a Type I error is the level of significance
beta error
AKA Type II error
When the researcher accepts the null hypothesis when it is false
power of a statistical test =
1 - beta
power means the test’s ability to correctly reject the null hypothesis
think of Type I and II errors like
a seesaw
the risk of committing one error goes up when the other goes down
how to lower the risk of chance/error factors
increase the sample size
differences revealed via large samples are more likely to be genuine than differences revealed using a small sample size
ANOVA
use to compare the means of > 2 groups (as opposed to t-test) on one variable
the resulting value is an F-value
consult the F-value
if your F-value is higher, reject the null
if your F-value is lower, accept the null
positively skewed
mode < median < mean
negatively skewed
mode > median > mean
kurtosis
peakedness of a distribution
kurtosis - types
leptokurtic - peaked
mesokurtic - normal
platykurtic - no peak, flat
dependent t-test
t-test where 2 similar groups are matched in some meaningful way, or the same group is tested twice
independent t-test
comparing 2 independent groups (that are usually assigned randomly) on one variable
independent groups might also be called unmatched/uncorrelated
a test for more than one IV and more than 2 groups
factorial ANOVA
Ex: if two treatments CBT and interpersonal therapy [IPT] are compared for effectiveness on males and females and different treatments were significantly more
effective with different genders—for example, CBT worked significantly better for males than females while IPT worked significantly better for females than males).
a test for more than one IV, more than 2 groups, where you want to control one of the IV’s
analysis of covariance
ANCOVA
e.g., examining the relationship between household income and work satisfaction, with gender as a covariate—that is, the statistical effects of gender are removed from the
analysis to control for any effects gender might have on work satisfaction
a test for more than one IV, more than 2 groups, and more than one DV
MANOVA (multiple)
a test for more than one IV, more than 2 groups, and more than one DV, controlling for one IV
MANCOVA (multiple)
parametric statistics
rely strictly on interval and ratio data
parametric tests are used when the following assumptions are met:
1. Data for the dependent variable(s) are approximately normally distributed.
2. Samples were randomly selected and/or assigned.
3. An interval or ratio scale of measurement was used for each of the variables involved in the study.
ex: t-test, ANOVA, factorial ANOVA, ANCOVA, MANOVA, MANCOVA
non-parametric statistics
used when we can’t make assumptions about distribution of true scores in the population like we can when we use parametric statistics
suggested when nominal or ordinal data are involved or when interval or ratio data are not distributed normally (i.e., are skewed).
ex: chi-square, Mann-Whitney U test, Kolmogorov-Smirnov Z procedure, Kruskal-Wallis test, Wilcoxon’s signed-ranks test, Friedman’s rank test
the non-parametric version of the ANOVA
Kruska-Wallis test
(so this will be an extension of the Mann-Whitney U-Test when there are more than 2 groups, just like the ANOVA expands the t-test)
the non-parametric version of the t-test when means are correlated
Wilcoxon’s signed-ranks test
want to test whether 2 “co-related” means differ - WilCOxon, “co”
synonymous with the dependent t-test
the non-parametric version of the t-test when means are uncorrelated
Mann-Whitney U-Test
want to test whether 2 Un-correlated means differ (U-Test, Un-correlated)
uses ordinal data instead of interval or ratio data
synonymous with the independent t-test
non-parametric version of Pearson’s r
Spearman correlation or Kendall’s tau
chi-square test
nonparametric
used with two or more categorical or
nominal variables, where each variable contains at least two categories that MUST be mutually exclusive. All scores must be independent
examines whether obtained frequencies differ significantly from expected frequencies
ex: the decision to terminate counseling (yes, no) and the gender of the professional counselor (male, female). A chi-square
would test whether the tallies for the decision to quit counseling by gender of counselor are significantly different from those expected in the population.
Kolmogorov-Smirnov Z procedure
nonparametric
use in place of Mann-Whitney U-Test when the sample size < 25
Friedman’s rank-test
Similar to Wilcoxon’s signed-
ranks test in that it is designed for repeated measures; also can be with more than two groups
one-way vs two-way ANOVA
one-way tests on IV
two-way tests two IVs
when it’s more than one DV, it’s a MANOVA
r
indicates the degree or magnitude of relationship between 2 variables
correlation coefficient
positivism
objective truth exists and can only be understood if directly observable
post-positivism
truth can only be approximated
constructivism
AKA interpretivism
there are multiple realities / truths
critical/idealogical paradigm
researchers take a proactive role in confronting social structure and conditions facing oppressed or underprivileged groups
biseral correlation
one variable is continuous, and the other is dichotomous
ex: correlate CPCE score to NCC status
phi-coefficient correlation
when both variables are dichotomous
ex: NCC status and CCMCH status
single-subject research designs
SSRDs
a type of within-series TRUE experimental design
can be with one person or a small group
often assess the effectiveness of programs for specific clients
AB (like post-test only)
ABC, etc
Spearman rho
a correlation coefficient for ordinal data (rank-order)
rhO - Ordinal data
bivariate regression
how well scores from an independent (predictor) variable predict the dependent variable (criterion variable)
multiple regression
AKA multivariate
more than one predictor variable (IV)
logistic depression
the dependent variable is dichotomous, may be similar to a bivariate or multiple regression
effect size
the measure of the strength of the relationship between two variables in the population
There is an effect size for each variable in a study
the effect size is the ____ in a meta-analysis
effect size
ABCD model for developing program objectives
A = audience (affected individuals) B = behavior (expected action or attitude) C = conditions (context in which behavior will occur) D = description (concrete performance criteria)
four major components of program evaluation
Needs assessment
Process evaluation
Outcome evaluation
Efficiency analysis (do gains outweigh costs?)
Inductive analysis
the data allow notions of a phenomena (theory) to emerge
infers conclusions from data
qualitative research
specific to general
Deductive analysis
starts with theory
confirms a hypothesis
quantitative research
general to specific
focus groups include ____ members
include 6-12 members
participatory action research (PAR)
focuses on change of the participants and the researcher as a result of qualitative inquiry; goals are emancipation and transformation
ex: working with a community agency and its clients to move toward improving the agency
ethnography
qualitative research
studies the culture of a group or system
consensual qualitative research (CQR)
combines phenomenology and grounded theory
researchers select participants who are very knowledgable about a topic
some hope of generalizing to a larger population
researchers reflect their own experiences when developing the interview questions
emphasis on POWER in all aspects — power is shared among researchers and participants
grounded theory
qualitative research; inductive approach
purpose is to generate theory that is grounded in data from participants’ perspectives on a particular phenomena
phenomenology
used to discover or describe the meaning or essense of participants’ lived experiences with the goal of understanding individual and collective human experiences for various phenomena
non-experimental designs (AKA quasi-experimental)
exploratory and descriptive
no intervention
no variable or conditions are manipulated
goal is to observe and outline the properties of a variable
types of quasi-experimental research designs
descriptive design (1 variable), comparative design (>1 group, 1 variable) correlational design (relationship between 2 variables) ex post facto (AKA casual comparative; "after the fact" nothing was manipulated)
coefficient of determination
shared variance between 2 variables
square the correlation coefficient r
e.g. r = 0.5
0.5 x 0.5 = 0.25
ex post facto design
assessing whether one or more pre-existing conditions (unmanipulated IVs) possibly caused differences in groups
quasi-experimental study
conducted after the fact
randomization and manipulation cannot be achieved
purposive sampling
used in qualitative research to get information-rich cases that allow for maximum depth and detail about a phenomena
AKA purposeful sampling
trustworthiness (definition and 4 components)
the validity or truthfulness of a qualitative study
4 components:
credibility - is it believable
transferability
dependability - consistency over time and potential future researchers
confirmability - what you found genuinely represents participants’ views
experimental research designs
involve manipulating conditions and variables
random assignment of groups is usually necessary
can be a single group
action research
done by counselors in an effort to improve their own practice or organizational efficiency
cross-sectional research
examines different groups or cohorts at a particular period in time, with differences in experience being compared
AKA synchronic method
threats to external validity of a study
novelty affect - an exciting new treatment
experimenter effect
history by treatment effect (can’t truly replicate a study at another time period bc of particular events that happened in the world the previous time it was implemented)
measurement of the dependent variable (type of measurement may affect results)
time of measurement by treatment affect (when you administer a posttest may influence the results)
Hawthorne effect
presence of the investigator affects participant responses
AKA reactivity
Hawthorn factory - production went up just because they knew they were being watched
threats to internal validity of a study
history
selection (lack of random assignment)
statistical regression
practice effects (memory effects) on multiple testing situations
instrumentation (changes in instrument - paper and pencil vs computer)
attrition
maturation (any changes in the participant over time can affect the DV)
diffusion of treatment (effects of an intervention in one group are felt by another - kids talk about sex ed to another group)
experimenter affect (Hawthorne)
subject effects (pick up demand characteristics from the researcher or setting that motivate them in certain ways)
demand characteristics
cues that research participants may pick up on from the researcher or research setting that motivate them in certain ways
messes with internal validity of a study
stratified random sampling vs. cluster
stratified random - population is divided into subgroups, you choose randomly from the subgroups (quota sampling is this but without random selection of the participants from within the subgroups)
cluster - counselor identifies existing subgroups and not individual participants - least representative sample compared to other types of sampling
multi-stage sampling
ID a. cluster then choose from within
e.g. randomly select 60 schools, then randomly select 10 classes from within
common in cluster sampling to provide better selection controls
Common Rule
45 CFR 46
research that uses human subjects must have their studies approved by an IRB (if receiving federal funding)
N
number of people being studied
when a single-subject research design (SSRD) is with just one person, it may be called
idographic study
single case investigation
case study
participant-observer model
researcher participates in the study while making observations about what transpired
ABA research model
ABA: A = measure baseline; B = intervention is implemented; A = outcome is examined via a new baseline
multiple-baseline design
when a researcher employs more than one target behavior
an ABAB design is utilized to…
better rule out extraneous variables
If removal of a treatment variable could be harmful to a subject, you should use a __ design
AB
Pearson r
correlation coefficient
used for interval or Ratio (Pearson ‘R’) data
___% of scores are within 1 SD of the mean
___% of scores are within 2 SD
___% of scores are within 3 SD
68%
95%
99.7%
The larger the ____, the greater dispersion or spread of scores from the mean
range
when a distribution has a lot of extreme scores (skewed), use the X rather than the X.
use the median rather than the mean
factorial designs include 2+ ___
IVs
because several experimental values are investigated and interactions can be noted
Solomon four-group design
helps you determine if your results are influenced by pretesting
two control groups, only one experimental group and one control group are pretested. the other groups are just post-tested
y-axis AKA
ordinate
scale for the dependent variable
x-axis AKA
abscissa
scale for the independent variable?
inclusive range
highest score - lowest score + 1
regular range - no +1
John Henry Effect
happens when participants strive to prove that an experimental treatment that could harm their livelihood isn’t that effective
AKA compensatory rivalry of a comparison group
threat to internal validity
compensatory equalization
when the control group lowers their performance because they have been denied the experimental treatment
AKA resentful demoralization of the comparison group
threat to internal validity
the range increases with ____
sample size
z-scores are also called ____
standard scores, because they are the same as standard deviations
ETS or CEEB score
Educational Testing Service or College Entrance Examination Board scores range from 200-800 mean = 500 SD = 100
you can __ and __ using interval scales, but you cannot __ or __
you can add and subtract
but not multiply and divide
IQ tests are a kind of ____ measurement
interval (no true 0; a score of 0 does not mean 0 intelligence)
Rosenthal effect
AKA experimenter expectancy effect
experimenter’s beliefs about a person may cause them to treat that person in a special way so that they confirm expectations
(e.g., teachers and favorite students)
what ANCOVA really does
tests the means of 2+ groups after the random samples are adjusted to eliminate average differences
can be referred to as an “adjusted averages” procedure
interquartile range
score distance between the 25th percentile and the 75th percentile
75th percentile - 25th percentile
longitudinal research might also be called
diachronic method
as opposed to synchronic method (w/ cross-sectional design)
non-directional experimental hypothesis
vs
directional experimental hypothesis
two-tailed t-test (IQ is different in the other group)
one-tailed t-test (IQ is higher in the other group)
if an ANOVA yields a significant F value, name 3 tests you can use to determine significant differences between group means
Duncan’s multiple-range
Tukey’s test
Scheffe’s test
multiple treatment interference
when a subject gets more than one treatment and you can’t tell which modality caused the change
counterbalancing
switching the order in which stimuli are presented to a subject ina study; used to control for the fact that order can affect outcome
Pygmalion effect
experimenter falls in love with their own hypothesis and the experiment becomes a self-fulfilling prophecy
AKA Rosenthal / experimentor effect
quota sampling
population is divided into subgroups, you choose from the subgroups (stratified random but without random selection of the participants from within the subgroups)
non-probability sampling
horizontal sampling
researcher selects subjects from a single SES group
vertical sampling
people from 2+ SES groups are utilized
systematic sampling
selection of every nth (i.e., 5th) subject
probability vs non-probability sampling
probability - everyone has an equal chance of being included (random selection)
non-probability - samples of convenience
