Study Guide Flashcards
What is internal validity?
Answers the research question
provides evidence by controlling variance enough to provide a clear picture of the relationship between the IV & DV.
Must eliminate alternative explanations
What is external validity?
focuses on the extent to which a researcher is justified in describing the impact of one variable on another or for concluding that an observed relationship is casual.
Degree that generalization can be made or transferred outside of the confines of the study.
What is history?
Events that may have occurred between treatment sessions. An extraneous variable that occurs “outside” the experiment
long term studies are more likely to be “contaminated”
What is maturation?
change in subjects’ themselves that cannot be controlled by the experimenter (e.g., age, biological, psychological)
this plays a role in long-term studies.
What is reactive pretest?
the effect of taking a test may have on scores achieved on subsequent administration of the same test.
Improvement may exist without Tx–based solely on test practice
An issue for studies using pre-test and post-test designs
What is instrumentation?
changes in the calibration of a measuring instrument or changes in the observers or scorers used may produce changes in the obtained measurements
faulty, inadequate, or changing calibration of the equipment
rating scales, questionnaire, attitude inventories, and standardized language tests
What is statistical regression?
phenomenon in which subjects who are selected on the basis of atypically low or high scores change on a subsequent test so that their scores are now somewhat better or somewhat poorer than they were originally
What is differential subject selection?
Differences in subjects in experimental and control groups may account for the treatment effects rather than the treatment itself (ex: age, IQ, sex)
What is subject randomization?
the random assignment of subjects to experimental and control group
What is attrition?
experimental mortality, the differential loss of subjects between experimental and control groups
(ex: incomplete surveys, death)
What is interaction of factors?
possible interaction effects of two or three jeopardizing threats
What is credibility?
when the interpretation fits the data and is true to the participants, the conclusions may be considered credible.
what is researcher bias?
researcher must be reflexive about his or her own voice or perspective
What is researcher reactivity?
design must account for the possible influence of the researcher on the participant’s behavior
What is subject selection?
degree to which the subjects chosen for a study are representative of the population the researcher wishes to generalize
What is generalizability?
the extent to which populations, settings, treatment variables, and measurement variables can be generalized
What is interactive pretest?
subjects exposed to a pretest may react to an experimental treatment in a way that is different from people who have not been exposed to the pretest. The effect of treatment may be demonstrated only for subjects who are tested just before treatment.
What are reactive arrangements?
problem generalizing to other settigs
What is the hawthorne effect?
The mere fact of being observed experimentally can influence the behavior of those being observed
What is multiple treatment inference?
This threat concerns the degree to which various parts of a multiple treatment interact with each other in determining subject’s performance on the DV. When more than one treatment is administered.
What is transferability?
the extent in which qualitative findings are externally valid. The ability to apply the results of research in one context to another
What is Pilot research?
during initial stages of experimental design this is conducted on a small number of subjects not meant for publication or to provide data that supplement another investigation, it is only done to assess feasibility.
Describe Internal Validity
It answers the Research Question
Provides evidence by controlling variance enough to provide a clear picture of the relationship between the IV & DV
Results must reflect object reality
Must be certain the change in the DV is caused by the experimental Tx and not by factors that could mimic the effect of Tx
Must eliminate alternate explanations
the fewer (alternate explanations) the greater the internal validity!
Note: The fewer reasons you can come up with why something has changed=GOOD!
What are threats to validity?
History
Maturation
Pretesting–“reactive testing”
instrumentation
statistical regression
Different subject selection
Attrition!
Credibility
What are two threats to qualitative designs that are often conceptualized as alternative explanations or as rival hypothesis, therefore rendering the results meaningless and uninterpretable?
researcher bias
researcher reactivity
In quantitative studies, describe external validity
ability of a study to extend its conclusions from the specific environment to the other individuals with similar characteristics
What may weaken the internal validity?
efforts to extend the generalizability of results
What do threats do?
limit the degree to which internally valid results can be generalized
What are the different types of group research design?
- between subject design
- within-subject design
- mixed design
What is between subject design?
- compare the average behavior of one group of participants to the average of another group of participants
- IV applied to experimental group but not the control group
- need equivalence of both groups
- random assignment–best for large groups
- subject matching–match members of two groups
(see block matching worksheet–last pg. of notes)
Describe a within-subject design
- compare the average behavior of a group of subjects in two different conditions
- want to see how you do on magic beans & how you do on hot cocoa (two different conditions) same study & same group
- performance of the same participants is compared in different conditions (i.e.:longitudinal studies)
- all conditions should be equivalent except for application of the various levels of IVs
- have to assure that changes in DV are attributed to IV rather than extraneous variables
Describe a mixed design
- both types of design
- study 1 IV with a between subject comparison and the other IV with a within-subject comparison
- study the effects of an IV on a DV w/two different types of subjects
- EX: experimental group receives Tx 1 & Tx 2==control group gets no therapy–comparison between groups & within experimental group
What are threats to internal validity?
- related to sequence of conditions
- participation in an earlier condition affects performance in a subsequent condition
Describe sequencing effect
- Order effect–change in the performance in the beginning to the end of the task (maybe better because of increased practice or worse b/c f fatigue)
- carryover effect–influence of treatment condition of the performance on the next condition (temporary or permanent)
How can you reduce sequencing effects?
control of overcome sequencing effect =randomize your sequence
-sequence randomizing
counterbalancing
what is sequence randomizing?
random distribution of tx conditions to participants in a random order
what is counterbalancing?
- arrange all possible sequences of tx then randomly assign participants to each sequence
- differences in performances attributable to the sequencing of tx conditions can be measured
Describe the Single Subject Design
- applied to one subject or a small number of subjects studied as separate individuals
- each subject participates in all conditions
What are the types of single-subject designs?
Withdrawal
treatment-withdrawal
reversal design
Multiple baseline designs
changing criterion designs
Describe the withdrawal or reversal designs?
ABA design–see figure 4.2
Baseline segment A=nonintervention
Treatment segment B=intervention
Withdrawal of treatment still taking data=A again
Controls for confounding variables
well controlled design
Describe the ABAB design
powerful way of plotting data points
withdrawal period where there is no therapy but we keep taking data points
A=baseline
B=tx
A=withdrawal/ no therapy
B=reinstatement
What does the methods section describe?
describes who or what has been studied
materials that were employed
how those materials were used to obtain useful data
What are 3 major components of the methods section?
- Subjects/participants
- materials
- procedures: describes what is down to the participants with the materials.
What is IRB
institutional review board
studies must gain approval of IRB
sometimes referred to as “human subjects” but includes animals as well
In protection of subjects/participants, what are three basic “ethical” principles?
- respect for persons-honor individual’s decisions
- Beneficence-maximize benefit and minimize harm
- Justice-Selection of individuals be fair and unbiased
What are the types of validity? (the holy trinity of validity)
content validity
construct validity
criterion validity
What is content validity?
logical examination of the content of the test items to see how well they sample the behavior or characteristics to be measured
What is construct validity?
the degree to which the instrument measures reflect a trait or construct
statistical means help define constructs
What is criterion validity?
established by empirical examination of how well the measure correlates with some outside validating criteria–cut off score (Ie: GPA for graduate school)
What are the measures of central tendency?
mean
median
mode
Describe what the mean is, and how to find it
the average of the numbers
add up all the umbers, then divid by how many numbers there are
Describe what the median is and how to find it
the middle number
place the #s in value order and find the middle #
Describe the mode
the # that appears the most
What are the different types of data, and which level is the highest level you can achieve?
nominal
ordinal
interval
ratio
**Ratio is the highest level of data you can achieve
Describe nominal data
If you need more explanation see slide 13, lecture 3
Mutually exclusive (i.e.: pass/fail)
Set of data; values/observations belongings to it can be assigned a code in the form of a number where the #s are simply labels
can count, NOT MEASURE or ORDER
ie: males could be coded as 0, females as 1
married=Y single=N
Describe ordinal data
If you need more explanation see slide 14, lecture 3
Set of data; values observations belong to it can be RANKED or have a rating scale attached
mutually exclusive and ranking (i.e.: mild, moderate, and severe)
natural order
ie: a ranking scale of 5 indicates more enjoyment than a ranking scale of 4
Describe Interval data
If you need more explanation see slide 15, lecture 3
Scale of measurement where the distance between any 2 adjacent units of measurement is the same, but the zero point is arbitrary
can be added or subtracted
could include height of tides, measurement of longitude, standard scores on test (no true zero)
Describe Ratio data
If you need more explanation see slide 16, lecture 3
interval data with a natural zero point
(distances and area)
weight is a ratio variable
physical measurements
ANYTHING THAT COULD HAVE A ZERO
Check study guide of irregular shapes of the bell curve
…
Describe the Null Hypothesis
(Ho)
States that there is no difference between groups or relationship among variables
presumed valid until researcher proves otherwise
Reject or fail to reject
What does Failing to reject the null hypothesis mean?
not statistically significant
probably due to the sampling error or chance
What does rejecting the null hypothesis mean?
statistically significant
due to some determining factor or condition, other than chance
Give an example of a null hypothesis scenario
Hypothesis: CILT-II will significantly improve the speech production of persons with confluent aphasia
Null Hypothesis: CILT-II will have no measurable effect on the speech production of PWAs.
Hypothesis: H1-tomato plants exhibit a higher rate of growth when planted in compost rather than in soil
Null Hypothesis (Ho): Tomato plants do not exhibit a higher rate of growth when planted in compost rather than soil
How can you disprove the null hypothesis?
by showing the unlikelihood of occurrence of chance relationships or differences
At the end of the study, what are our two options in terms of the null hypothesis?
we can either reject the null or fail to reject the null
What are the two types of errors?
Type I
Type II
What is a Type I error?
to reject the Ho of no differences when it is true; to conclude falsely that a difference exists in the data when in fact it doesn’t
What is a Type II error?
To failed to reject the Null Hypothesis of no differences when it is false; to conclude falsely that a difference does not exist in the data when in fact it does
True or false/ accept or reject
What are some examples of Type I and Type II errors?
you are a researcher and fail to detect an actual statistically significant difference in your study. you reported that you failed to reject the null–this is a type II error
You are a doctoral student and have spent 2 years on your dissertation. At the end o f your research, you ran your statistical analysis over and over until the results that you were hoping for finally “popped up” you happily decided that your data reflected statistical significance and you rejected the null–Type I error
Check The graph of Hypothesis testing outcomes…
….
What are the key steps and terms in conducting a study?
- Develop research question
- hypothesis
- Literature review
- Choose research design
- experiment (test hypothesis)
- Analyze data
- Write report
- conclusion –Reject or not reject null hypothesis
